“How come your benchmark ranks devices differently than [insert other benchmark here]?” It’s a fair question, and the reason is that each benchmark has its own emphasis and tests different things. When you think about it, it would be unusual if all benchmarks did agree.
To illustrate the phenomenon, consider this excerpt from a recent browser shootout in VentureBeat:
- SunSpider: Edge wins!
- Octane: Edge wins!
- Kraken: Chrome wins!
- JetStream: Edge wins!
- Oort Online: Chrome wins!
- Peacekeeper: Firefox wins!
- WebXPRT: Firefox wins!
- HTML5Test: Chrome wins!
WebXPRT, like the other XPRTs, uses scenarios that model the types of work people do with their devices.
It’s no surprise that the order changes depending on which aspect of the Web experience you emphasize, in much the same way that the most fuel-efficient cars might not be the ones with the best acceleration.
This is a bigger topic than we can deal with in a single blog post, and we’ll examine it more in the future.