Mozilla's Rob Sayre:
"We believe that the benchmarks used in Kraken are better in terms of reflecting realistic workloads for pushing the edge of browser performance forward. These are the things that people are saying are too slow to do with open Web technologies today, and we want to have benchmarks that reflect progress against making these near-future apps universally available."
In his announcement, Sayre notes that synthetic benchmarks tend to be the least useful, while testing real or modified applications will give developers the most realistic view of how fast their browsers really are.
It remains to be seen if Kraken will be able to take the position that Sunspider, Google's V8, and Mozilla's own Dromaeo currently hold. As with all vendor-sponsored benchmarks, there will surely be some concern among Mozilla's competitors that the benchmark is geared towards favoring the organization's own products. Kraken's code will be released under an open-source license, however, so it will likely be easy for the team to counter these arguments.