When we reported on Monday that IE9 outperformed other browsers in HTML5 compliance testing, the news was greeted with a mixture of surprise and skepticism.
And rightly so, perhaps, as Philippe Le Hégaret, the person who oversees the W3C‘s work on HTML5, has written a blog post clarifying the results of these tests, urging people to not draw any conclusions about “whether one browser or another is better.”
According to Le Hégaret notes, the 212 tests and the results they may indicate about one browser’s compliance – or lack of – are “way far from significant.” The tests that were reported on Monday only looked at a few specifications, including “attributes”, “audio”, “video”, “canvas”, and “xhtml5.” Other aspects, including web workers and the file API were not tested. And Le Hégaret contends “we’ll need several dozens of thousands of tests to make those results significant.”
Le Hégaret admits that the test report page was misleading, and the page now features a giant red box, clarifying that the tests are a work-in-progress. But he also adds, “the report is still bogus… and the percentages are incorrect,” something that raises questions not just about whether or not IE9 is HTML5 compliant, but about how the W3C is testing and reporting these specifications.
Le Hégaret does call for more community participation in helping the W3C with the testing, something that hopefully help move HTML5 – the proposed standard and the testing process – forward.
