It is one thing to say that apps built in HTML5 function across every device on the market. It is another thing to run those apps.
Development studio Spaceport.io used its PerfMarks II testing platform to determine which browsers and devices run HTML5 best. PerfMarks II tests “attempt to translate, scale, or rotate as many objects as possible on various devices while maintaining 30 frames per second or better,” according to the studio's report.
30 frames per second is considered the benchmark for whether an application runs smoothly. For some apps, like high-quality games, 60 frames per second is ideal. At this point, no mobile device or mobile browser performs anywhere near the level of desktop/laptop computers. On average, mobile browsers perform 889 times slower than desktop browsers. The top mobile devices, both Apple and Android, are between six and 10 times slower.
As we noted last week, Google’s Chrome Beta for Android is one of the most HTML5-capable browsers in the mobile ecosystem when tested by Rng.io. The difference between Rng.io and Spaceport.io’s PerfMarks II is that Rng.io only tests for browser capabilities while PerfMarks II tests actual performance across a variety of techniques.
PerfMarks II uses three main categories to test HTML5 rendering: canvas, WebGL and CSS. For the sake of clarity, here is a screenshot of exactly what PerfMarks II is testing:
Spaceport.io only tested the top browsers and devices. That makes sense. While there is something to be said for testing Microsoft’s mobile version of Internet Explorer and Research In Motion’s BlackBerry PlayBook OS 2.0, as long as the operating systems they run on represent less than 10% of combined mobile market share, the results are not really relevant.
Spaceport.io tested mobile Safari and Google Chrome Beta on the Samsung Galaxy S II, Galaxy Nexus (running variations of Android 4.0.x), iPhone 4 and iPhone 4S (running iOS 5.1.x). For desktop browsers, it tested Google Chrome and Safari (running WebKit r116996, akin to Safari 5.1.5) on a MacBook Pro running OS X 10.7.3.
The desktop results served as benchmarks. It should be noted that WebGL is not currently supported on iOS or Chrome Beta for Android, so those results can be tossed immediately. The best results for iOS were with canvasDrawImageFullClear, only slowing 6.24 times on an iPhone 4S in comparison with a laptop (an iPhone 4 was 45.95 times slower). Two CSS techniques worked best with Chrome Beta: css3dImg and css3Dbackground; both were more than 11 times slower than a desktop test. The Chrome Beta results suggest that canvas should pretty much be avoided, even though it works great on the desktop versions of both Chrome and Safari.
“It’s easy to see an HTML5 demo on a laptop and get excited about how HTML5 will revolutionize mobile application development, but with the slowdown factors ranging from 6 to thousands, it’s very important to take such demos with a grain of salt,” the report stated. “The problem is that on desktop and laptop computers, the Flash Player plugin is effectively universal and provides a high-performance, cross-platform experience. Thus, HTML5 is not so desperately needed on the desktop web. It’s the mobile platforms, where Flash is not available, that so desperately need HTML5 and sadly, it is exactly these platforms where performance is still lacking.”
So, once again, it all comes down to Flash… or the lack thereof. The results highlight how big an opportunity Adobe had in creating Flash on mobile devices, an opportunity it bungled and eventually scrapped. Adobe has stopped making new versions of Flash for mobile devices, and the last iteration will be Flash 11 for Android 4.0. Adobe could have had a proprietary system that controlled nearly all the media on iOS and Android. The company is now making HTML5 tools, an area with a lot of competition coming from the likes of appMobi, Sencha, Brightcove, Netbiscuits and others.
"They ignored it until it was too late," said Carlos Icaza, former Flash manager at Adobe and cofounder of Ansca Mobile, to ReadWriteWeb in 2011. "They were not looking out for the best interest of developers."
So now the open source community takes on HTML5 and attempts to drive better performance. Sencha and appMobi are leaders in this area and do significant testing and release product suites designed to increase performance. While those systems are on the cutting edge of mobile HTML5, right now the cutting edge is just not sharp enough.