Yesterday, we reported that Nielsen Online’s April numbers showed that the number of unique streams on Hulu grew 7.9% since March, though the number of unique users dropped slightly to about 7.4 million. As the New York Times reports this morning, however, Hulu questions these numbers and argues that they grossly underestimate Hulu’s real reach, which comScore, another online measurement firm, pegs at 42 million.
Nielsen’s numbers are pretty close to those we have seen from other measurement firms like Compete (7 million unique visitors for April), though Quantcast, which gets its data directly from a piece of code embedded on Hulu’s site, reports about 26 million. While these other companies might not agree on the exact numbers, though, most publicly available data shows that Hulu’s growth has indeed slowed down in the last two months. In its own press releases, Hulu generally quotes comScore’s numbers.
To gather its data, Nielsen monitors about 200,000 panel members, a technique that is clearly informed by Nielsen’s method for gathering data about TV viewers. Other metrics companies use data from toolbars, ISPs, and other sources, though Quantcast also gives site owners the option to embed a code snippet on their pages that reports data directly to Quantcast (Hulu does so, for example, and so does RWW).
Stats Need Standards
The real problem here, of course, isn’t even about knowing exactly how many people watched videos on Hulu last month (even though we have to admit that this discussion is quite interesting in its own right). Instead, this kerfuffle once again shows how hard it is to correctly estimate usage numbers on the web, especially in the absence of any real standards. As every blogger can easily attest, three different stats programs will give you three different numbers.
We have to take publicly available stats, no matter from which provider, with a grain of salt. In our experience, it is always worth looking at a number of different sources, and while the trends that these services show tend to be relatively trustworthy, the exact numbers are always open for debate.
