"I don't have the data to prove it, but my guess is if you looked at the percent of all pageviews that are generated each month, a much smaller portion exist on the top 10 properties today than in 2000, at the height of the first Internet era."
Essentially Fred's theory is that the Long Tail of Internet content makes up a higher percentage of total Web page views today, than 6 years ago.
I asked Web Analytics firm Compete if they could come up with some data to prove or disprove Fred's hypothesis. Compete kindly provided me with some great data, which in a nutshell disproves Fred's theory. According to Compete's data, the top 10 domains are not shrinking - but proportionally increasing.
Internet has grown by 77% in last 5 years
Looking first at the overall growth of the Internet, according to Compete's data the Internet has grown by 77% in the last five years to over 5 million unique domains. (Note: This count includes misspelled and unhosted domains people accidentally find themselves at).
Top 10 account for larger % of total PV, not smaller
Contrary to Fred's theory and the larger theory of the Long Tail, the top internet properties are accounting for a larger percentage of total pageviews across the web. According to Compete, currently the Top 10 domains account for 40% of the total pageviews on the internet - a 29% increase over the last five years.
Social Networks to blame
The driver of this Top Domain growth can be summed up in two words: "social networks". If you were to remove MySpace and Facebook from consideration in 2006 (also removing their pageviews from the total) Top Domains would only account for 33% of total pageviews - basically on par with 2001.
The following chart shows the difference in top 10 domains over the past 5 years.
Note on The Long Tail
The Long Tail is a theory that has been popularized over the past year by Wired editor Chris Anderson - with his book and blog of the same name. The easiest way to define The Long Tail is to present this graph (c/o Wikipedia):
The long tail is the part colored in yellow.
The usual disclaimers apply about Internet statistics - you would probably get different stats from comScore and Alexa. But the overall trends are of most interest here. It does appear that the top 10 web properties are claiming proportionally more page views in 2006 than in 2001.
The impact of the Long Tail can still be felt strongly on the content creation side, on sites like MySpace and YouTube. So in one sense you could argue that The Long Tail has had an impact on the top 10 domains after all - in terms of who is creating the content. But the fact is that those properties are still owned and operated by big media (News Corp and Google in the above two cases). So in that respect, and according to Compete's stats, the top 10 domains have more reach now than 5 years ago.
Update: The Compete blog has further analysis.