Google today announced that it is now indexing the amazing amount of 1 trillion unique URLs. Google’s first index in 1998 only had 26 million pages and by 2000 that number had jumped to 1 billion. Today, the Google index is growing by several billion pages per day alone. Not too long ago, Google used to have a counter on the front page of its search engine, displaying the number of sites in the index, but they dropped this information from the site around 2005.
When there was still real competition between search engines in the late 90s, the size of a search engine’s index was one of the main methods of comparing search providers. Today, the number of pages in any given search engine’s index has dropped out of our collective conscience – and that might be a good thing, as the focus has shifted towards returning relevant search results over the ability to return the most results.
That, after all, was the real advance that Google brought to the search engine market. Early search engines like Altavista, Excite, or HotBot simply weren’t able to return the most useful results to users.
However, we are also getting close to reaching a new crossroad again, where even Google’s results are often polluted by spam. Yet, at the same time, the great promise of semantic search engines is still just a promise for now. Given the latest data about the search engine market and the end of the Microsoft/Yahoo negotiations over acquiring the Yahoo search business, Google is pretty much becoming the de-facto standard search engine for most people.
Chances are that anybody who wants to enter this market and compete with Google is simply going to be bought by the search giant, so if anything, Google’s strong position is going to get even stronger in the foreseeable future.
For now, though, the real question about Google’s search index is when it will reach a size of 1 googol…
Photo by Flickr user Mykl Roventine.
