A few weeks ago, Google announced the beta launch of Caffeine, the company’s next-generation search infrastructure. At that time, Google said that most of the changes in this update were under the hood and that users wouldn’t notice a difference in search results. At its core, Caffeine is basically a major overhaul of the Google File System. There have been some discussions about whether this update will bring any other major changes to page rankings or the importance of certain categories in the search results. Summit Media, a UK-based digital marketing agency, compared search results for 9,000 keywords (PDF) in Caffeine and Google’s default (‘vanilla’) search and, interestingly, didn’t find any major differences between the two.

While the first version of the Google File System obviously scaled quite well, it wasn’t built to support the blended search results Google displays today (images, videos, news, etc.). Even Google’s own Matt Cutts argues that this update won’t have any major effect on Page-Rank and doesn’t constitute a change in Google’s search philosophy and algorithm.

There has been quite some discussion about whether this actually turns out to be true, though. Judging from this new study by Summit Media, there really haven’t been too many changes to the rankings in Caffeine, though the Summit Media team did notice some interesting differences.

There was no major difference between the kinds of categories of sites Caffeine prefers compared to Google’s ‘vanilla’ search. Caffeine, for example, doesn’t give any more weight to price comparison sites than Google’s current search infrastructure. One slight difference that the Summit Media study noticed was that slightly more co.uk domain names now appear in searches that originate from the US. The difference there, though, was only minor.

Difference: News Sites

One area where Summit Media noticed a difference, however, was sites that fall into the ‘Information,’ and ‘News’ categories. Here, at least for generic search terms, Caffeine shows a bias towards more timely information and seems to punish sites that don’t update often and mostly consist of archived material.

Does it Help to Have a Keyword in Your Domain Name?

Summit Media also tested another popular SEO theory: does it help to have your keyword in your domain or page URL? Summit didn’t find a difference between the Caffeine and ‘vanilla’ search results and maybe more importantly, having a keyword in your domain name doesn’t seem to make a real difference when it comes to search rankings. According to Summit, only about 6% of all search results on the first three search results pages actually contain the specific keyword in the URL. That’s a very small number, and, as the report concludes, it is also “a demonstration that SEO needs to be based on actual research – rather than perceptions over browsing a few pages of results.”