Written by Alex Iskold and edited by Richard MacManus
In
an article in the January 1st 2007 issue of NYTimes, reporter Miguel Helft writes
about the race in Silicon Valley to beat Google. Certainly the future of search has been
much talked about lately. Last year Read/WriteWeb had a number of big posts on this
topic, including Emre’s Search 2.0 and
my post about vertical
search. We have also profiled many search players, including Retrevo, Hakia, Quintura, Pluggd, Microsoft
Live Search, Snap and ChaCha.
Since we have been following the battle closely, we are excited to see the coverage in
New York Times – which signals that the search space has heated up enough to be worthy of
attention by a tech-savvy mainstream audience.
The reporter Miguel Helft works through the main issues of the race. He explains that
a huge amount of venture money has been poured into the search space – not necessarily to
beat Google, but perhaps to get little slices of the gargantuan advertising market.
However, Helft concludes that even though many companies are trying to catch up, he says
it’s not clear whether they can. Google is not a monopoly like Microsoft, he
states, but still it seems unfeasible to unseat to Google at this point.
Obviously, we all have the same feeling about Google’s dominant position – but why? To
understand, we need to segment Google’s competitors and their approaches. In this post we
will use Emre’s
categorization to see why we all intuitively feel that Google is going to remain the
king of search.
The categorization of competitive technologies
There are two classical business cases for a new company: a better way or a cheaper
way. In the case of the search market, cheaper simply does not apply. So the companies in
the current search space are racing towards different shades of ‘better’. In his post, Emre
outlined three major approaches to outdoing Google:
- Better Technology – the results are of better quality/relevancy
- Better UI – the results are presented in a better way
- Vertical/Semantic Search – a combination of better technology and better UI
Better Technology
Google’s automated algorithm, based on
the original PageRank idea, is a phenomenon. The contenders to overthrow Google are
deploying two radically different strategies: 1) natural language processing/AI
techniques and 2) power of people.
In the AI camp, Hakia and Powerset are building technologies that aim to deliver
better search results by better understanding what you mean. Both companies have received
a lot of attention in the press lately and we also reviewed
Hakia here recently. While it is clear that there are benefits in understanding
natural language, what is not clear is how much difference it makes in the quality of the
results.
Based on what we have seen so far, it is difficult to see how these companies can beat
Google. Firstly, being able to enter the query using natural language is already allowed
by Google, so this is not a competitive difference. It must then be the actual results
that are vastly better. Now that is really difficult to imagine. Somewhat better maybe,
but vastly different? Unlikely.
People powered search is interesting, since it brings to the game what
computers lack the most – common sense. del.icio.us is
not typically thought of as a search engine, yet it is probably the best example of
people power in action. The great thing about del.icio.us is that we get high-quality
filtered results. The not-so-great thing is that the results are limited to what most
people have found popular. Arguably this is the intent, but in some cases it will also be
the limitation.
Another problem is that del.icio.us has only been used so far by techies, so the
results in non-technical areas (medicine for example) are not going to be great. A
different approach is taken by ChaCha, which literally has people help you find things.
It is interesting idea, however in our
review in December we pointed out a few issues – including high monthly burn rate
(since search guides have to be paid) as well as potential scalability problems.
While it is clear that solutions like del.icio.us are superior to static web
directories, it seems that the search algorithm is going to be difficult to beat by power
of people. The speed and completeness of the Google algorithm is going to be particularly
out of reach when it comes to people powered solutions.
Better UI
Google’s motto is simplicity. This idea
runs through their entire product stack and is closely guarded by the company. This is
why many competitors are racing to present their search results in a better way.
Snap and Microsoft
Image Search are showing Previews to improve the user experience.
Previews are interesting and useful. I particularly like the Microsoft Image Search UI,
which is a big improvement in how image search is presented. Unfortunately both of these
technologies have to improve on relevancy and even quantity of results. Also what works
well for images, does not work as well for search results – since it is difficult to
determine if it’s worth drilling in, purely based on the preview.
Next we look at Clustering solutions, that essentially attempt to do
what vertical search engines do – but across the board and using the entire result set.
Clusty is very impressive, because it manages to add
a lot of value without changing how we think about search results. Along with the search
results, the tab with clusters allows the user to refine the query – to basically get to
the answer in no more than 2 steps.
The second engine in this category is SearchMash, which is a Google property and
apparently their playground. What is striking about SearchMash is that its ascetic look –
so typical of Google – is actually creating a richer way of viewing the search results.
It’s done using Ajax and has a mode that displays nothing but the headlines of the
results. I found this mode to be surprisingly useful. There are also knobs for slicing
the results along verticals like Blogs, News and Images – which are very straightforward.
Both of these sites (Clusty and SearchMash) seem very useful and capable of reaching
mainstream acceptance.
In the last sub-category of better UI, we find Quintura and Kartoo applying innovative
Visualization technologies to search. Quintura deploys a graph/tag-cloud
like approach to present search result clusters. In a way, the idea is very similar to
Clusty – just done in a different way. Kartoo deploys a graph/heatmap-like approach to do
exactly the same cluster partitioning. The problem with both of these approaches is that
they are too complex, in my view, to make it to mainstream. Even though I have a
background in graph visualizations, I do not find either one obvious or intuitive. It is
unlikely that in this incarnation, visual search would be widely accessible.
A lot of these UI improvements seem neat and interesting, but none of them is going to
be able to beat Google. The main reason is that it is straightforward to replicate them.
Of course legally there can’t be exact clones, but in this case close enough will also be
good enough. None of these technologies has barrier to entries for Google, so if any of
them gets enough market share Google is simply going to roll out an upgrade. Note that
their toying at SearchMesh.com can already hurt some current vertical search engine
providers.
Vertical Search
There has been a lot written about vertical search and for a good reason. These
technologies aim to unseat Google by having both better technology and better UI
and focusing on a specific vertical. There are a lot of players going after Blogs,
Classifieds, Electronics, Health, Jobs and Travel engines – because of both popularity
and monetization opportunities.
Consider this simple example. If you wanted to find blog posts about web 2.0, would
you use plain Google or Google Blog search? Clearly even Google itself recognizes the
utility of having specialized, focused search – if not for every vertical, at least for
some like Blogs and Music. So there is no doubt that vertical search makes sense and it
is here to stay. The key question is how much of the Search market can vertical search
engines bite off?
Actually, it could be a lot. The categories that we named are all money
generating/short-pass to transaction kind of categories, which are highly sought after by
advertisers. So what exactly does that mean? Given that Google already has specialized
search for Blogs and Music, it seems like it is going to be just a matter of time before
Google ventures into other categories. Depending on how soon this happens, it might mean
a lot of trouble for current players (think Google Blog Search surpassing Technorati).
Only those who have managed to build strong brand awareness (like Technorati) will
survive.
Conclusion
So overall, even though there is a lot of activity in the space, it seems like Google
will remain the search king for the foreseeable future. Various approaches will have
different degrees of success in seizing bits of the market, but to make a serious dent
will require time, flawless execution, big marketing dollars and, of course, a better
technology. This is not a trivial combination of things.
Given that the first three are difficult points, but doable, we still need to
understand what better technology means. In my mind it is the one that delivers relevant
results faster. From that point of view, another promising contender (mentioned by Emre)
is personalized search. With this technology, search results are going to be
organized not by PageRank but by your personal interests. It is likely that a combination
of a vertical search and personalized search is going to deliver fundamentally better
results than Google, so that might have a chance. However, as we pointed out with the
other technologies. Google is not going to sleep through this.
Let us know what you think and tell us what alternative search engine is your
favorite. Also vote for the search technology you think stands the best chance of beating Google, in our poll:
