The Semantic Web has been in the making for some time and people think it is nearing maturity.
We have written about this trend extensively, with our two most notable posts being an analysis of the challenges of the classic bottom-up approach and the promise of the new top-down one. Regardless of how the Semantic Web will come about, for it to flourish
it needs to hit the mainstream. There is no way that consumers will appreciate the elegance and mathematical soundness of RDF and OWL. People don’t care about math, they care about utility and even more, about fun. What the Semantic Web needs, then, is a killer app.
Whatever it is, it needs to layer an understanding of semantics on top of a consumer application.
The consumer application needs to be so cool and so viral that people will be open to learning that it is powered
by semantic technologies. In that case, it will be possible to further market applications as Semantic Web apps. Consumers will understand that if one Semantic Web application has potential, so might others. In math, this is called
proof by induction. In marketing this is called creating a market. In any case, it needs to be done.
In this post, we analyze several existing and potential applications of semantic technologies and look for the killer app.
Natual Language Understanding
Since the beginning, the Semantic Web has been associated with Artificial Intelligence. The idea
of representing information in structured form so that computers can “understand it” and then solve complex problems was one
of the keystones of the Semantic Web vision. The problem is that representing billions of existing web documents as RDF is a rather
daunting, if not impossible task. An alternative would be to “teach” computers natural language. If an application could read the page
the way we read it and interpret what it says, the annotations would not be necessary.
Natural language processing has been the Holy Grail of AI for awhile now. However, it is a very difficult problem, because humans
are born with the innate ability to understand language and we learn it not in a vacuum, but in the context of life. Certainly if we could
replicate that with computers, it would be amazing and it would be the killer app. The problem is that this is not on the horizon. The
Semantic Web technologies of today are not able to represent natural language in its entirety, and this is not really even their goal. Even if we could represent each page completely, there is still the matter of interpreting structure into semantics, which is the magic
that our brain does so well and so easily.
Genie In The Bottle
Related to natural language understanding, is another idea that is not on the horizon. John Markoff called it
“the perfect vacation.” I call it the “Genie in the Bottle” to illustrate the impossibility of this.
There is a misunderstanding about the Semantic Web which is floating around, which equates the Semantic Web with ability to solve really hard problems. It is simply not true.
For example, if you go to a new travel agency and ask them to book the perfect vacation for you, the travel agent will not be able
to do it, because she does not know you. In order to find the perfect vacation there needs to be constraints: where you’ve been before,
who you are going with, what you like to do, what is your budget, etc. Finding the “perfect” vacation is not a one shot deal, it is a process,
which leverages iteration and memory.
True, with the Semantic Web the information is structured, but it does not mean that the computer can necessarily solve complex problems.
These are two completely different things. Just because you have a map, does not mean that you know the best way to get from point A to point B. Having a map is necessary, but it is not sufficient, you need the algorithm to find the best path. There is a big difference between asking what is the capital of France and what is the cheapest airfair today to fly from New York to Paris. And the even harder question is: Where should I go on vacation next? Computers are not going to give us an instant, perfect answer to this question anytime soon, if ever. Again, this would be the killer app, it is just not likely to happen.
Semantic Knowledge Databases
So what is realistic and possible today? The first in the list of growing applications
are Semantic Knowledge Databases. The two examples that we will look at here are Freebase and Twine.
While Freebase is focusing on building essentially a semantic equivalent of Wikipedia, and Twine is focused on a personal semantic database,
both are databases, both focus on knowledge management, and both are Wikipedia-like. The advantage of these databases over Wikipedia is that
they represent information in a structured way and support queries. To understand the difference, take a look at the Alicia Keys page
on Freebase and on Wikipedia.
At first glance they are very similar, but Freebase “knows” that Alicia Keys is a blues singer and it then knows other blues singers.
For Wikipedia, blues is just another page, not a music genre. So Freebase can potentially answer a question of listing all blues singers, while Wikipedia can not.
This is certainly interesting but the question is will people care? Can the end consumer tell the difference? Unlikely. Today
Wikipedia contains definitive references on a vast number of topics. Like Google, it is easy to search and find relevant information, and as a result, people
are not likely to be in need of a better Wikipedia. With Twine the situation might prove to be different, because personal knowledge management
is an important problem. The first question is: Are their enough people who want to be efficient in managing personal knowledge? I think the answer is increasingly likely to be “yes.” And the second question is: Does knowing the semantics of knowledge help you build the best application? At the very least Twine has to beat del.icio.us bookmarks and ideally needs to do for personal knowledge management what Highrise is doing for CRM.
But beyond the execution, there is still another problem. For a semantic knowledge base to be the killer app it needs to
ignite imagination and capture people’s hearts and minds. This is not likely to happen. We appreciate libraries, we can not
live without them, but we take them for granted. Knowledge has been commoditized thanks to Google, Wikipedia, and the blogosphere,
and is perceived as abundant and unexciting. For this reason Semantic Databases are not likely to be the killer apps — but they
might become a stepping stone towards one.
Semantic Search
An early candidate for the killer app in the semantic web category was search. First
Hakia and more recently Powerset marketed the idea that a semantic search engine, one that is based on the understanding of natural
language, can beat Google. On top of having the pressure to deliver qualitatively better results,
Semantic Search companies also have to, at least approximately, solve the problem of natural language understanding,
which as we discussed earlier is a very difficult one.
Where things stand right now, it does not look like search is the killer app for semantics. The understanding
of natural language does not seem to give you a noticeable edge in getting better search results. At least in the comparisons that we have performed earlier
there is no major difference. The statistical algorithm deployed by Google is precise and good enough, which is why it has been the clear leader in web search for the past 8 years. To unseat Google will require more than incremental improvement in search, it will likely take a paradigm shift and
the creation of a different web experience. Below, we discuss how “discovery” could possibly take a bite out of the pie, but as of now
Google’s algorithm remains good and strong.
Social Graph
After Tim Bernes-Lee posted his thoughts on the Social Graph, a discussion began on the web in which people wondered if the Social Graph is in fact the Semantic Web. This, however, is a gross misinterpretation of the post. The Social Graph is not the Semantic Web, nor is it
the killer app of the Semantic Web. They are just two separate concepts. The confusion comes from the fact that they both are Mathematical Graphs
or a Network. The underlying structure of both consists of nodes connected by links. Many things
in the nature and society are networks, so it is not surprising that meaning and people fall into this category.
If anything, it is more correct to say that the Social Graph is a subset of the giant, all encompasing Semantic Web. Knowing how people
are connected is important in order to solve the perfect vacation problem. After all, a perfect vacation should be taken together
with perfect friends, right? But jokes aside, the Social Graph is an interesting and important trend for 2008, however, it is not really related to Semantic Web.
Shortcuts
Increasingly, we are seeing a new breed of Semantic Applications, which we generalize as shortcuts.
This category includes SnapShots from Snap, BlueOrganizer and SmartLinks from AdaptiveBlue, Shortcuts from Yahoo!, and In-text search from Lingospot.
What is common between all these technologies is that they leverage the simple semantics of the content to deliver additional information.
In the case of Snap and AdaptiveBlue, the semantics is defined by the URL, while Yahoo! and Lingospot perform text analysis.
Regardless of the method, all of these technologies deliver related information via Ajax popups. That is, they leverage semantics to
pull the information from the web. This is essentially discovery or reverse search. When the user is looking at a book there is a preview
with a brief description and the cover image, when the user encounters a stock symbol he is presented with a stock chart, analysis and
additional links to the company, when the user is looking at a music album there is a play button, and when the user encounters a movie
there is an ability to watch the trailer in place. The shortcuts remove the need to search, instead, the related content from the web comes right into the page.
Today’s shortcut technologies are simple and still in their infancy, but they are among the most successful examples of semantic applications.
However, we can not call them the killer app for several reasons.
First, people perceive them as advertising, which is not the point.
Snap certainly made an early push into ads, but this is not a representation of what these technologies will look like in the future.
Second, in their current implementation, all of these technologies are utilities. For the same reason that people are not going to get
emotional about personal knowledge management, they will not be emotional about shortcuts. Shortcuts will also be taken for granted.
Yet, shortcuts hold the most promise. With a few more iterations these technologies are going to get slicker and more precise.
They will leverage content and micro-context to reduce the amount of search. They will become more personalized based on user
behavior. And once this happens it will be a big deal.
Full Disclosure: Alex Iskold is the founder and CEO of AdaptiveBlue.
Conclusion
We are still waiting for the killer app for Semantic Web, something that can get viral and turn semantics into
a marketing term. Problems like natural language understanding still remain difficult to solve, and the solutions do not appear to be on our horizon right now. It also appears that
a semantic search engine, at least based on the ones we have seen to date, does not have a substantial advantage over Google.
We are seeing the rise of early Semantic Knowledge Databases, but while we expect them to get better and more interesting,
they are more likely to be the stepping stones to the killer app, rather than the app itself.
In the mean time, we are seeing the rise of shortcut technologies, which leverage the basic semantics of the content,
like URL and simple context analysis, to deliver relevant information, links, and media directly into the page. While still very early,
these technologies hold the most promise because they are simple and useful. We expect that the next generation of these technologies
in conjunction with personalization will deliver an interesting alternative to search — contextual discovery. We will discuss
this alternative in more detail in a future post.
Now tell us what you think the killer app for Semantic Web will be? Which of these technologies do you think is the most promising?
