Google and Microsoft are both in talks with Twitter to buy access to the company’s full fire hose of messages and shared links, according to a report this morning from Kara Swisher at AllThingsD. Could these be blockbuster deals that bring the search giants into the world of the real-time web?
In fact, the Twitter fire hose may be less valuable than outsiders might think. We’ve been having extensive conversations with a wide variety of real-time web companies in preparation for the Real-Time Web Summit next week and we’re lead to question the value of Twitter links for two reasons.
First, there aren’t really that many links that get passed through Twitter. The service may be the most visible example of real-time communication but there’s a whole lot more going on around the web in real time outside of Twitter. Twitter may see between 3 and 5 million links shared through it each day. Those are heavily biased towards the relatively small number of active content creators on the site and much of it is spam or republished RSS feeds.
Almost every real-time search company we’ve spoken to has chosen to supplement the indexing power of Twitter with other, larger data sources. OneRiot, for example, says it collects 4X more links from the click-streams that its toolbar users have opted-in to sharing than it does from Twitter. Faroo and Wowd both rely on P2P clients to power indexing far beyond what they can get from Twitter. Implicit behavior offers a lot more data than explicit behavior.
AllVoices uses a whitelist of trusted Twitter users to improve the signal-to-noise ratio and Evri watches trends on Twitter to trigger the building out of their index of the much larger web outside Twitter.
Evri also does some really interesting semantic parsing of the contents of those tweets, adding links as pivot points to the news entities people mention in text. Almost all of the companies we talked to had to do extensive text analysis of Tweets and the pages they linked to. That may not be difficult for Google or Microsoft, but is quick discovery of a few million new links a day challenging enough for them to be worth a big price tag?
You want real-time web links? There are probably already more blog posts being pushed out in real time for free consumption using Pubsubhubbub and RSSCloud than there are links shared on Twitter. As we discussed in our recent post Ten Useful Examples of the Real-Time Web in Action, there’s a lot more going on around the web outside of Twitter. Much as we love Twitter, most people suspect its growth is slowing. The rest of the real-time web is believed to be at the beginning of an explosion of growth that will take over the entire internet – eventually almost all information online may be pushed to those who want it, in real time.
Why else do we question the value of the full fire hose? Business Insider reports today that Twitter is already selling access to startups for up to $3k per month. It’s hard to know how credible that number is but that’s a really low price.
We do know that many startups feel they can get a representative sample of Twitter activity from the company’s free API. Twitter may be having a hard time up-selling to the full feed if what’s already available is good enough and its crown jewels can’t be proven to be that much better.
We love Twitter and with some serious clean-up it may provide big search companies a handy little bundle of high-interest links to advertise against. Is it valuable enough for Google and Microsoft to pay big money for, though? We’re not so sure.
Come join us and some very smart people from throughout the diverse real-time web industry to discuss this and related topics next week at the Real-Time Web Summit.