Imagine a tiny little sun, just bursting with heat and light, but trapped inside a hard metal cover with a few holes to let beams of energy stream out from inside. Now imagine there were millions of those little suns, maybe the size of basketballs or tennis balls, all rolling down an assembly line one after another, each with a unique pattern of holes and beams of light streaming out into the world.
That's what Twitter is. Inside every unborn tweet you can find infinite potential - someone will be in a place, with social context and they will say something, anything, and give that potential a form. They will say something and it will be instantly available to anyone in the world who's subscribed. Each tweet has more than 30 fields of metadata under the hood; the value populating each of those fields makes up the unique patterns of holes in the metal cover that lets the light out from inside. A company launched today that lets you control a robot that drills holes in the metal covers trapping the infinite potential of the sun inside.
"One could spend months mining Twitter using @DataSift," said Paul M. Watson,
CEO CTO of stream curation startup Storyful, today on Twitter. "Great balance of usability & power in the CSDL." (Curated Stream Definition Language)
Were one to look into the screaming firehose of hundreds of millions of tweets on Twitter and call out to the robot gods of data sifting, "Give me the tweets by self-proclaimed South Africans, living in Ireland, with positive sentiment and that have been retweeted by data-loving tech investor Roger Ehrenberg!" Were one to dip into that river in search of a sliver like that, which may or may not exist (it does, Watson's fits the bill), then the freshly launched startup Datasift would be the tool one would use to do so. It would cost you pennies, too.
Years in the making, Datasift launched today as the second licensed reseller of tweets. The startup doesn't just resell tweets like yesterday's news, though.
Datasift lets anyone parse the full firehose of Twitter messages with its simple Curated Stream Definition Language and see the resulting flow of messages that fit the criteria described.
You wouldn't likely ask for all the tweets from self-proclaimed South Africans living in Ireland, but you might ask for all the tweets posted by women living in a particular state in the US and using any of a list of keywords. Market research, political monitoring, news reporting, there are all kinds of use cases. Whether the set of would-be customers intersects substantially with the set of people curious enough to think of the right questions to ask the data is a big question.
Anyone can say anything on Twitter, and with Datasift you can ask whether anything is being said. 80%+ of retweets of @justinbieber are from Females, the company says.
Datasift just opened to the public today, so despite its best efforts there are still some kinks that need to be worked out. While the team attended its launch party, the preview rendering functionality stopped working for a little while. The pricing, while explained a number of different ways, still needs more clarification and development, founder Nick Halstead said today on Twitter.
The query and filtering tool isn't as easy to use as the company would like it to be, either. "It is decidedly not easy to use," says recent University of Washington Masters of Science in Information Management graduate and former ReadWriteWeb researcher Emily Cunningham. "I find their UI clumsy. Creating streams is confusing, not intuitive at all, hard to understand."
But the potential here is huge. It's a simple proposition, too: Twitter is now an incredibly rich source of information about all kinds of topics. "I learn about most things through Twitter," DataSift's new CEO Rob Bailey told Vator.tv this week. "I spend more time on that platform than I do on any cable channel . . . I've learned so much more about events like the occupy movement that cable news just wasn't covering . . . and I was able to see that the RIM audience was getting more and more frustrated leading up to the outages they experienced. It's such a powerful news source."
Those are words that many of us can relate to, but Bailey now leads a team of engineers building a tool that aims to make it relatively simple for anyone to create filters to capture the messages about the events that we all catch wind of on Twitter.
Anyone who's willing to pay for some tweets, that is, even at a low low price. Datasift is intended for businesses users.
Can Datasift build a business serving a broad range of business users interested in juggling those spheres of light, filtering by patterns and in turn using that information to create new levels of value? Time will tell. It's a very ambitious undertaking.
Disclosure: The author leads a stealth startup that works in the social media data space as well, more likely a Datasift customer than a competitor though.