Last night, the folks over at Infochimps, an online data marketplace – or the “Amazon of data,” as they like to call it – were celebrating the launch of their new website and the thousands of new API calls it contains. These API calls are like plug-and-play bits of code developers can insert into their applications, so they can focus on other things like the overall design, the user interface and the features.
After only a few minutes of looking through the types of dataset queries Infochimps has on hand, I was excited about the kinds of new applications they could enable. How about an app that finds everyone on Twitter with a particular keyword in their bio and then lets you follow them or add them to a Twitter list, for example? Or an app that helps you identify the most influential Twitter users? These were just two of the API calls I saw in action last night, but there are thousands more (and they’re not all Twitter-related, of course). If you’re developing a new data-dependant application, Infochimps is a valuable resource you should check out.
Using the API Explorer
On the infochimps website, recently redesigned for ease of use, a new API Explorer lets anyone search for these sorts of API calls from a simple search box. The API calls are identified in the search results with a blue badge reading “API,” so as to separate them from the other results (Infochimps’ downloadable datasets).
To give you an idea of the kind of things you can find, check out the examples below:
- Twitter People Search: Look for the people who mention a certain word in their Twitter bio.
- The 100 million word British National Corpus: Great if you are looking for commonly used words.
- IP to Demographic: Understand what kind of visitors come to your site based on their IP addresses.
- Wikipedia Articles Abstract Search: Look up a term in Wikipedia and get general descriptions that contain that word.
- Qwerly: Connect a person’s identity across social networks
How Data Nerds Like to Party:
Making Me a Spreadsheet of Everyone with a Particular Keyword in Their Twitter Profile
The Twitter People Search is my favorite of these new API calls (so far), since I’ve been looking for a great application that would do this exact same thing. Unfortunately, the apps I’ve found so far haven’t been as well-designed as I’d like. They’ve also been desktop applications, when I’d prefer a mobile one. But with this Infochimps API call, a developer could help me (oh please!) by creating a mobile application that could perfect the process of finding Twitter users by words in their profile descriptions and then provide me with an improved interface for following those people en masse – or, even better, building me a custom Twitter list containing those Twitter accounts. Maybe I could even customize that list in advance of its creation, by unchecking the accounts I don’t want included? Maybe you’re a developer with an even better idea than that?
At this point, anything would be better than Twitter’s default user interface which forces you to manually add people to lists, painstakingly, one-by-one. I don’t have time for that. There needs to be a better tool. And if you’re a developer, you could be the one to build it.
How to Build a Spreadsheet of Twitter Accounts, Using the People Search API Call
In the meantime, with help from Infochimps’ co-founder Flip Kromer, I have a tool that helps me create a spreadsheet of everyone on Twitter who has a particular keyword in their Twitter profile.By particular keyword, I’m referring to the search query. For example, if I wanted to find everyone on Twitter talking about mobile chipsets, I might create queries that looked for the keyword “Qualcomm” or “NVIDIA.”
Here’s how you can do the same thing:
- Go to the TwPeopleSearch API call here.
- In the box that says “q,” erase the word “infochimps” and insert the word you want to search for. (Note that you can search other parts of the profile here, too, but that’s not my focus right now).
- Hit the “Submit” button
- The data shows up on the screen. That was easy!
What about if you want that data in a spreadsheet, you say?
Then just run this query in Terminal on your Mac, also available at the easy-to-remember bit.ly URL: bit.ly/json2tsv. (Thanks, Flip!)
You’ll just have to edit the bits in bold, as shown below, because those will be specific to your needs. This sample query looks for keyword “qualcomm,” for example. You’ll also need to insert your own API key (provided to you when you sign up for Infochimps on their website) and the “File to Save” part is whatever you want your spreadsheet to be called when it’s created.
curl “http://api.infochimps.com/social/network/tw/search/people_search?q=qualcomm&apikey=YOURAPIKEY” | ruby -rubygems -e ‘require “json” ; $_ = $stdin.read.gsub(%r{^george_api_explorer((.*))$}, “\1”) ; rows = JSON.load($_); ks= rows[“results”].first.keys ; ks.sort! ; rows[“results”].each{|row| puts row.values_at(*ks).map{|el| el.gsub(%r{[“,]+},””) }.join(“,”) } ‘ > FILE_TO_SAVE_IN.csv
Have fun!
