Home Amazon Exposes 1 Terabyte of Public Data to Developers

Amazon Exposes 1 Terabyte of Public Data to Developers

Amazon.com changed the retail world. In the process the company built up so much surplus computing power that it started a dirt cheap “computing in the cloud” business that changed the computing world. This week the company’s newest project Public Data Sets on Amazon Web Services began offering more than 1 Terabyte (1000 GB) of fascinating public data for developers to access on the fly through Amazon’s cloud computing service.

We’re talking about an annotated collection of all publicly available DNA sequences, including the Human Genome, huge amounts of chemistry data, machine readable encyclopedic entries about millions of different topics and an entire dump of Wikipedia. US Census data, data from the US Department of Transportation and more. It’s all accessible by web applications in no time at all. What do you think this is going to change?

The company made a blog post last night announcing the availability of four new public data sets.

This includes data from:

  • The Bureau of Transportation Statistics.

  • DBPedia Knowledge Base – which “currently describes more than 2.6 million things including 213,000 people, 328,000 places, 57,000 music albums, 36,000 films, and 20,000 companies.” All in handy semantic markup.

  • The Freebase Data Dump – the giant collaboratively build semantic database on a wide variety of topics, data that high profile startup Metaweb has spent millions of dollars assembling.

  • The entire English section of Wikipedia, dumped into a machine readable format.

  • A number of large genetic and scientific databases.

We counted all the databases up and it passed 1 TB of available data. The company says that accessing this data is “trivial” for developers.

What are developers going to do with this data? We can’t wait to find out. The prospect of mashing up, cross referencing and user interfacing with this amount of data is nearly unfathomable. Really. This data will be leveraged by all kinds of different web applications, for a long time.

You’ve read, or can imagine, the impact that the first Public Libraries had on human culture. Now imagine the opening up of not just this, but other libraries of data, so huge that economies of scale blast the project off beyond any analogy that could be drawn with our everyday experience or historical memories. It won’t just be Amazon that offers up this kind of data – it will be relatively commonplace soon, we imagine.

It will be like a network of libraries – for robots. Robots that go to the library frequently, read very fast and make serious use of what they’ve learned.

Congratulations, Amazon, on passing 1 TB of public data made available. May all our robots of the future please live in peace.

About ReadWrite’s Editorial Process

The ReadWrite Editorial policy involves closely monitoring the tech industry for major developments, new product launches, AI breakthroughs, video game releases and other newsworthy events. Editors assign relevant stories to staff writers or freelance contributors with expertise in each particular topic area. Before publication, articles go through a rigorous round of editing for accuracy, clarity, and to ensure adherence to ReadWrite's style guidelines.

Get the biggest tech headlines of the day delivered to your inbox

    By signing up, you agree to our Terms and Privacy Policy. Unsubscribe anytime.

    Tech News

    Explore the latest in tech with our Tech News. We cut through the noise for concise, relevant updates, keeping you informed about the rapidly evolving tech landscape with curated content that separates signal from noise.

    In-Depth Tech Stories

    Explore tech impact in In-Depth Stories. Narrative data journalism offers comprehensive analyses, revealing stories behind data. Understand industry trends for a deeper perspective on tech's intricate relationships with society.

    Expert Reviews

    Empower decisions with Expert Reviews, merging industry expertise and insightful analysis. Delve into tech intricacies, get the best deals, and stay ahead with our trustworthy guide to navigating the ever-changing tech market.