Home Devs Hack iPhone API for True Augmented Reality

Devs Hack iPhone API for True Augmented Reality

An international team of computer scientists has created software that lets anyone perform on-the-fly analysis of live streaming video on the iPhone. Used alongside existing methods of displaying data on top of the camera’s view, this new functionality signals a fundamental change in the kinds of Augmented Reality (AR) that iPhone developers can create. Existing AR apps, like Yelp, Layar, Wikitude and others display data on top of a camera’s view but don’t actually analyze what the camera sees. This new development changes that.

The iPhone has a private API for analysis of live-streaming video but developers’ requests that it be made accessible haven’t been granted by Apple. The new software opening up access to that API was made freely available to anyone this morning by the team that built it.

The Visual Media Lab at Ben Gurion University in collaboration with HIT Lab NZ wrote the code in question and unveiled it along with video demonstrations at the AR-specialist blog Games Alfresco today. The unveiling comes just days before the International Symposium on Mixed and Augmented Reality in Orlando, Florida.

In a demonstration video the team showed how software built on top of the now-exposed API could look at a 2D image drawn on paper and render the image in 3D. Then the 3D rendering is subjected to a physics simulation.

This is of course just one use-case. Video AR-enabled software could do almost anything in direct response to the actual images seen through the iPhone’s camera view, in real time. Image processing locally will be easier and faster than comparison with a large number of related images, something that would likely require some connection to the cloud, but these are early days.

GamesAlfresco author Ori Inbar calls this the dawn of an era of “user-generated Augmented Reality.”

For the first time ever, the core code necessary for real augmented reality (“real” here means precise alignment of graphics overlaid on real life objects) on iPhone 3.0 is available to the public

How will Apple respond? That’s a big question; the company has had an ambivalent relationship with the emerging field of Augmented Reality so far and exercises infamously obtuse control over applications distributed through its app store.

For now the code is being distributed for its creators by Ori Inbar, whose email address to request it is available at the conclusion of his coverage on GamesAlfresco.

The possibilities here are huge. While location-based AR is clumsy at best so far, due to the imprecise nature of GPS and mapping data, these kinds of object-centric AR tied to the actual viewed world open up a whole new world of potential developments. Let’s see what you’ve got, AR devs of the world!

About ReadWrite’s Editorial Process

The ReadWrite Editorial policy involves closely monitoring the tech industry for major developments, new product launches, AI breakthroughs, video game releases and other newsworthy events. Editors assign relevant stories to staff writers or freelance contributors with expertise in each particular topic area. Before publication, articles go through a rigorous round of editing for accuracy, clarity, and to ensure adherence to ReadWrite's style guidelines.

Get the biggest tech headlines of the day delivered to your inbox

    By signing up, you agree to our Terms and Privacy Policy. Unsubscribe anytime.

    Tech News

    Explore the latest in tech with our Tech News. We cut through the noise for concise, relevant updates, keeping you informed about the rapidly evolving tech landscape with curated content that separates signal from noise.

    In-Depth Tech Stories

    Explore tech impact in In-Depth Stories. Narrative data journalism offers comprehensive analyses, revealing stories behind data. Understand industry trends for a deeper perspective on tech's intricate relationships with society.

    Expert Reviews

    Empower decisions with Expert Reviews, merging industry expertise and insightful analysis. Delve into tech intricacies, get the best deals, and stay ahead with our trustworthy guide to navigating the ever-changing tech market.