Apple’s acquisition of mobile assistant Siri and its partnership with speech recognition leader Nuance Communications (the latter confirmed by references found in code), appear to be coming together in the launch of a new feature called “Assistant,” to appear in the forthcoming update to iOS 5, Apple’s mobile operating system.
According to leaked information, it appears that the smart technology found in the Siri iPhone application will now be fully baked into the operating system itself. With Nuance’s ability to understand natural language queries, iOS 5 will have it all – voice navigation, voice control and voice assistance – allowing users to go beyond simple search and basic actions. When Siri’s technology is fully integrated, users will be able to direct their iPhone to actually “do” things, too.
New “Assistant” Feature Uncovered
The news of the Assistant feature comes by way of 9to5Mac.com, an Apple-tracking blog, which reported this over the weekend. A source told the outlet that the development of these features was incomplete, however, and may not even be finished by the time the next iPhone ships.
In the screenshots provided by the source, you can see a new toggle button which will let a user switch the Assistance technology off or on. Why would anyone want such a useful feature off? Well, according to 9to5Mac, the assistant also extracts details from the device, including its location, contact information and music metadata, in order to perform its actions. In addition, the speech recognition technology will involve crowd-sourced data, which is sent to Apple in order to improve the service over time. Privacy-minded folks may not want to share their data with Apple, no matter how well it is secured.
In a follow-up post, 9to5Mac also found additional references to Nuance in the iOS 5 SDK (software development kit). The SDK reveals that the technology will be activated by a button, and may display a screen which shows the speech translated into text, similar to what Google’s Android operating system has today.
Siri Knows What You Mean. Now Your iPhone Will, Too
Many in the industry expected that Apple was going to make an announcement about its speech recognition and virtual assistance technology at its annual developer conference, WWDC. When the news was a no-show, the theory was that, perhaps deals between Nuance and Apple had not yet been finalized. Given that no official Apple announcement about either Siri or Nuance has been made, everything about when, if or how the technology will arrive in a final product is speculation.
What we do know about the technology itself comes from earlier conversations we had with Norman Winarsky, VP of SRI Ventures, the R&D outfit that made Siri possible. In June, we talked to him about what sort of scenarios deep integration of Siri’s core technology could allow for on an iPhone.
Winarsky explained that with Siri, the engine doesn’t just understand words, but is smart enough to also understand the “intent” of a sentence. So, for example, it can understand a complicated request like “find me a great hotel in Palo Alto that has a pool and a fitness center,” or “find me a good Italian restaurant that serves spaghetti bolognese.” It can even be directed to perform various actions, like “remind me to call Sam tomorrow at 3 PM.“
An Assistance Layer on Top of Apps?
In the past, Siri had relationships with several services like Fandago, OpenTable, Hotels.com and Menus.com, to provide its answers. Incidentally, many those same services also have mobile applications in iTunes, which begs the question: how far will the Siri integration go? Will iOS 5 not just provide answers, but also launch apps in the process of doing so? That would certainly go a long way in helping users deal with the growing problem of “app overload.” With some 400,000 iOS applications now housed in the app store, the trouble is not finding “an app for that,” but knowing which one works best for a given need.
With deep Siri integration, we imagine the service could not just direct you to the apps you have installed on your phone, but even suggest apps that could help you in the future. The idea isn’t without precedent – after all, Microsoft is doing something similar with “App Connect,” an upcoming app recommendation feature found in the “Mango” update for Windows Phone. But Siri wouldn’t just be an iteration on Genuis for Apps, it could possibly be the launching point for performing actions within apps.
Although Apple now owns Siri, SRI International is continuing to develop new virtual personal assistance technologies that could also come to your smartphone someday. For example, it’s developing technologies that speak back to the user in order to further refine queries, bots that perform reasoning, act on your personal preferences, and much more.
What Apple may choose to do with the Siri technology is still unknown, but one thing is clear: if the Siri integration goes live, the smartphone is about to get much smarter.
Image credits: Assistant: 9to5Mac; Siri
