a trillion sensors connected to the Web, which will result in an explosion of online data. How will this mass of new and mostly real-time data be processed and analyzed? Will current data analytics software be able to cope? The short answer is, no it won't. New types of analytics software will be required, together with much more powerful computers.In my previous few articles, I've explored the potential impact of sensors on the Internet. Soon there will be
During my visit to HP Labs last month, I sat down with Meichun Hsu - director of the Intelligent Information Management Lab at Hewlett Packard - to discuss this issue. Hsu has been researching new real-time, sensor analytics solutions for the coming Internet of Things era.
Hsu told me that sensor data, along with other Web data such as feeds, should be managed as enterprise assets. There is a large amount of data, she said, and it needs "integrated management."
She also thinks that analytics and the compute function should be brought "closer to the data." By this she means breaking down the walls between data and analytics, barriers such as CRM systems and similar structures traditionally used in enterprise data management. She noted that enterprise information management and business intelligence systems have not changed much in the last 20 years.
ReadWriteWeb Real-Time Web Coverage:
- Google Tests Mind Reading, AKA Full Page Previews
- Semantic Startup Evri Goes Mobile
- Twitter Goes Live with Real-Time Streaming API
- Rapid Innovation: The Philosophy of Betaworks CEO John Borthwick
- Bing Now Recommends Interesting Twitter Users
- Live Matrix Launches a TV Guide for the Scheduled Web
- Seesmic Desktop Goes Beyond Twitter: Becomes a Platform for All Things Real Time
- Google Docs Gets a Taste of Wave with Collaborative Highlighting
In terms of actual analytics, we will see changes such as new methods of visualizing data and correlating patterns in it.
Processing needs to evolve too. The emerging era of the Web, said Meichun Hsu, will bring not just an increasing amount of data - but an increased rate of data. HP expects the data rate to increase by a factor of about 10,000 times in the near future, due largely to sensors. This will require massively parallel processors to analyze, Hsu said. HP is looking for a 10,000 times increase in the ability to crunch data.
As for some of the potential commercial applications that might arise out of this advanced data analytics, Hsu talked about the impact on retail as one example. A consumer might be able to scan a tag on a product and not only get information back about the product, but combine that with data about who you are and your preferences. Retailers will be able to do advanced targeting, real-time micro campaigns, and more. This type of functionality requires real-time analytics on a massive scale.
Let us know in the comments about the new forms of data analytics that you are seeing and how you think data analytics will evolve.