Big Data Is Just For Big Companies - And Other BS

2013 is going to be a 12 full months of hype surrounding the Big Data craze, with tantalizing tales of e-commerce analytics that will move beyond predicting pregnancies to doing something really cool: like figuring out the Last Twinkie Ever Bought.

As the hype continues, smug enterprise IT departments will whip out their thick wallets and plunk down serious coin for multi-cluster hybrid cloud Hadoop systems that can figure out the worldwide effects of a butterfly wings' flapping before brunch and knock out the Ultimate Answer to Life, The Universe, and Everything before afternoon tea rolls past.

Meanwhile, poor and tattered small business IT managers will look enviously on this feast of big data as they huddle out in the cold, shivering as they wrap themselves in tiny spreadsheets and Quickbook invoices.

Big Data Doesn't Have To Be That Way

Its easy to get that impression when thinking about Big Data, but the reality can be far different: While enterprise-level datasets can be costly and difficult to manage, there's no reason analytical and visualization techniques can't be applied to small- to medium-sized business (SMB) datasets - on an SMB budget.

Plugging small businesses into big data is not a new idea (see Intuit CEO: Big Data Can Be The "Great Equalizer"). Perhaps the most popular approach to solving the problem includes "Big-Data-as-a-Service" options.

There are two large obstacles to taking on big data methodologies, according to Amit Bendov, CEO of Tel Aviv-based SiSense. The first is breaking past the barrier of cost. SAP HANA appliances, which Bendov holds up as an example of what he's talking about, can handle about half a terabyte of data, with a price tag of $500,000. That may work for the enterprise, but is nowhere near realistic at the small business level.

The second obstacle is the notion of complexity: Right now, the impression is that you have to have full-blown data-scientist talent on the payroll in order to figure out your own data. That's hogwash: "I know my business very well," Bendov said. "I don't need statistical languages to figure out what the data means. I just need to see the data."

SiSense Tries To Solve The Problem

Bendov's company is in the business of making that happen. Working with a customer's existing data stores, SiSense will pull the data into a columnar database that's either hosted by SiSense or held locally by the company. Customers can then use whatever tools with which they're familiar - from Excel spreadsheets to SQL-based databases - to mine the data for the information they're seeking.

General-purpose hosted data services like SiSense or sector-oriented vendors like LeisureLink in the hospitality industry represent a new front in the Big Data cycle: taking the expense and complexity down several levels so smaller businesses can get at least some of the benefits of data analytics without the pain.

It's not a bad idea: by acting as the middle man to handle the hard parts of Big Data management, these vendors are essentially running Big Data co-ops.

But there are risks with this approach. While SiSense has a local-hosted option, some vendors do not… so you'll need to decide where your comfort zone is with storing critical company data in the cloud. You will also need to decide if your data actually has enough value to recoup whatever costs you will have to outlay if you work with these vendors. That's a bit of a Catch-22 for many smaller companies, since you may not know your data's value until you can pull it together and mine it for useful information.

The best option is to shop around with various vendors, perhaps carrying in a subset of your data and seeing what can be done with it - and at what cost. Perhaps most importantly, this can also give you a sense of how much of a learning curve will be involved in getting up to speed with a new service.

Big Data options are out there for businesses of all sizes, just be sure you understand the benefits versus the risks.

 

Image courtesy of Shutterstock.