Guest author Scott Geng is the CTO at cloud management software company Egenera.

It's no secret that the public cloud market has been growing like gangbusters. In fact, a recent Gartner study found spending on public cloud services is growing at more than 28% per year and private cloud spending is three times that of public cloud. That projects total cloud spending in 2016 to hit $240 billion.

Cloud computing (both public and private) will pave the way forward for how companies will deploy new IT services. Lower price points will help those organizations innovate faster, launch new services more quickly, be more responsive to market conditions and evolve their own business models.

Management And Specialization

The focus in the industry over the past few years has been on the core cloud management services of SaaS, PaaS and IaaS. But to truly understand how cloud computing is evolving you have to dive deep below the surface. Two major developments are driving the evolution of cloud: Management and Specialization.

In the management space, innovations like self-service portals have given IT shops and end-users a much-preferred way to request and consume services.

Specialization, meanwhile, is a natural development of any market. A few of the specialized services that will contribute significantly to the adoption of cloud based products and services in 2013 include Desktop-as-a-Service (DaaS), Metal-as-a-Service (MaaS) and DisasterRecovery-as-a-Service (DRaaS).

DaaS: Portable Desktop Services

Desktop management is a fundamental service for IT organizations. It’s critical for keeping the employees of a company productive. But there have been long standing challenges with managing the traditional desktop. The investment in desktop hardware can be a significant capital expense, especially for large organizations and day-to-day management of these devices can be a huge time sink.

DaaS solutions are secure, cost-effective, easy-to-use and portable – you can get the same desktop on any device.

According to the 451 Research Group, “Interest in third-party DaaS is at a fever pitch.” IT consumerization, BYOD (Bring Your Own Device) initiatives, increase in mobile workers, Windows 7 migrations and security/IP concerns are driving organizations to reevaluate their desktop strategy.

Virtual Desktop Infrastructure (VDI) was supposed to address many of these challenges, but it came with its own set of issues. While it has been promoted as a technology that can save businesses money, large upfront capital expenses and complexity have created barriers to virtual desktop adoption.

With DaaS, savings come from operational expense reductions from centralizing and reducing administration and hardware savings over time. DaaS delivers faster desktop deployment, enhanced security, less downtime and lower support costs - and can enable a truly mobile workforce.

MaaS: Bare Metal In The Cloud

MaaS - the dynamic provisioning and deployment of whole physical servers, as opposed to the provisioning of virtual machines - is a drastically underrated cloud service. MaaS services will finally open the floodgates to allow any application to be run in the cloud – any application with any service level. That means multi-tiered apps with a backend Oracle database, home grown, performance-intensive applications, low latency trading applications, etc.

It’s been hard for people to pay attention to MaaS, mostly because server virtualization has been “the shiny new toy” over the past few years and frankly MaaS is not an easy thing to provide. But that may change once IT administrators see the speed, scalability, agility and simplicity with which they can deploy and protect their underlying server infrastructure.

The statistics are clear – a large percentage of servers have been virtualized in the enterprise (40% - 50% now and heading to 60% - 70%). However, there are still a large number of applications that remain running on bare metal. That important (and underappreciated) fact means that MaaS could be a key ingredient to driving more widespread adoption of cloud technology.

DRaaS: Increased Demand For Disaster Recovery

Over the past few years, IT departments have had to live in a culture of cost reduction – it’s just been the way of life. That culture has resulted in aging equipment, overworked staff and lots of cut corners - a perfect recipe for higher failure rates. The fact is that hardware failure and human error are still the leading causes of unplanned outages - but devastating storms and other catastrophes are also forcing businesses to get serious about geographic disaster recovery planning. Some estimates put 2011 weather related disaster costs at almost $150 billion worldwide, up 25% from 2010. And that is just weather, and doesn't include the earthquake and tsunami in Japan.

Public Cloud Failures Drive Demand

Another strong driver for disaster recovery is public cloud outages. The public cloud companies are under intense scrutiny - every major outage is noticed and publicized. The statistics show that public cloud outages are on the rise year-over-year, and because so many businesses use these services, public-cloud service failures are felt very broadly – e.g. the Amazon outage that impacted Netflix.

One of the forces driving the next phase of cloud computing adoption is the delivery of specialized services like DaaS, MaaS and DRaaS. These services will help improve the service level of cloud resources, boost efficiency and automation and deepen the consumerization of IT resources. They will also give companies more confidence in placing business-critical applications into cloud infrastructures.

Image courtesy of Shutterstock.

100 years from now, when the historians look back at the beginning of the 21st Century and shake their heads in amazement that we hadn't yet figured out flying cars, one thing they should give us credit for is that we finally figured out how to scale... everything. Even though the promise of the Web as a center of knowledge has been overrun with rampant commercialism, sometimes commercial interests actually align with the delivery of knowledge.

Software, databases, customer relationship management… these are all key elements of information technology that haver been pushed into the cloud to be deployed "as-a-Service" (or *aaS). This follows the model of Software-as-a-Service (SaaS), Data-as-a-Service (DaaS), and the like. Other, more rudimentary, aspects of IT have already been deployed this way, to great effect: Witness how online bookseller Amazon now dominates cloud computing by introducing Infrastructure-as-a-Service (IaaS) a few years ago.

As the idea of a sharing and scaling services that were otherwise once local and isolated continues to spread, we are now seeing just about every function you can imagine being delivered as-a-service to any business who wants them.

Marketing-as-a-Service

The most recent example of this new *aaS trend is the release of the Vocus Marketing Suite, a hosted marketing service that enables small- to medium-sized businesses to access marketing tools and (more importantly) expertise for those SMBs to use.

Anyone, really, can toss together a bunch of tools to market with social media, email and press releases. A one-pane aggregate application could handle that. But using the publicly available big data that's readily available on the Internet, Vocus' new application is designed to push out very targeted information that pertains to a business' marketing goals.

Say a business wants to sell jewelry, outlined You Mon Tsang, Vocus' Senior Vice President of Products. The Marketing Suite will listen for keywords on social media channels to determine who's an influencer in the jewelry scene or maybe just which desperate significant other is out there looking to buy an anniversary gift fast.

If you're using email marketing, the tool will make sure you're not spamming potential clients, either in frequency or through the language you're using. Human editors will also step in to help craft messaging. The key to this new service making the expertise affordable to SMBs who might otherwise have to go it alone.

Big data, as mentioned, makes this all possible. In the past, marketing was sort of a gray area when it came to hard results. Using a new class of metrics, marketing's return on investment is now much more easily calculated, and results can be concretely measured.

Healthcare-as-a-Service

There are some who would argue that some things still don't scale very well. Medical information and healthcare services seem to be one of them.  Sure, you can go on to WebMD and find out how to treat the cold that seems to be coming on… but without medical expertise at your disposal, you may decide that you really have the bubonic plague. And while "the plague is upon us" has a nice historical ring, it also tends to be a bit alarmist.

Healthcare professionals don't scale terribly well online, if only because the medical arts depend, usually, on face-to-face contact between the patient and the caregiver.

This is not to say that some aspects of healthcare can't be found in *aaS. A new startup in Indiana called hc1.com, for instance, has created a very niche cloud approach to resource management for medical labs, so they can work with multiple providers and deliver analyses more efficiently.

As medical providers continue to work with government and market requirements to use electronic health records, vendors like ClearDATA are working the edges, delivering secure messaging and cloud computing services.

So healthcare and medical-sector services are finding their way into the cloud, though still more on the edges instead of a full-on approach. But after a few more years of medical-monitoring innovation, who's to say you won't someday get a text message that says "Stop eating that pastrami, your arteries are about to pop!"

Anything-as-a-Service?

The world around us, thanks to connectivity and much faster computing platforms, seems destined to push all manner of services on to the Internet, where they can be acquired on demand, without having to build your own infrastructure to support them. Distance learning, shopping, news gathering and many more are already there. Others are coming, and there's no telling how far the trend will go.

Image courtesy of Shutterstock.

When it comes to the mobile industry, we talk a lot about new devices and the virtues or detriments of each one. Is the iPhone 5 the best thing ever or does the Samsung Galaxy S 3 blow it out of the water? These questions, while pertinent and fun to talk about, really are ancillary to what makes the mobile industry run.

Why do people buy smartphones and tablets? It is not because they have quad-core processors or super fast LTE connections. People buy mobile devices for what can be done on them. Apps, and the services that run them, are truly the backbone of the mobile industry.

So, we present our top 10 mobile products of the year. To make this list, a product needed to be mobile-Web or -platform enabled. So, no devices (which we ranked earlier this month) or the platforms themselves. 

What made our top 10 in 2012? See the list below.

10. Instagram

No company or service has had a bigger roller coaster year than Instagram. The social photo sharing app released an Android version, had a very public backlash against its Terms of Service and, oh yeah, was acquired by Facebook for nearly a billion dollars. 

Instagram also saw the type of exponential growth that startups can only dream about. It grew by tens of millions of users when it announced its Android app and then added tens of millions more when it was acquired by Facebook. Instagram in 2012 went from a popular iOS photo-sharing app with around 20-25 million registered users to a 100-million-user behemoth that has become synonymous with social photos. 

The elm trees of Central Park, New York City by Dan Rowinski

9. LevelUp

When it comes to mobile payments, we gave Square the No. 1 honor for mobile products in 2011. The dongle-based mobile payments system was truly deserving and we could very easily include it on this list this year. But we are taking this opportunity to highlight another mobile payments startup that is beginning to make waves in the industry and has a ceiling that could lead it to be a major player (or acquisition bait) in 2013 and beyond.

Boston-based LevelUp is an app that uses QR codes to make payments at local merchants using the company’s system. LevelUp will provide retailers with Android smartphones to act as QR code scanners (or Near Field Communications later, if needed) and provides loyalty discounts to consumers. For instance, if you are at Four Burgers in Cambridge, Massachusetts and you pay for the first time with your smartphone on LevelUp, you get a $3 discount on your order. Come enough times and you can “level up” to greater discounts. 

LevelUp has two important factors working in its favor. First is what the company calls “interchange zero.” Interchange zero is the concept that merchants do not have to pay a fee for every transaction made on the system, such as the 2.75% or so that they have to pay through financial processors like Visa or American Express. LevelUp makes its money through the loyalty/advertising sector as opposed to interchange. The other prong in LevelUp’s attack is that it is making significant progress in getting its system into actual retail locations, something that larger companies are having trouble with (right now). As of the beginning of December, LevelUp had 500,000 registered users and had processed two million transactions. That is a drop in sea of the large payments industry, but the model LevelUp is using has significant disruptive potential.

8. Waze

Social… driving. When we first heard of Waze, this seemed like a bad idea. A very bad idea. The last thing we want is for people to be on their phones while in the car. Yet, upon further inspection, Waze is much smarter than just an app that helps you tweet traffic alerts. 

Waze is a background location and turn-by-turn navigation app that works with your friends and the people around you to know where traffic is, where cops might be hiding or where the cheapest gas on your route is. Waze taps the collective consciousness to get you where you are going faster and smarter. 

Waze also saw huge growth in 2012. The Israeli-born, San Francisco-bred company scored a big win when it partnered with Apple as part of its iOS Maps system and has added many users through its expansion to various mobile operating systems.

7. Nuance – Dragon/Swype

Nuance is a repeat offender on this list, mostly because it is very difficult to keep it off. Swype, the typing feature that allows users to input text by swiping letters across the keyboard without taking their fingers off the keyboard, is one of the most addictive features included in many Android devices. Nuance licenses Swype to many mobile manufacturers, such as Samsung.

Nuance has built a neat little business licensing its technology (whether it was built by Nuance or acquired, like Swype was) to mobile manufacturers. Nuance also licenses its speech recognition service to smartphone makers and it is believed to be one of the technology suppliers for Apple’s Siri personal assistant app on the iPhone. 

2013 will bring good things for Nuance as it works on its own voice-controlled personal assistant mobile systems and makes additions to its Swype and Dragon line of products. 

6. Geoloqi

We normally do not like to add products on this list that have been acquired and are now a cog in a larger organization, but Geoloqi deserves its spot as one of the bright startups working to solve many of the problems associated with location services on smartphones and tablets. 

Portland, Ore.-based Geoloqi, which joined Esri this year, provides accurate and granular location services to app developers and manufacturers while still trying to preserve the battery life of a device. Essentially, Geoloqi provides developers with a software developer kit (SDK) that runs its location services in the background of any app in which it is included. The ability to run a data-gathering, persistent background location service without killing battery life cannot be understated. The use cases are vast, from government agencies using apps to track employee locations in the field to helping manage power consumption in your home through your location. 

Geoloqi is one of those companies that most mobile users will never really hear about. But, the old cliché applies, “good technology should be almost indistinguishable from magic.” Well, if you are ever using your phone and it performs and action that say to yourself, “how’d it do that?” There is a good chance a company like Geoloqi is behind it.

5. Lookout

If we saw anything in 2012, it was that the proliferation of mobile devices is changing the nature of computing on a vast, accelerating scale. Well, with any industry-changing event, the good comes with the bad. In this case, the bad is the pace and volume of mobile malware in the smartphone ecosystem looking to steal your data and cost you money. This is especially true for Android, but if you have an Internet connection with your cellphone, you are not immune from spam, scams and viruses. 

A variety of companies are working to tackle the mobile malware problem. Kaspersky Labs, Sophos, Bitdefender, Symantec and others are all on the forefront of research and defense against malicious hackers. But, for the second year in a row, we think that Lookout is one of the best companies working on preventing and detecting mobile malware. 

Lookout updated its Android app in September and added several new features. The app will scan all of your apps looking for vulnerabilities in your system. In and of itself, that is not really profound. Add Lookout’s ability to find your phone (even your iPhone) anywhere you may lose it, scan all your apps to determine what permissions they can use to share your personal information, institute Safe Browsing in Chrome for Android and backup just about every piece of data on your device, and you have one of the most comprehensive and powerful mobile security services available.

4. Chrome for Android/iOS

If we have to pick just one mobile browser, it has to be Chrome. Dolphin, Opera and Firefox for Android are all worthy candidates, but the Chrome browser for iOS and Android from Google is the best of the best. 

Chrome for iOS and Android brings all the great features that you expect from Chrome on your PC. It is fast, remembers your history and has a robust bookmarking capability. The best feature is the ability to sign in to your Google account and sync your Chrome browser across all of your devices, remembering pages that you may have visited on your Android smartphone, iPad or PC. This cross-platform sync is not unique to Chrome, but now that you can tie your browser to your Google account across any type of computing platform you might use, the ease and benefit of using Chrome is apparent. 

3. Evernote

When Evernote landed $50 million in funding in July 2011, the startup’s CEO Phil Libin said that he thinks his company can last for 100 years. 

We’re inclined to believe him. 

Evernote Toolbar Widget for Android

If you are an Android user, there is a pretty good likelihood that you have downloaded and installed the Evernote Toolbar Widget… and cannot live without it. If you are a copious note taker on your mobile device, Evernote is one of the best apps you can find to take photos, audio files, save articles or just jot down thoughts and sync it to the cloud. If we think of the factors driving new era of computing – mobile and cloud – Evernote is the undisputed thought leader in the personalized cloud productivity space. 

2. Spotify

Whenever I ask people about apps they cannot live without, I get a fairly short list: Maps (usually Google’s variety), email, Twitter, Facebook, Zite/Flipboard, Instagram (or some type of photo app) and maybe something specialized to a person’s specific interest, like RunKeeper or Strava Cycling. Those who have been indoctrinated also add one app that may slip off a casual user’s list.

Spotify.

The cloud-based music streaming service engenders such loyalty among its users that many of them have trouble remembering a life before the ability to look up just about any song they want and stream it immediately, from anywhere. It also allows users to save files locally, has a Pandora-like radio streaming service, can be social or anti-social per the user’s preference and generally has any song you are looking for (yet, for some reason, a surprising paucity of Hootie And The Blowfish).

  Spotify Widget

Spotify and streaming services like Pandora, TuneIn, Rdio and the rest of its ilk are also causing a subtle shift in how people buy and manage their mobile phones. For instance, the Apple iPhone has long come in three storage sizes: 16 GB, 32 GB and 64 GB. Apple has built a very robust business on this tiered-pricing model. But, as more people move to streaming services for audio and video, the need for extra storage for media purposes is lessened. The priority then falls on the amount of data and speed you can get from your carrier. These services are changing user spending behavior and Spotify is leading the charge. 

1. Google Now

You have heard of artificial intelligence. Chances are, you probably do not quite know what that means in its entirety. True artificial intelligence (robots that are as smart as humans and can think and behave on their own) may never become a reality, but Google is trying its darndest to give us the smartest kinds of computer intelligence, straight into our pockets. 

Google Now has the ability to know where you are, what you are doing and then give you help along the way. Leaving on your morning commute? Now will detect traffic and give you an alternate route, if desired. Its Card-based system can hold your boarding pass, local weather, reservations, events, appointments and meetings or the score of your favorite sports team. What makes Now special is that it has the machine learning Google is known for, the cloud to sync its information anytime, anywhere and the information intelligence that is unique to Google’s approach to the world. 

Now should just be getting started. Google ramped up its team in 2011 to work on Now, making it a priority within the company. In successive versions of Android, Now should become smarter, more intuitive and more able to serve your needs no matter what you are doing. 

Lead image courtesy of Shutterstock.

While it might be an overstatement to claim that Microsoft news dominated 2012, the company’s drumbeat of publicity around the launch of Windows 8 gave it a disproportionate significance.

And Microsoft did gear up and execute on a series of milestones: Windows 8, the Surface tablet, Windows RT, Windows Phone 8, Windows Server 2012, and Office 2013 (at least within Surface). But what are the big trends behind those headlines? What's really shaping Microsoft right now?

1. The Headline: Windows 8

Beginning in February 2012, Microsoft launched the Consumer Preview of Windows 8, followed by the Release Preview of Windows 8 on May 31. Microsoft was out in front of reporters and analysts, explaining via blog posts how it built Windows 8 and what its goals were. The company touted how Windows 8 was built for a variety of screen sizes and delved into the touch keyboard through discussions of the new apps. But Microsoft has been surprisingly cagey about Windows 8’s success so far: We won’t really find out the final results until January, when Microsoft reveals its sales numbers as part of its first-quarter results.

1a. The Real Trend: Metro

If there was one word that summed up Microsoft during 2012, it’s this: Unity. The Consumer Preview of Windows 8 ushered in the “Metro” interface that will define this generation of Microsoft’s products. Flat, but bright, iconic and ambitious, with a reliance on typography rather than icons, the Metro interface (now referred to, unofficially, as the “Microsoft design language”) was quickly adopted across Microsoft’s other product lines, most notably within Windows Phone, Microsoft’s Web Apps, and sites like MSN.

2. The Headline: Surface

Microsoft’s big reveal of the Surface tablet in the summer was a masterpiece of temptation: a slick demo, a quick hands-on with the product, then out the door, appetites whetted. Microsoft’s refusal to discuss pricing or even allow a hands-on until days before the product was actually released forced many buyers to pre-order a Surface sight unseen. That, combined with the uncertainty around the Windows RT operating system used on the base model Surface (how close was it to Windows 8? How did Metro apps work? Was Office RT really Office?) probably added too many questions to Surface to make it a real success, although it “sold out” soon after launch. Still, like Google’s Nexus line, the Surface launch taught us that consumers prefer the hardware that the ecosystem builder “owns.” So far, third-party Windows 8 tablets seem to have gained even less traction.

2a. The Real Trend: Owning Your Own Platform

As ReadWrite noted in July, a key lament of ecosystem control is designing the hardware, software and services so that all three work together in a unified whole. Apple does this with the iPhone, iOS and iCloud; with Google’s purchase of Motorola, it can do the same. Until Microsoft decided to build Surface, however, it lacked the hardware component. Time will tell how Surface fares (the price still seems a bit high) but Microsoft has all the pieces of the puzzle in place to control its destiny.

3. The Headline: Windows Phone 8

Living with the HTC Windows Phone 8X for a month after its October launch taught me two things: Windows Phone 8 is a bright, beautiful mobile operating system, and that it deserves more traction. Yes, there was the “smoked by Windows Phone” viral efforts. But almost more than Windows 8, Windows Phone feels modern and connected, and Live Tile integration makes it fast and easy to use. So far, the general perception is that from a quality perspective, at least, Windows Phone 8 belongs in the same category as Google’s Android and iOS. But from a sales perspective, Windows Phone 8 isn't really close to competing with the market leaders.

3a. The Real Trend: Apps And The Web

Unfortunately, apps still remain the Achilles heel of Windows Phone 8 - and, if you consider Windows RT or Metro apps in the same category - of Windows 8 on PCs and tablets as well. Microsoft has made a valiant effort to convince users and developers that optimized apps like “Contre Jour” are the wave of the future, even as detractors sniff that Microsoft’s Web browsers are the least standards-compliant of all. If Microsoft can convince users that Web apps fill the bill, then it can whitewash any apps deficiency. But critics have also loudly begged for Facebook to build an app, rather than an HTML5 mobile page. So far, Microsoft is on the losing side of this trend.

4. The Headline: Office 2013

As the name suggests, Office 2013 is a 2013 product, at least for consumers. Microsoft began making its versions of Office 2013 available to businesses in early December.  But when Windows RT launched with Surface, one of its more potent features was the inclusion of a “preview” of Office 2013 Home and Student, that quickly morphed into a final version. ReadWrite has complained about the inadequacy of Microsoft’s Office Web Apps, but Office 2013 has been well-reviewed; it just costs more. The year ended with rumors that Microsoft would begin selling Office for the iPad and iPhone, a development that looks increasingly more likely as time goes by.

4a. The Real Trend: Office 365 Subscriptions

Subscriptions. The cloud. They go hand in hand. By pricing Office 365 subscriptions more cheaply than the standalone editions, Microsoft has encouraged consumers to bury their Office payments as a recurring charge on their credit card, rather than as a major purchasing decision to agonize over. Google’s cloud services have made save-all-the-time, run-anywhere cloud office suites indispensible, and Microsoft, no dummy, has embraced the concept whole-heartedly.

5. The Headline: Windows Server 2012

Unless you’re an IT geek, chances are that Windows Server was one of the least interesting Microsoft announcements this year. But for IT administrators, the release of Windows Server 2012 on Sept. 4 was an important milestone. Boasting improved virtualization (Hyper-V) capabilities, the new ReFS file system, and improvements to Active Directory, Windows Server also simplified the number of versions, consolidating them to essentially three: Foundation, Standard and Datacenter.

5a. The Real Trend: Enterprise Is The Place To Be

From IBM ditching its ThinkPad and computer hardware business in favor of enterprise services to Dell’s transition away from “Dude, you’re getting a Dell,” the fact is that tech vendors are increasingly looking to stable, high-value, recurring contract services. Microsoft’s Server and Tools business isn’t Microsoft’s most profitable - that’s the Business Division, with about twice the profits of the server business - but both share high-margin subscription business models. And with more and more users turning to tablets, the emphasis is shifting to the datacenters powering the cloud services that connect these portable devices. Microsoft certainly isn’t alone here, but it's making a big effort to be a significant player.

Conclusion

Say what you will about the quality, utility and pricing of the products and services Microsoft launched in 2012. But recognize that those launches pretty much all went smoothly, without major availability issues, bugs or other glitches. We’ll learn the final results - revenues and profits - in the quarters to come. Microsoft has talked optimistically about its successes all year. Come January 2012, we’ll see how well all of Microsoft's hard work really paid off.

The conventional wisdom has it that Big Data has been good for large enterprises, and very, very bad for consumers and small businesses. According Intuit CEO Brad Smith, though, big data can also be "the great equalizer."

Smith bases this idea on a new study released by the company on Friday. Called "The New Data Democracy: How Big Data Will Revolutionize The Lives Of Small Businesses and Consumers," and conducted by Emergent Research with "a mix of research and forecasting" (wishful thinking?), the study contends that "the emerging availability of data and analytics… gives small businesses and consumers greater access to cost-effective, sophisticated, data-powered tools and analytical systems."

Data Is A Raw Material 

According to Smith, "Data is becoming the newest raw material for business, equal to or greater than capital" in its ability to drive growth. Access to this kind of data has largely been confined to large enterprises, but if it can be shared, it will let "small businesses and families make smarter decisions" and help "level the playing field," Smith told a small group of journalists and analysts on Thursday.

That's critical, because more and more critical decisions and risks are being pushed down to consumers, covering everything from retirement planning to healthcare, Smith said. "Data helps us navigate that." 

Intuit, of course, has access to vast amounts of data from its 60 million customers, and wants to be in the business of helping its customers use that data to their advantage. "We try to use all available data that our customers give us permission to use," Smith said. If Intuit can put that data in their hands, Smith said, it's like "Moneyball for the small business owner… creating a power shift from big business to small business."

Or at the very least mitigating the trends going the other way.

Big Data Really Has Helped Big Business

According to Steve King, partner at Emergent Research, which conducted the research, "Big data has un-leveled the playing field. Big business has definitely gained a competitive advantage" from Big Data. "We are in a period where big businesses are at an advantage."

""Big data is definitely going to kill some small businesses," King told ReadWrite. Small businesses that don't get with this will be severely disadvantaged, "especially firms reliant on location or opaque pricing. They're going to get hammered."

On the other hand, small companies that do manage to take advantage of big data will have an advantage compared to other small busineses, and be better positioned to compete with big businesses, King said.

For consumers, King said, data helps empower us to deal with critical decisions in a more organized way. "More and more people are going to be able to take advantage of this in a positive way," King said. Many consumers, King predicted, will take advantage of big data by proxy, via "digital concierges" like Siri, as well as "the personal services they use, which will become smarter, more efficient, and more personalized." Not just in shopping, but also in healthcare, for example, where big data tools will lead to better diagnostics and better choices of where and how to treat various conditions.

The key to this rosy future, of course, is getting these big data tools to small businesses and consumers in ways they can understand, afford and use. Intuit is well positioned to do that, and has a number of related projects in the works. (See Inside Intuit: How A Software Kingpin Is Remaking Itself For Mobile & Services.) And plenty of other companies will no doubt try to do the same thing. (See How "Big-Data-as-a-Service" Can Help Smaller Companies Compete.)

Making Big Data Easy Won't Be ... Easy

But it won't be easy. Big data is inherently complex. That's why it's only now being properly used in even the largest, most sophisticated organizations. And even they don't always understand how to use it properly. (See Utilities and Other Industries Not Ready for Big Data, Say New Oracle Reports.)

That small businesses and average Joes will come out on top in what Smith called a "data revolution" is by no means assured. Fortunately, King said that unlike small businesses, consumers who don't participate won't end up as "losers," they "just won't benefit as much."

Photographs by Fredric Paul.

New information from IDC this week reveals that despite global economic doldrums, IT spending will still grow by an average of 6% globally. But don't look for a corresponding increase in IT jobs, which are getting substituted by more automated software and services.

The growth projection comes at a time when new signs of life in IT are being seen in many large companies, fueled by the demand for more specialized talent as well as coping with an increase in bring-your-own-device (BYOD) policies.

That was some of the takeaway from IDC's 2012 edition of its Worldwide Black Book Query Tool, which is pegging overall growth at higher-than-expected levels.

More Spending, Not More Employment

"There has almost been a decoupling between IT spending and hiring, with a real divergence in the last two to three years," said Stephen Minton, VP at IDC. "Businesses are spending money to save money," Minton explained. But what they're spending on, more than hardware expenditures and staffing, are software and services. Software services have enabled IT departments to save money on personnel, and they are taking advantage of the savings.

"Instead of hiring new people, we're seeing a kind of labor substitution with the acquisition of these services," Minton elaborated. Such services range from the obvious - such as virtualization and storage management - to the not-so-obvious, including collaborative tools and social software. These last two categories, Minton emphasized, were seeing the fastest growth in software and services right now.

The news is not all bad in IT hiring; General Motors announced last week that it would be hiring an additional 500 IT staffers in Austin, Texas, part of an overall plan to put 10,000 more IT workers on its global payroll in the next three to five years.

Still, hiring across the board is still slow, as IT departments work to replace staff with more automated services.

Hardware Weak Now, But Look Out

The other big theme from this year's report is the expected uptick IDC sees for PC and hardware sales. In the fourth quarter, the analyst firm is looking for an big boost in PC sales due to consumer purchases of Windows 8. On the enterprise side, Minton expects that see a stronger-than-usual hardware upgrade cycle over the next two years.

The reason? IT managers and CIOs have been holding off buying new systems for a long time.

"The last three years were so bad for hardware," Minton explained. That repressed spending, Minton believes, is about to end as older PCs and servers will simply have to be replaced as they give up the ghost.

"Those companies committed to using Windows have been waiting for this for a long time," he added.

The question is, who will be responsible for buying the new hardware? Will it be in the hands of IT procurement, or will the rising trend of bring your own devices shift the purchase decisions to employees?

Employees Spend More on IT

In the last 18 months, Minton and his colleagues at IDC have seen a steady rise in Chief Information Offices who are giving up on stopping employees from bringing in their own hardware. Many are now considering subsidization programs that let the employee purchase a system of their own.

While this type of spending decentralizes IT expenditures and creates more heterogeneous corporate environments, Minton also believes it actually increases the rate of IT spending. "When business users have more choice, they are enthusiastically buying more powerful hardware," Minton said.

As evidence for this supposition, Minton had only to point to the rise of smartphones, which might never have exploded in growth without pressure from end users who wanted to use these devices in corporate settings.

With so much decentralization in software, services and hardware management, it's clear that IT departments have a lot of restructuring to do. How this translates into jobs is not terribly positive right now, but for overall IT spending, the future is far from bleak.

Image courtesy of Shutterstock

Working virtually sounds like heaven to many startups. After all, not having a central office staffed with employees saves money on rent, utilities, parking, etc., freeing you to invest in research, development or marketing.

On the other hand, operating virtually is no panacea. Before you make the virtual leap, you need to figure out exactly what working virtually means to your business.

The concept of virtual work has many names, from telecommuting and teleworking to distributed companies and remote workers. And virtual companies can be structured in several ways:

1. Lead a distributed workforce, consisting of full- and part-time employees, independent contractors or some combination, all working out of their respective homes.

2. Operate out of an office space (either your own or shared). You allow your staff to split their time between working at the office and working at home.

3. Be the only actual employee of your company. When you need something done, you outsource it to freelancers and independent contractors.

Thanks to improvements in broadband and collaboration technology, the ranks of virtual workers are swelling. According to the latest (albeit a little dated) numbers from the Bureau of Labor Statistics, in 2010:

• 64.2% of self-employed and contract employees worked at home

• 25.8% of part-time employees worked from their homes.

Good for young and old companies

While it obviously makes sense for brand-new startups to begin virtually, the concept also works for existing startups that began life in more traditional offices. My own company - GrowBiz Media - did just that. We leased space for about 18 months before realizing we could save many thousands of dollars by working from our homes.

Even established businesses can benefit from virtual arrangements. Matthew Goldstein founded Idea Consulting about 12 years ago, but the company went virtual about seven months ago. The firm’s 53 freelancers and part-timers are now happily collaborating on a huge project, the EuraMedia 2012 summit, while scattered across the United States, the United Kingdom and Russia.

But Goldstein admits: “We are still making mistakes.” His biggest hurdle was finding high-quality virtual workers.“The majority of the people we found were not qualified and overcharged,” he explains.

Goldstein also found that, despite what the virtual job placement agencies he worked with (Odesk and Elance) promised, “There is very little accountability" when it comes to contractors. You may not be satisfied with the completed work, but since the placement companies get a commission, “they tend to always side with the contractor.”

So how can you make virtual work fly at your company?

“You must put in systems of accountability that replicate those of a brick and mortar office,” advises Goldstein. “There needs to be constant monitoring, incentives and penalties, and strict management.” Also key is finding the right technology to make your project work. “Collaborative sites like Basecamp are great for helping to engage workers and get them to collaborate,” says Goldstein.

Ask the Expert

Sara Sutton Fell, CEO and founder of FlexJobs, a website for flexible jobs, offers more tips for virtual startups:

• Plan ahead. If you want to take a physical business virtual, you first have to think about all the facets of your business and how you envision them continuing to run virtually. What will this change mean for outside stakeholders — clients and partners? How can you make sure the transition goes smoothly for everyone? Sometimes the best way to go virtual is gradually, transitioning existing staff to home offices [or adding contractors] in small increments, so that your clients and partners barely notice the changes.

• Think about the technologies (computers, software, services, phones, etc.) that are essential for working virtually. Security and usability are two big concerns for teleworkers. You want to make sure everyone can access important information securely and easily, using reliable technology. Sometimes [usually] your workers' home computers just won’t cut it. If possible, provide remote employees with a complete suite of required hardware, such as laptops, monitors, scanner/copiers, webcams, microphones and more. If you use independent contractors, don’t give them equipment, it can raise questions with the IRS.

• Consider communication services. When you’re managing a remote team, it’s vitally important to keep everyone communicating with each other. This helps keep people on task, reduces feelings of isolation and boosts engagement. Employing a wide range of collaboration tools like Yammer, instant messaging (IM), email, Join.Me and Skype can help keep everyone in the loop.

Technology isn’t everything. Schedule regular meetings (at least weekly). It’s also helpful to create channels and leave time for virtual water-cooler conversations to let the team connect and build camaraderie and trust informally.

No matter what you do, don’t expect an overnight transition. “Working virtually is not easy,“” Goldstein says, “and you must be ready to put aside at least three months to track people and test them out before you can count on truly developing a team.”

Image courtesy of Shutterstock.

A conversation with Dr. Gautam Shroff, Vice President at Tata Consultancy Services (TCS) and head of the TCS Technology Innovation Lab in Delhi, reveals that they’re making progress, but that they still have a ways to go.

Tata, of course, is a huge collection of more than 100 companies across 6 continents, including everything from car makers to consultants, chemicals and consumer products. And this year, TCS opened a facility in Silicon Valley.

Innovation is complicated

Shroff acknowledged that firms like TCS are not known for innovation, but said the picture was more complicated than that. There are two sides of innovation, Shroff said, creating ideas, and getting those ideas out into the world to create solutions out of them. TCS, he said, has done a lot of work on the latter side.

The company has been investing in research since 1981, he said, and now has the largest academic computer science effort in India. Those efforts have contributed to significant businesses, but mostly for Tata itself. Shroff said that in the 1990s, TCS research created software development tools that led to the company’s entire financial product business, as well as the only end-to-end cloud business in India, with hundreds of small and midsize business customers.

Most of Tata’s R&D isn’t productized, though, Shroff said. Instead of producing “great science,” it’s used for practical, incremental business improvements within Tata’s activities for its customers. “We also innovate for our customers where we have replicable [innovations]," Shroff said. “We just don’t call them products.”

That’s useful, certainly, but not exactly what most observers think of when they hear the word “innovation.” So, how exactly is Tata moving toward more innovation in its offerings?

Trying to get smart about business intelligence

The key areas Tata is focusing on include social media, cloud computing, mobility and big data. And for Shroff, those all come together in business intelligence, which he sees reaching an important inflection point that requires major changes - fusing deep analysis of big data from both inside and outside the enterprise, and looking for new patterns and correlations.

As an example, he cited companies that are monitoring Twitter streams to identify “adverse events” that might not reach news outlets but could still impact business operations. “If that matters to you, it’s better to know now, so you can alert people in the field on how it’s likely to affect their business,” Shroff explained.

“People are looking at it with great curiosity in the business world,” Shroff said, “exploring how more data can improve the business. What was traditionally a niche market can now be a force … something the CEO needs to know about. And we are right in the middle of that.” While Shroff wouldn’t name individual BI customers, he said Tata is working with firms in the retail, consumer packaged goods and financial services markets on the consumer and supply sides of their businesses.

Finally, Shroff said opening a facility in Silicon Valley has given Tata access to a new talent pool. “We’re now able to get people who work in startups who don’t want to leave the valley.” The company also uses the outpost to work with academics at Stanford University and UC Berkeley and partner with startups that are developing useful technologies.

Donna Burbank - who's a senior director of product marketing for CA Technologies' long-time data visualization tool, ERwin, has a new phrase for this class of customers: business sponsors. "When I talk to our customers, they tell me it's a whole new... thing, for lack of a more technical word. They've heard of SQL Server, but what is this SQL Azure thing? They don't have the skill sets, and may be nervous about that. These business sponsors might not be moving the information, but they want to see it. And they don't want to look at those database scripts. They want to look at something they can understand."

So it is that CA Technologies found itself in the business of manufacturing a class of software that a new and growing chunk of its customers might not actually care about all that much: database visualization tools. ERwin has been the market share leader in this category ever since its creation in 1998. Its typical customers have been database architects (DBAs), the people whose jobs are to model the classifications and structures for the relational data that businesses rely upon every day.

Welcome to Your World

But the shift to cloud technologies has partly been fueled by the need for tremendous space for data warehousing - to house the huge data stores generated by millions of Internet customer transactions. It's that shift which is pushing data outside of the constraints of traditional SQL relational databases. That push is forcing businesses to examine, some for the very first time, the structure of their data. And what they're seeing, they don't understand.

"A lot of the move to the cloud is a business decision. The technical people doing the move are probably still the DBAs, but they're challenged," CA's Burbank tells RWW. "There will always be that core group of people who want to use a data model, that's our sweet spot right now: the data architect, the DBA. Those people and more would like to use a Web-based interface."

ERwin's new Web portal, she explains, is a browser-based interface for information that has otherwise been modeled for DBAs by ERwin Data Modeler (which itself moves to version 8.2 this week). This new portal will help both architects and Burbank's "sponsors" to analyze the relationships of data from a business impact standpoint. "If I'm building a data warehouse, I want to see how data moves from the source system to the target warehouse to the reporting tool. Maybe I'm changing a data element; what other parts of the organization are affected? You could sort of get that through ERwin's repository [in Model Manager] with some queries, but it wasn't the tool for that."

The new, more objective breakdown aims to give multiple classes of users a comprehension of the data that they may have never had before. A "sponsor" who wants to search for relationships is going to expect search to behave like Google, Burbank explains. So the Web Portal tool gives that user a text-based search query line (shown above). What that user gets in return will be something that may explain what tables or fields relate to the search criteria, but it might not directly correlate to the model as the original DBA intended.

As Burbank explains, the business user, not getting a complete overview from the initial response, may decide to export the data he's seeing into Excel, and generate some PivotTables from them. The DBA, on the other hand, may use the Portal's new graphical impact analysis tools to drill down further, or perhaps execute a "What If?" experiment. If a column is changed on a table, for instance, the DBA can see how the rest of the schema is impacted. "It's that type of drilldown over the Web that they could never do before," she remarks.

Big data vs. "lots of data"

All this said, ERwin is not quite yet a data warehousing assistance tool. While CA's Donna Burbank says it's something her company is considering, she points out that Hadoop and the restructuring of data it entails, lend themselves to very different situations.

"I've done several presentations where I've explained to people that there's 'big data,' and then there's 'lots of data,'" she relates. "And these are different use cases. Maybe I'm an energy company, and I'm trying to use a Hadoop-type structure to see uses across my different [operating units], and I need to eventually manage that in a warehouse. It's that analysis of that big data that then goes into a data model. One use case [involves] massive volume, real-time, more of a programmatic approach to data. There's a lot of messaging there around, is data modeling going away? Is data warehousing going away? Today, it's two different use cases. You're doing an analysis, and then you use the data model to make sense of that raw data. And if I'm going to use it for a BI report, that's when your data model comes in. I've done my data analysis with the big data; here's my data model to say which pieces of that I used in the warehouse."

Real-time analysis of big data, she goes on, may enable DBAs to add some elements to the relational data model that they may not have seen before.

As for the other use case, Burbank agrees that data modeling may never be appealing to 100% of the "sponsor" audience. But making it appeal to a somewhat greater audience through more intuitive graphics, along with Google-like search, could go a long way toward enabling those tasked with new responsibilities to be able to better understand what they are, and carry them out with a greater sense of confidence.

It supports all the great IRC client apps, of course, but Grove takes care of the fiddly parts of setup and hosting. All that's left for teams to do is sign up and start using it. Starting today they can do so for free at Grove.io.

Grove is the latest effort from Leah Culver, CEO and co-founder of Convore, and Convore developer/designer Jori Lallo. Culver was a co-founder and lead developer of Pownce, which was an early challenger to the Twitter way of communicating that also allowed attachments and events. Pownce was acquired by SixApart in 2008, and the service itself was shut down.

She got into real-time chat in 2009 when she and Defunkt built a Web-based IRC client called Leafy Chat for the Django Dash that won second place. The popularity of that client and her experience with Pownce sparked her interest in figuring out chat. "Actually, most of the content sent was replies to other people's content," Culver says of Pownce, "because people really want to talk with each other."

Insights From Convore

That insight led to the creation of Convore, which splits the difference between real-time chat and forums. It allows users to create topic-based forums, but replies are posted in real time. It works like chat if you're present, but it logs conversations like a forum. Culver says the Convore team thought its open-ended appeal would be an advantage, but it ended up making it difficult to identify a clear use case.

Several different use cases emerged: liveblogging, conference chat and internal team chat for businesses. The team chat was the part that piqued Culver's interest. She decided to create a new solution for that using everyone's old favorite chat protocol, IRC, but taking the effort out of it by hosting the service, offering a Web client, and providing all the logging, archiving and search. That's Grove. It launches today, and it's free.

Why IRC?

Geeks love IRC, but it comes with a few hassles, mainly having to host it, that have led teams away from using it in favor of easier IM solutions. As an old protocol, it also doesn't support user accounts in the way we've gotten used to in the Web 2.0 age.

But IRC has advantages over proprietary tools. It's a stable, open protocol - "like email," Culver points out - which means users can use whatever client application they want, on any platform, most of which are open-source and free. Without having to build apps for every platform, Grove can concentrate on eliminating the fiddly parts of IRC, and what's left is an easy, real-time, logged chat service for teams built around a trusted protocol.

Grove provides its users hosting, user accounts, channel access controls, and searchable archives, as well as a swanky Web-based client. But it still allows all the benefits of an open protocol like IRC, so team members can use whatever client app they desire on any device. Grove lists a few recommended apps on different platforms, as well as the easy instructions for connecting, at grove.io/help.

Chat For 21st-Century Teams

"We're moving towards a more distributed workforce," Culver says. "People are working remotely. You want to stay on the same page. Not everybody's always going to be at the office." Grove can help 21st-century teams keep in touch, and by handling the tricky parts of IRC itself, the barrier to entry is gone. And since it's open, teams who so desire can build their own custom clients, or modify existing open-source ones, so Grove can be a backbone for chat that's tailored to its team's exact specifications.

As of today, Grove is open for anyone to sign up. It's currently free.

Culver is speaking today at the Keeping It Realtime Conference about why Convore uses long-polling over websockets for moving its real-time data. She'll be talking about Grove as well. We'll post the video here as soon as it's available.

As part of their security service, they monitor more than 60,000 criminal websites and chat rooms where bulk IDs are traded. The idea is to leverage this expertise and provide a turnkey ID protection service that can be used for large-scale rollouts, such as in the case of the Sony Playstation breach earlier this year or when a company wants to offer such a service prophylactically for its own employees.

Here is what the consumer sees from their end with the service: once you enroll, you provide personal information that you want monitored, including credit cards, phone numbers and email addresses. The system can handle multiple entries for each item, although some of the resellers or implementations might only limit one phone number, address and credit card. When illegal activity is detected, such as the trading or selling of personal information, Global ID Protector will alert the consumer and provide instructions on how to prevent further exposure or fraud and take action to restore their identity.

An early reseller of this service for retail customers is security software vendor AVG, who incorporates Global ID Protector into their Premium Security software for $70. The version includes a variety of security features such as anti-virus and host-based firewall, was released in July, and is available for any recent Windows PC. Another is AT&T, who offers two levels of protection that range from $2 a month for basic online ID protection to $10 a month for full protection.

ClickToSecure can handle PCI 6.6 compliance checking in addition to more than 20 other tests for common Web application coding errors such as cross-site scripting and SQL injection. Yes, it isn't the only securing scanner in town; a listing for just PCI compliance has more than 150 links to others such as AppLabs PCI Scanner, Comodo HackerGuardian and eEye Retina just to name three. It is very easy to get started on and is priced reasonably. Your website must have a publicly-reachable IP address or URL to be scanned.

Once the scan is complete you are emailed the notification and you can download their report from their website. Ours took several hours for a simple Wordpress site. A sample of what the resulting report looks like can be seen below (our scan shows several failures):

To get interest up in its product, Cenzic will give away a free copy to the first 500 takers today that come in through the Microsoft storefront. Otherwise, there are several different prices, starting at $8 per month for a one-time only scan and can be as high as $133 per month for the "gold" service which includes five site scans that conduct a total of 24 different tests.

We wrote about ChaCha many years ago and Wolfram more recently last January about their APIs. Both companies had big splashes when they introduced their technologies.

In the partnership's first day, Alpha answered 32,000 of ChaCha's incoming questions, according to ChaCha. Given that ChaCha is used by millions each month, that isn't all that impressive, but there is opportunity for growth, and maybe a goal to make the Internet a more accurate place in this sector. Alpha can answer the most arcane factual questions, such as solve mathematical equations or delve into scientific knowledge.

Rails Hotline, which was just launched this morning, is powered by Pocket Hotline, a platform designed for companies to crowdsource technical support.

The hotline was created by Chap Ambrose, a long time Rails developer. Ambrose told us that he was looking for a way to give back to the Ruby community, since both Ruby and Rails are free open source software and he'd never contributed a single line.

He thinks other volunteers have similar motivations. "It's pretty satisfying to talk to Ruby beginners and help them out," he says.

Ambrose says so far the questions he's been getting are about things like best practices, or bits of the documentation that the caller is having a hard time with. He says these sorts of things make more sense to discuss live with another person than to post about in forums.

If you're interested in volunteering, you can find an application here.

The inaugral listings include jobs from Adservice, TWAR, Voxer and Yammer.

The board is powered by SimplyHired. It costs $350 to post a job for 30 days, but as part of the launch promotion you can post jobs for $100.

The interest in Node.js is extremely high, and this job board demonstrates that production use is starting to pick up.

If you've got Node.js skills, but don't see a job on that board for you, you might also be interested in Node.js Knockout, a 48 hour competition to write the best Node.js app. It's inspired by Rails Rumble. Node.js Knockout will take place Aug. 27-29 2011.

And if you don't know Node.js yet, check out full-text preview of Up and Running with Node.js.