Despite Google Android's long market-share rise against Apple iOS, developers continued to stick with iOS as their first deployment target. While Android offered superior volume, that volume was fragmented between different versions of the OS and disparate hardware. Meanwhile, Apple offered better development tools plus clearer, more profitable revenue options. Even open-source developers tended to congregate on highly proprietary iOS.

Something changed in 2012, however, and Android-related open-source development exploded.

This makes sense, given the target audience for mobile applications: consumers. Even though open source now permeates server-side computing, and drives industry trends like cloud computing and Big Data, it has had a negligible impact on the desktop, where mainstream users don't want access to source code and simply want polished products that work. Hence, despite the impressive efforts to clone Microsoft Office with OpenOffice and now LibreOffice, the world still happily gives Microsoft billions of dollars of Office profit each quarter. 

It's easier to stay on that beaten path.

Hence, while I don't expect open-source developer affinity for Android to squash iOS anytime soon, it's still a troubling sign for Apple. Even on the desktop, many mainstream applications are open source, including Adium (IM client for the Mac), VLC Media Player, Handbrake, and more. And if Android is the place open-source developers target for their innovations, we're likely to see the next Big Data-like trend emerge on Android, not on iOS, just as Linux is the home of cloud computing and Big Data on the server.

Open-source developers matter. And, apparently, they matter most to Android. 

For years, the software industry has been trending away from so-called 'copyleft' licenses like the GNU General Public License (GPL) and toward permissive, Apache-style licensing. Given the rising importance of developers, this isn't surprising: developers just want to get work done without being bogged down by license requirements. It's perhaps not surprising, therefore, that permissive Apache licensing may simply be a way station on the road to no licensing at all.

That's what GitHub seems to be telling us, anyway.

A Trend Toward Extreme Permissiveness

Early in the life of free and open-source software, copyleft licensing reigned supreme. But for years, permissive licenses like BSD and MIT have been climbing, as Redmonk analyst Donnie Berkholz nicely pictures:

Not content to stop there, however, we seem to be entering a new phase: the no-license model. As free-software advocate Glyn Moody notes, "the logical conclusion of the move to more 'permissive' licences [is] one that permits everything."

While Moody talks about public domain software, the GitHub generation seems to be less fussy about legal mechanics.

The GitHub License Black Hole

As Aaron Williamson, senior staff counsel with the Software Freedom Law Center, presented at this year's Linux Collaboration Summit, the vast majority of projects on GitHub don't appear to carry any license terms at all. (The Register's Neil McAllister offers a great summary.) GitHub has become the gathering point for modern open-source development, so it's hugely significant that a mere 14.9% (219,326) of the 1,692,135 code repositories Williamson scanned had a file in their top-level directories that specified a license. 

In other words, the vast majority of code on GitHub isn't necessarily open source. Or proprietary software. Or, well, anything. It's just code.

Redmonk analyst James Governor nailed this trend in 2012, arguing that "younger devs today are about POSS - Post open source software." For such developers, Governor holds, licensing and governance are an afterthought: the code is all. Both Gartner and Forrester find that open source is booming precisely because developers want flexibility.

Less licensing = more flexibility.

Is Licensing Necessary?

Not that this approach is unproblematic. Outercurve Foundation board member Stephen Walli posits that such "promiscuous" sharing without governance and licensing will lead to "software transmitted diseases." But it's unclear that the GitHub generation cares. Maybe they will. Maybe they'll wake up and smell the need for licensing. 

Or maybe the project/company they create will attract the interest of a would-be buyer, and suddenly source code hygeine will matter. A lot. As a Black Duck study shows, open-source compliance is becoming an increasingly common question in mergers and acquisitions:

But all is not lost. Berkholz analyzed a wide array of projects to determine the interplay between project size and licensing. As he summarizes, "as projects grow, they tend to sort out any licensing issues, likely because they get corporate users, professional developers, etc."

License rebels, in other words, tend to become less rebellious as their projects mature.

Ultimately, then, we almost certainly don't face an industry meltdown stemming from uncertain code provenance. Instead, we have a highly permissive license culture that helps to foster the development of code in the early phases of open-source development, which graduates to Apache-style licensing as projects catch on. Lawyers can rest easy. 

Lead image courtesy of Shutterstock.

Ever wonder why it has taken so long for your robot butler to arrive? It's 2013, so why aren't those long-promised robotic domestic servants helping out around the house yet?

One reason for the delay: Robot engineers lacked a common platform on which to communicate and collaborate with one another. Robotic hardware and software systems had to be built from the ground up every time. 

Open Source Robotics

But just as open-source operating systems for computers have amped up digital innovation, the robotics industry has undergone a similar transformation over the last five years. Ever since the advent of ROS (Robot Operating System), an open-source platform on which engineers could build robotic programs and apps, robotic innovation has picked up speed.

On Friday, robot engineers from around the world gather for the second annual ROScon in Stuttgart, Germany. Meanwhile, ROS has become a requirement for several high-profile DARPA (Defense Advanced Research Projects Agency) robotics projects - in this year’s DARPA Robotics Challenge, every contestant will use ROS.

"ROS has also started to appear in job listings and on resumes," said Tully Foote, ROS Platform Manager at the Open Source Robotics Foundation (OSRF). "At robotics conferences and presentations, most people are using ROS on their robots," Foote said, "and those who are not often justify why the are not using ROS if they are not."

“ROS is distinguished by its focus on building a community of collaborators,” Foote added. “From its inception, ROS has been designed to facilitate sharing of the software between members of the worldwide robotics community.”

Leveling The Robotics Playing Field

Steve Rainwater, a robotics expert and blogger at Robots.net, agrees that ROS is today’s "leading software framework for robotics," because it integrates exceptionally well with prior robotics research frameworks. 

“There have been other projects that tried to write complete robot operating systems from the ground up, but where [ROS creators] Willow Garage got it right is they understand how open source works,” Rainwater said. “They invent the parts they need and integrate them with the parts that already exist.”

ROS "keeps the playing field level to an extent between students and hobbyists at one end of the spectrum and governments and universities at the other," Rainwater added. "Improvements to robot software can come from either end of that spectrum and because of the way free software licenses work, everyone's contributions are accessible to benefit the entire community."

How ROS Works

Robotics research center Willow Garage invented ROS to solve the common platform problem. Today, the platform is overseen by the nonprofit OSRF to ensure that is remains easy to share and distribute. 

As an example of how ROS works, imagine you’re building an app. That app is useless without hardware and software - that is, your computer and operating system. Before ROS, engineers in different labs had to build that hardware and software specifically for every robotic project. As a result, the robotic app-making process was incredibly slow - and done in a vacuum. 

Now ROS, along with complementary robot prototypes, provide that supporting hardware and software. Robot researchers can shortcut straight to the app building. And since other researchers around the world are using the same tools, they can easily share their developments from one project to another. 

What Hath ROS Wrought?

The PR2, Willow Garage’s most sophisticated robot (built on ROS), has been prototyped to a variety of apps already. It can walk the dog, fold the laundry, bring you a beer and even plug itself in when it senses its battery is running low. At $400,000, it’s designed for researchers, not customers, and only 60 exist so far. 

“One of the ones I consider most impressive is the PR2 and Rosie making pancakes in Munich,” said Foote. “This is a demonstration of situational awareness, multiple robots coordinating, perception of deformable objects and they are doing it repeatedly with many visitors watching.” 

We're still a long way from affordable, personal robot assistants doing real work in homes and institutions. But the common ROS platform is helping roboticists create workable robot butlers - and many other useful robotics applications - far sooner than would have otherwise been possible.

Images courtesy of Willow Garage.

For years open source and proprietary software camps have fought over which model produces better software. According to Coverity's annual Scan report, released today, both sides are right. And wrong. Depending on how big the code base is.

Coverity's Scan report has long served as the state of the union for open-source software quality, though Coverity analyzes proprietary software, too. In Coverity's 2012 report, which analyzed over 450,000,000 lines of code, both open-source and proprietary software saw an increase in quality, as measured by average defect density (errors found per 1,000 lines of code tested). According to Coverity, this can be attributed in part to an overall increase in organizations that have implemented formal development testing processes for their software code. 

The most interesting part of the report, however, is its analysis of the impact of project size on code quality.  

Both open source and proprietary software had roughly equivalent average defect density rates: .69 for open source and .68 for proprietary software. Open source projects had the highest quality when there were between 500,000 – 1,000,000 lines of code: 70% fewer defects, yielding a .44 average defect density. Proprietary software?  It had the best quality (or, lowest defect density) in projects over one million lines of code, registering a .33 average defect density in larger projects.

For smaller code bases, then, open source shows dramatically better quality. In larger code bases, open source has more defects, but isn't far off from proprietary software: .75 vs. .66.

While there’s no single factor that can explain this phenomenon, it’s likely due at least in part to the fact that open-source projects are often purpose-specific, and maintained by a core group of committed developers. As the projects grow in size and scope, and more developers come on board, there’s a greater hesitancy to make changes to the core kernel for fear of a ripple effect that could adversely impact the larger project.

Conversely, proprietary software projects usually need to get to a certain point of critical mass – somewhere around one million lines of code, if the results of Coverity’s report are to be trusted – before an organization implements formal development testing processes to ensure quality software code. 

Here’s an infographic that encapsulates the main findings of this year’s Scan report:

Lead image courtesy of Shutterstock.

Worried about your longevity as a worker in the fast-moving tech industry? What you need is some inspiration from John Sloan.

Who's John Sloan? He's the man pictured in a photo I used in a recent post on 10 Technology Skills That Will No Longer Help You Get A Job. (See that photo below - or on the iPad in the photo above.) While Sloan may look like a symbol of outdated technology in the older photo, he's actually the polar opposite. As, in fact, the newer photo of him above should lead you to believe.

Sloan, aka Chip Overclock, has not only participated in the many amazing, globe-spanning, nano-shrinking changes in computer tech over the past four decades, he has stayed current with the changes - and kept himself gainfully employed - by taking full responsibility for his own career and professional development.

Pictures Tell The Story

The picture below shows Sloan at Wright State University, Ohio, circa 1976. He would soon go on to earn his B.S. in computer science, and later his master's degree. He's seated next to an IBM System/360 Model 65.

Sloan, now 56, lives in Denver. Several of his friends saw my story and noticed the photo, and forwarded it to him, which prompted him to contact me. Sloan describes his current career as a "freelance product developer specializing in real-time and highly concurrent systems." The photo at the top of this post shows Sloan next to his Mac Mini with Cinema Display. That's a first-generation iPad he's holding - displaying the original picture, no less.

Why John Sloan's Story Matters

Talking to John Sloan made it clear that long-term survival in the tech industry was about much more than just mastering a specific set of skills. Instead, it's all about taking personal responsibility for learning and adapting over the years and decades:

ReadWrite: Getting a computer science degree in 1976 was an awfully forward-thinking move.

John Sloan: It was the closest I could get to living in the kind of science fictional universe that I was reading about and watching on television. I still remember today watching the very first episode of the original Star Trek.

ReadWrite: Are you currently employed?  

John Sloan: I'm a self-employed, offering consulting and contracting through my one-man company, Digital Aggregates, and have been since 2006.

ReadWrite: How do you stay connected to all the changes in the computer industry?

John Sloan: I spend almost all my time (when not working with clients) on professional development. I read, I attend conferences, I learn new skills. This is the kind of professional development that companies simply no longer offer anymore to their regular employees. My company exists not just to service my clients, but to keep me current and employable.

Favorite Tech And Tools

ReadWrite: What technologies have you worked on over the years?

John Sloan: I've had a lot of career success, almost all in the systems area, much of it working low in the software stack, close to bare metal. I've worked on IBM mainframes, PDP-11 minicomputers. Cray supercomputers [all the way] to huge distributed Linux systems. My current gig is developing a tiny PBX with Iridium transceivers that will allow flyers in business jets to make phone calls from anywhere in the world.

John Sloan: My iPhone 5. I have no idea how I'd live without it.

ReadWrite: One of the big tech shifts has been the move to open source. How have you tackled this change?

John Sloan: Open source has been very good to me. For several years now a lot of my income has come from hacking open source software ranging from various portions of the Apache software stack written in Java to the Linux kernel and various portions of the GNU software stack in C.

The economics of open source is the most interesting part about it. Back in the mainframe days, you bought the hardware, and got the software and support for free. Now the hardware is almost free, the software is mostly free, and the support is how a lot of companies book revenue.  

ReadWrite: What about the shift from desktop to mobile?

John Sloan: Every time I pick up my iPhone 5 or my iPad, I feel like I'm in a science fiction movie. Same goes for the Android mobiles. I also have a Samsung Galaxy tablet. It's astounding.

Tech Advice For Young And Old

ReadWrite: Do you have any advice for workers who have been in the industry a long time?

John Sloan: Do not trust your career and professional development to your employer. You absolutely must take charge of this yourself.

ReadWrite: What about advice for those just starting out?

John Sloan: No matter what technologies is being taught when a freshman enters university, they will almost certainly not be the ones being taught when that senior graduates. And whatever technologies that student learns will not be what he ends up needing expertise in when he enters the workforce. Continuous, life-long learning isn't a buzzword, it's a requirement.

People who grasp specific technologies but can't quickly learn new ones on their own are the ones who are going to be laid off or whose jobs are going to be outsourced.

ReadWrite: Anything else you want to add?

John Sloan: Hardly a day goes by in which my wife of nearly 30 years and I don't remark on how lucky we've been. Just the other day one of my friends and former Bell Labs colleagues remarked that she was surprised that she still got paid good money to do what she loves to do. I feel the same way.

Lead image courtesy of Flickr.

Enterprise IT keeps trying to shove the public cloud genie back into a private cloud bottle, but the majority of developers are having none of that, according to Forrester principal analyst James Staten (@staten7), speaking at the Open Business Conference (OSBC) on Tuesday in San Francisco. Interestingly, these cloud-savvy developers aren't newbie troublemakers just getting started in enterprise IT, but instead skew older and more experienced. Perhaps with Twisted Sister cranking on their Walkmans, this rising breed of middle-aged cloud developer isn't "gonna take it anymore."

Which, of course, is exactly how open source made its way in the enterprise.

Open Source: Not So Young But Very Restless

Back in 2002, Boston Consulting Group surveyed (PDF) the open-source developer community to get a feel for the demographics of the movement. While early open-source development was thought to be marshalled by anarchists and free code-loving hippies, BCG's study revealed that the open-source community was actually comprised of experience IT professionals with an average of 11 years of programming experience.

And while the open source ranks weren't filled with Baby Boomers, they also weren't being pushed by Generation Y. The average age was 30 years old. While not exactly middle aged, it skewed much older than expected.

This shouldn't have been surprising. Often, those who have the most value to contribute are more experienced programmers. In addition, such programmers have also been working in enterprise IT long enough to recognize a better, more efficient way of developing software, and to have the job security needed to take a risk on coloring outside the lines of enterprise IT policies.

Cloud As An Antidote To Corporate Bureaucracy

It's therefore not surprising to see cloud computing also driven by experienced developers, rather than newly minted graduates. According to Forrester's Forrsights Developer Survey, Q1 2013, 71% of cloud developers have at least six years of programming experience, and some 11% have been writing code for over 20 years. These aren't novices trying the cloud because it's "cool."

Indeed, delving deeper into Forrester's data, the primary reason developers turn to the cloud is speed of development:

In other words, as with open source, these developers can't be bothered with corporate bureaucracy. In an earlier Forrester survey, developers said the primary benefit of the cloud is that it's the "Fastest way for me to get my project done and deployed." This calls to mind Redmonk analyst Stephen O'Grady's assertion that "Convenience trumps just about everything" when it comes to cloud adoption.

A Race To Capture Middle-Aged Hearts And Minds

Amazon was the first to spot this market, and is now the preferred  Infrastructure-as-a-Service (IaaS) offering for 71% of developers, according to Forrester, with Microsoft Azure (25%) and Google (23%) playing catch-up. Cloud developers overwhelmingly want IaaS because they want "deep platform access" to things like app servers, web servers, and databases, as Staten noted in his OSBC presentation.

Again, cloud developers are not neophytes. They're serious developers who understand core IT infrastructure but want the freedom to get work done without waiting on corporate procurement or legal policies to catch up.

As such, the IaaS platform that best serves this need will win.

Image courtesy of Jørund F Pedersen, licensed under Creative Commons Attribution-Share Alike 3.0 Unported license.

Taking the stage at the Open Business Conference, Red Hat CEO Jim Whitehurst opened the event with a provocative question: is the industry's choice to go open... or die? Whitehurst clearly has a dog in this fight, given that his company is the first open source vendor to reach $1 billion in annual revenue. But he laid out a compelling argument that enterprises no longer choose between competing technology products, but instead must decide between competing innovation models.

And in Whitehurst's mind, there's only one real choice to make: for open data, open source, open APIs.

Credit: ReadWrite

From Vendor-Led Innovation To User-Led Innovation

As Whitehurst argues, commoditization of technology changes the way innovation occurs, moving from vendor-led to user-led. This is the natural progression for any industry, as he highlighted by referencing the automobile industry. While the companies that make engines and steering wheels are unquestionably important, we don't really think about them much. Instead we focus on Toyota, Ford, Volkswagen, etc. as the companies really driving innovation in the automobile industry.

We have seen this play out in technology, With the Web giants driving innovation in mobile, cloud and Big Data. Each of the major Web companies not only uses open source, but depends fundamentally upon it:

Credit: Red Hat

This isn't where open source started, of course. Open source started with existing categories, commodifying well-known categories like operating systems (Unix->Linux), databases (Oracle->MySQL), application servers (WebLogic->JBoss). But a funny thing happens on the way to the market: as more users get involved, they started to move at a faster cadence than any vendor can. Indeed, as Whitehurst noted, open source got to a point in a number of categories where it wasn't merely replicating the state of the art, but advancing the state of the art. 

As such, enterprises no longer pick products. They pick innovation models.

Picking Innovation Models

By choosing an open model, enterprises necessarily make the choice to participate in the future, rather than simply accepting the technologies their preferred vendors hand them. In such a model, even vendors must collaborate with communities, rather than dictate to them. For example, Red Hat can't provide a long-term roadmap because it works collaboratively with the wider community of open source technology users, and can't impose its will on that group.

Nor is open source merely a matter of ones and zeroes. 

Whitehurst said he had asked the CTO of a major Web company why the company insisted on contributing its internally-developed software to the broader open-source community. The response: "If there is anything we can do to make a data center run more efficiently and hence save energy, for example, we have a moral obligation to the world to contribute it back."

While this moral element of open source isn't universally shared, it remains a strong motivator for a significant segment of the developer population.

Who Is Ready For An Open Future?

Not everyone wants to assume such moral obligations, and not everyone wants to give up control to a community. While we're unlikely to see the death of proprietary software anytime soon, we certainly seem to be witnessing a move toward open innovation. Is the future as cut and dried as Whitehurst hypothesized?

Please share your thoughts in the comments.

The Big Data boom has largely been fueled by a simple calculation: Data + Technology = Actionable Insights, Magic Ponies, and Superpowers. The reality, of course, is far more pedestrian, because while Big Data technology has indeed increased our ability to store and process lots of disparate data in real-time, the technology is only as useful the people managing it. As Bill Wise, CEO of Mediaocean, highlights, the costs of getting it wrong increase as our reliance on data grows.

To be clear, we've long been able to query so-called "Big Data." We've had expensive data warehousing and Business Intelligence tools for many years. The great innovation of tools like Hadoop is that they've made such capabilities available as free, open-source tools that run on commodity hardware, essentially paving the way for anyone and everyone to become a data scientist.

Therein lies the problem.

Taking an influential paper on economics and intelligence efforts around the Boston bombing suspects as background, wherein a few missing rows in Excel and a misspelling of Boston Marathon bombing suspect Tamerlan Tsarnaev's name, Wise points out that "data management tools (i.e., the FBI’s systems and Excel) were undone by fairly simple errors," with terrible results. In other words, as much as we may believe Big Data is as simple as "Input data into Hadoop, out come insights!", the reality depends heavily on the people querying that data.

And the bigger the data, the bigger the likelihood we'll read it wrong, as Wise posits:

[M]ore human/data interaction means a lot more room for error (and inefficiency) around increasingly critical data sets - which... can have very serious results... If Big Data can’t fit hand-in-glove with usability and workflow, a lot of the promise of big data will be empty data crunching. That’s not just a problem for getting where we want to be in the evolution of computing. It’s a situation that can lead to bad data management - which translates into bad economics and, sometimes, far worse.

This confirms renowned statistician Nate Silver's arguments that data doesn't speak for itself, but is instead corrupted by our biases. Worse, the bigger the data set, the more noise to sift through: "the noise is increasing faster than the signal. There are so many hypotheses to test, so many data sets to mine - but a relatively constant amount of objective truth."

Often, misunderstanding our data simply means our businesses will run more inefficiently or, at least, no more efficiently than before. But if Wise is correct, getting our data wrong can have disastrous consequences.

Which means, as I've argued before, that we really need to look inside our organizations for "data scientists," because context is critical to effectively querying our data, as well as knowing which data to collect in the first place. It also means, as Kate Crawford argues in Harvard Business Review, "data scientists should take a page from social scientists, who have a long history of asking where the data they're working with comes from, what methods were used to gather and analyze it, and what cognitive biases they might bring to its interpretation." 

In other words, the more data has the potential to impact our organizations, the more humble and circumspect we should become in using it. The consequences of reading our data wrong scale with the volume and velocity of that data.

Image courtesy of Shutterstock.

Two surveys surfaced last week that paint widely divergent pictures of enterprise adoption of open source. But based on the continued rise of open source in the enterprise, only one is likely correct.

The first comes from Univa, a data center automation company that also offers an open-source version of its Grid Engine product. Univa found that while 76% of enterprises surveyed are using open source, a full 75% experience problems running it in mission-critical workloads.

Given that so many enterprises apparently struggle to use open source successfully, one might wonder why so many persist in doing so. Back in 2008, Gartner found that 85% of enterprises were using open source, but even that high number is surely underreporting actual adoption of open source because, according to Forrester, "developers adopt open source products tactically without the explicit approval of their managers."

Conflicted Much?

Fortunately, Univa doesn't leave us to guess how to resolve this seeming conflict between mass adoption and poor quality. While open source is rarely mentioned on its website, the one page that gets a lot of open source mentions presents a highly conflicted view on open source, like the following customer testimonials:

"...we were finally able to switch our focus away from a malfunctioning [open source] Grid Engine."

"If I went to another company that was using purely an open-source Grid Engine, I would take Univa with me to assure this kind of flexibility and security. I know Univa has my back."

And this product pitch:

"Univa Grid Engine is the next generation product that open source Grid Engine users have been waiting for." 

These sorts of statements would be a great way to bash one's competition, but in this case Univa's marketing is designed to bash itself. Or rather, the open-source project upon which it is based. This message carries through in its survey, which found that 64% of enterprises will pay for better quality, which translates to stability (25%) and enterprise-grade support (22%).

"That open-source product we give away? It's not very good! You should pay us instead of using our open-source software" seems to be the message.

Different Survey, Very Different Results

It's a very different message conveyed by the results of Black Duck Software and North Bridge Venture Partners 2013 Future of Open Source survey.  While vendor support was a top-three consideration in 2012 for adopting open source, in 2013 it falls to number 11, well behind competitive functionality, solid security, and better TCO as reasons to use open source.

In fact, this survey finds that "Better Quality Software," which was the fifth-placed reason for using open source in 2011, is now the top reason:

So open source goes from quality nightmare for 75% of enterprisesr in Univa's survey to quality king in Black Duck's survey. What gives?

Reading Between The Lines

Well, vendor motivations may help to sway the kinds of questions asked, and the recipients of the survey itself. I'm not suggesting that either company set out to skew results, but neither data sample is likely purely random.

Still, I'm more inclined to give credence to Black Duck's results, despite it being an open-source management and consulting firm. After all, open source is driving the top-three trends in enterprise computing: Big Data, cloud, and mobile. If enterprises were struggling to make open source work, they wouldn't be using so much of it, and in such business-critical areas.

Which is not to suggest that open source has "won" and all proprietary software is doomed. Indeed, according to a recent Barclays survey of IT executives, a mix of proprietary and open-source software will likely persist for some time:

But let's not kid ourselves: the days of open source failing because of a lack of enterprise support or insufficient quality are well behind us. There is no shortage of quality companies providing support for leading edge open-source software. And there is no shortage of exceptional enterprise-grade open-source software.

The proof? Open source is being adopted in droves. That's really the only number that matters in figuring out whether open source provides high-quality software.

Image courtesy of Shutterstock.

It's happening again. Red Hat, which for years has dominated both the development and monetization of Linux, has turned its code contributing hand to OpenStack, the popular open-source cloud computing project. While Red Hat initially fought OpenStack, today it has become OpenStack's biggest contributor.

This bodes well for Red Hat. And for OpenStack.

Source Code Vs. Source Of Code

To understand why, it's important to understand how commercial open source works. In proprietary software, source code matters. A developer or company writes software, locks it up under a proprietary license and sells the right to use the software. Proprietary software licensing attempts to make digital goods sell like physical goods.

But in open source, being the source of the source code matters most. Since open-source developers essentially give away software for free, the key to monetization lies in being known as the source of the code, such that one becomes known as the best source of support, updates and add-on components for the software in question. 

Red Hat has turned this open source development and distribution strategy into more than $1 billion each year in support for Linux, a project that it heavily influences by contributing roughly double what any other vendor contributes, as the Linux Foundation's annual Linux development report states. It may not seem like much, but Red Hat's contribution rate of 11.9% gives it outsized influence with prospective Linux customers. No other vendor is better able to influence the inclusion of customer requirements in the Linux kernel.

Now the same thing seems to be happening in OpenStack.

Red Hat Gets Behind OpenStack

While OpenStack was once controlled almost exclusively by its founder, Rackspace, today Red Hat has taken the lead on contributions to OpenStack:

Source: OpenStack (Data compiled by Bitergia)

Again, Red Hat's contributions double those of the next two largest contributors, Rackspace and IBM. Again, Red Hat's contributions put it in the pole position to profit from an open-source project.

Particularly OpenStack, which has been criticized as being long on community and short on actual deployments. 

Red Hat isn't particularly concerned with winning popularity contests. As a public company, it needs real customers paying real money for real OpenStack deployments. As such, it has released RDO, Red Hat's community distribution of OpenStack (similar to Red Hat's Fedora project for Linux), and this week Red Hat announced the availability of the Red Hat OpenStack Early Adopter Program, which provides early access to its enterprise-grade distribution of OpenStack, similar to Red Hat Enterprise Linux (RHEL).

It's not hard to imagine Red Hat's customer base extending their RHEL, JBoss, and Red Hat Enterprise Virtualization (RHEV) deployments to the cloud with Red Hat OpenStack.

Just What OpenStack Needed?

After all, this is the playbook Red Hat perfected with RHEL and has put to use selling middleware, virtualization, and now cloud. While Red Hat's involvement offers no guarantee of success, when Red Hat sticks to markets it knows - enterprise infrastructure - using a business model that fits - enterprise hardening of community code - its success rate is pretty impressive. With $35.5 billion at stake in the cloud market, according to recent Gartner projections, making OpenStack work is a big deal.

Since shifting into a true community project, OpenStack has steadily attracted new, active contributors, Red Hat chief among them. The next phase involves making OpenStack safe for the enterprise. Arguably no company has more success turning open source into a safe investment for CIOs than Red Hat. 

As such, Red Hat being number one in OpenStack contributions may go a long way toward making OpenStack number one with CIOs.

Image courtesy of Shutterstock.

M$: Short for Microsoft, used to imply Microsoft cares more for money than it does for security, stability, and anything else that could make a good Operating System." - Urban Dictionary, 2004.

"Microsoft sucks." Too many times, the conversation stops there.

Yes, Microsoft gets plenty of criticism, much of it justified, on everything from its nasty attacks on Linux to the failings of its latest operating system. Along the way, Microsoft has helped write its own narrative as a money-grubbing monopolist - old, litigious, out of touch. For Pete's sake, it took years for Bill Gates to recognize the potential of the Internet.

But that's only part of the story. We too often overlook the many things that Microsoft does right: its philosophy of open research; its willingness to adopt and contribute to open source; even its willingness to admit when it's wrong. The case here is not necessarily what Microsoft does best, but what it does well, what it deserves to be recognized for, and what we generally overlook.

1. More Open Than You'd Think

Microsoft, open?! Really?

Of course not, if you're looking strictly at Microsoft's commercial products. There's no way that you'll ever see Windows released as open-source code, nor will true open-source advocates ever put Microsoft in the same camp as say, Red Hat.

But in 2006, Microsoft began changing its tune toward open-source software - forced by IBM and Red Hat, admittedly. The tide turned when Microsoft and Novell signed a cooperative agreement shielding themselves and non-commercial free software developers from being sued; by 2012, Microsoft had entered the list of the top 20 contributors to the Linux kernel. Linux never really cracked the desktop PC, but Microsoft seems content enabling Linux to run on virtual machines, and possibly even developing Office for Linux, too.

Basically, Microsoft has achieved detente with open-source software; acknowledging its role, using it to Microsoft's own advantage, competing with it on its merits and contributing back to the community, where warranted. Argue all you want how Microsoft arrived here - kicking and screaming, perhaps - but Microsoft's attitudes toward open-source software have significantly improved.

2. Open Research, Too

One of the few companies that opens the doors to its labs is Microsoft, with events like TechFest held each year in either its Redmond headquarters or in its Silicon Valley facilities.

Many companies host developer conferences to engage with partners and announce new products. The difference is that Microsoft seems to emphasize research and showing off the fruits of that research to the world at large. There seems to be a sense of pride there that only a few companies (Intel, for one) seem to share.

Finally, there's the discovery aspect. Google Scholar does a fine job of assisting searches for academic papers, but compare Google Scholar and Microsoft's Academic Search. Not only is Microsoft's tool arguably more interesting than what Google offers, it also allows you to search by organization. Compare Microsoft versus Google versus IBM in terms of citations and papers, and decide whether or not you believe Microsoft's numbers, which show Microsoft publishing much more research than Google.

3. Winning The Game (Console)

In little more than a decade, Microsoft has forced Nintendo, one of the pioneers of the modern video game console, into near irrelevancy. It hasn't managed to do the same with Sony, yet, but its Xbox has outsold Sony's PS3 for well over a year (at least in the United States). Meanwhile, Microsoft continues to innovate with its use of the Kinect peripheral, both as a camera and a form of gesture input. 

Both Sony and Microsoft have struggled to bundle their consoles with music, movie and app stores, in much the same way Apple has. But Microsoft has also kept its eyes open. If the reports of the "Stingray" Xbox are true, Microsoft may be smartly attacking on two fronts: developing a low-cost Xbox derivative to take on Roku and Boxee in the video streaming market, while maintaining its dominance in game consoles. 

4. A Sense of Vision... And Touch

In the last decade, Microsoft has either bought or developed products for productivity (Windows, Office), collaboration and connection (Skype, Lync, Windows Phone) and entertainment (Xbox) and embarked on an ambitious bid to tie them all together.

Microsoft chief executive Steve Ballmer now describes the company as a services-driven organization, but Microsoft is increasingly committed to pushing the boundaries of hardware, whether that be its gigantic Perceptive Pixel displays, its Kinect depth camera, the Xbox or Windows Phones, and its Surface tablet. Microsoft has built outwards from a formidable presence in Office and Windows, adding powerful communication tools in Lync and Skype, and tying together tablets, phones and console together via the cloud. There's no other company in the industry - no, not Google, not Apple - whose software and hardware ecosystem traverses as broad a spectrum as Microsoft.

Microsoft may have fallen short with Windows 8's touch interface, but Kinect is impressive in its own right, even if it was licensed from a startup, Primesense. (I don't know why Microsoft hasn't made a corresponding investment in speech recognition, which would fit so naturally alongside a touch-based interface.)

Microsoft is now the chief steward of the PC, responsible for pushing its boundaries. Tablets, phones and Chromebooks attract the ink these days, but preserving the future of 350 million PCs is no small task.

5. Microsoft Peripherals: So Good, We Forget About Them

Every day, we sit down at our laptop, PC, or other computing device, put our hands to our keyboard and type away. And, in general, many of the best of those keyboards and mice have said "Microsoft" somewhere on them. 

Set aside arty attempts like the Microsoft Arc Mouse. When you get right down to it, Microsoft's basic Comfort Desktop keyboards and basic mice have been under our fingers for years and years. Microsoft's keyboards are one of the reasons many people can't imagine typing on a tablet's sheet of glass for any length of time. 

  6. Investing In The Future

Few companies have the resources to invest in startups, whether that be a company or a teenager. Microsoft does both: efforts like BizSpark give out Microsoft software and assistance to startups, while efforts like the Kodu Cup help kids learn how to code. The Imagine Cup crosses borders to incentivize student innovators develop their own products and the business models to run them. And while Microsoft helps launch solar-powered broadband in Africa, it must know that it's not going to earn a front-page story.

Much of what Microsoft is simply doing here is a high-profile effort to seed Windows and its other products around the world, employing many of the same practices that other technology companies employ. But is "good" being done here? Certainly.

7. Windows Phone: Committed To Being Different

During the holiday season, comScore reported, Microsoft's Windows Phone actually lost market share. Ugh. It's hard to write positively about Windows Phone when consumers obviously aren't falling in love with it. But Windows Phone provides an attractive, easy-to-use alternative to the iOS/Android duopoly, forgoing dozens of static app icons for dynamic "Live" tiles that transform the phone's home screen into a dynamic mosaic of information. It's, well, iconic.

We all know Windows Phone's weakness: apps. What Microsoft hasn't done is convince application developers to embrace the platform, and that's a big reason consumers have shied away. But the Windows Phone OS itself has a lot to recommend it, and the hardware from Nokia and HTC isn't bad, either.

8. Still Owning The Enterprise

For all of its emphasis on consumer-facing technologies, Microsoft's empire was built on productivity and the enterprise. Microsoft's Server and Tools and the Business Division typically report both the highest profits and revenue of any divisions within the company. Microsoft has forged relationships with thousands of businesses, generating stable, consistent revenue streams especially with the creation of subscription models like Office 365.

While Google Apps continues to cut into the Word/Excel/PowerPoint triumvirate, Microsoft has made an end run around Google's services by making collaborative services like Lync the centerpiece of Office.

Does Microsoft need to own the enterprise via hardware like Surface? In the end, no. If Surface doesn't end up as the Bring Your Own Device (BYOD) option for enterprise workers, the rumored Office for Android and iPad will likely serve instead. That's the end goal: capturing attention and generating revenue from enterprises, no matter the medium. As long as that keeps happening, Microsoft can afford to bet on riskier ventures like the Surface and Windows Phone.

9. Microsoft's Mea Culpas

I was honestly impressed by Microsoft's apology that it had fallen short of its commitment to provide "browser choice" to European customers in 2012, as part of a settlement agreement. Microsoft owned up to its mistake, described what had happened and what steps it would take, and again took responsibility for the error when the European Union slapped the company with a $731 million fine.

Google, by contrast, faces fines and a concerted EU investigation after allegedly ignoring requests to rework its privacy policy. Many expected Microsoft to have looked for excuses and appealed the EU's decision. It didn't.

Wouldn't it be nice if Microsoft didn't screw up in the first place? Sure. Microsoft obviously regrets its bawdy song-and-dance routine at a Norway developer conference showed last year, or its "big boobs" gaffe a month later.  But even the world's largest companies make mistakes. A company's character is determined by how it deals with them.

If It Bleeds, It Leads

Failure interests us. Microsoft climbed to the top of the market, creating arguably the world's richest man in the process. Tech journalists remain eager to write the story of Microsoft's fall, me included. Some of Microsoft's tactics are still downright embarrassing: the Scroogled campaign, for example. Windows 8 might well end up as the second coming of Vista. There are still questions whether or not Microsoft can tie its software, services and hardware together into a cohesive whole.

But refusing to acknowledge the other side of Microsoft's story isn't right, either. There's some good work coming out of Microsoft, and ignoring that creates an incomplete, inaccurate picture.

All images courtesy of Microsoft, except Microsoft's Open Source Lab Room by Todd Ogasawara, Microsoft mouse image by Fredric Paul, and roses image courtesy of Shutterstock.

Microsoft may be a distant runner-up to iOS and Android in the smartphone race, and still lags Amazon EC2 in the cloud wars, but executives from the Windows Phone and Azure divisions aren't hurting for respect. In the past week, senior Microsoft executives have joined disruptive challengers in the mobile and cloud markets, suggesting that Microsoft's brainpower isn't lacking, even if its market share is.

The first executive departure was Charlie Kindel, the former Microsoft executive who managed developer outreach for Windows Phone, who actually left Microsoft nearly two years ago but just now found his way to Amazon. While it's still anyone's guess as to what Kindel will be doing at Amazon - given his past role with Windows Phone, some are mooting the possibility that he will be helping build out an Amazon phone - this is becoming a bit of a habit for Windows Phone executives to leave for Amazon.

After all, just last year Brandon Watson, also from the Windows Phone developer outreach team, left for Amazon. In his case, Watson joined Amazon to help on the Kindle cross-platform team.

Nor is it just the Windows Phone team that has been leaking.

Today Red Hat announced that it had snagged Radhesh Balakrishnan, Microsoft's Azure chief for Asia-Pacific, to run its virtualization efforts, including OpenShift, Red Hat's open-source PaaS offering. If Red Hat were to borrow from any competitor to steal a march on VMware in the virtualization market, or Amazon in the cloud market, Microsoft was a great place to look.

After all, Microsoft has been cutting into VMware's virtualization market share for years, and Azure has become a solid #2 to Amazon, as a recent Forrester survey indicates:

One way to look at this is that Microsoft is hemorrhaging talent and must be a sinking ship. But I think this would be an incorrect reading. After all, though Microsoft is late to both the cloud and mobile parties, it's making progress in both, and continues to own the affections of CIOs. 

No, I think the better interpretation is that for all Microsoft's execution issues, it continues to have a bevy of super-smart employees. Amazon and Red Hat certainly seem to think so. 

And for every Microsoft executive that leaves, there are many more who are choosing to stay. If anything, these departures say little about Microsoft's fortunes and instead simply indicate that Amazon and Red Hat may offer exciting options of their own. While it's tempting to assume that executive departures are a clear sign of a company's struggles, reading the tea leaves in this way would put Apple, Facebook and Google, among others, in serious jeopardy.

Most people would love to have that kind of "jeopardy."

Image courtesy of Shutterstock.

Google's pledge to back open patents is a typically Google gesture: an elegant combination of genuine altruism, PR spin and protecting its own self-interests.

Thursday morning, Google outlined the Open Patent Non-Assertion Pledge, which boils down to one statement: "we pledge not to sue any user, distributor or developer of open-source software on specified patents, unless first attacked," the company said.

Google said that it would place ten patents under the aegis of the OPN, all tied to MapReduce and designed to minimize the computing power of large data sets. Google has previously made those technologies available to developers; the most high-profile example of the MapReduce library is Hadoop, designed to run applications across large collections of low-cost, commodity hardware.

Hadoop is licensed using the Apache v2 license, which allows for the modification and distribution of contributed code, without royalty. However, the license applies just to a specific project; the OPN pledge applies to any software, past present or future, that might use those patents. Google also said that the pledge remains in force even if the patents are transferred, which will presumably be apart of the licensing restrictions that are applied if the patents are sold or licensed to another party.

But the key component of the pledge is this one: "The Pledge may be terminated, but only if a party brings a patent suit against Google products or services, or is directly profiting from such litigation," Google said.

He Hit Me First

So far, the patents that Google included under the pledge are fairly innocuous: there's simply no way that Google would ever try to enforce its MapReduce patents, not when Hadoop has become such a critical component within the enterprise.

Over time, Google said, the company intends to expand the patents covered by the OPN pledge to other technologies, and it's here that Google lays the foundation for a future opportunity.

Two of Google's most important products are based upon technology that Google open sourced: Google's Chrome browser and Android. Chrome, based on the Chromium Project, covers both the browser as well as the foundations of Chrome OS. Google's mobile operating system, Android, was also founded as an open source project, and its pervasiveness as both an OS for phones and tablets - as well as forks like the Amazon Kindle - is based upon Google's choice of license.

But, over time, Google has been sued by Apple, British Telecom, Microsoft and Oracle, among others, who have claimed that the company's Android patents infringe their own. 

If Google were to place its key Android patents under the "protection" of the OPN, the company would be able to paint Microsoft and others as bullies. Look, Google would be able to say, we gave the world MapReduce, which is the basis for the Hadoop technology you use to run SkyDrive, Microsoft - and your iCloud, Apple. How can you turn around and sue us over Android? It's not clear whether Google's pledge applies on a patent-by-patent basis, so that a suit contesting one patent under the pledge would allow Google - from a public relations standpoint, if not a "legal" one. And this all assumes that Google will take some of its more controversial patents and place them under the OPN, as well.

Google is keenly aware of the disgust the software community has for patents, and the sort of restrictive lawsuits that not only tie up potential progress, but also force developers and researchers to constantly protect their innovations. Open-source licenses are a great relief: developers can simply develop, sharing their knowledge and improving their lot as a whole.

In keeping with this, Google has elevated the bar for corporate behavior. You can argue that the open-source movement embodied Google's "don't be evil" mantra before Google did. (Linus Torvalds released Linux in 1991, seven years before Google was incorporated.) This is the Google of the self-driving car and Google Glass, innovating for all.

And there is something to applaud here. It's just that Google is looking out for its own self interest, too.

The meek may inherit the earth, but at this rate, DevOps will inherit the enterprise. At least, that's one lesson to take from a Puppet Labs survey of over 4,000 IT operations and development professionals. Whereas developers used to be second-class citizens within the enterprise, today they're taking on new authority and forcing a change in mindset as to how software is developed and deployed.

(DevOps, for those not in the know, is a relatively recent style of collaboration between software developers and IT departments intended to speed the deployment of new applications and services.)

This enterprise shift reveals itself in a number of ways. As 451 Research analyst Jay Lyman illustrates, DevOps communities around Puppet and Chef, two of the industry's most popular configuration management tools, have been booming, as have the commercial opportunities for Puppet Labs and Opscode, the two respective companies behind these tools. But there is one particular sign that DevOps is making waves in the enterprise, Lyman notes:

Further evidence of these tools and practices going mainstream lies in expanded integration and support for Windows management and Microsoft environments, which represent a growing number of customers for CFEngine, Opscode and Puppet Labs.

Small wonder, then, that Puppet Labs found a 26% increase in the rate of DevOps adoption by organizations of all sizes, compared to 2011, with 63% of organizations now indicating they use DevOps practices. This could be easily discounted — "Well, of course a DevOps-oriented vendor survey would find lots of DevOps adoption" — except that one would expect the numbers to actually be higher. Given that Puppet Labs presumably sells to the converted, why isn't the number 100%? Presumably because even among those interested in Puppet Labs configuration management tools, the ambition to deploy DevOps practices outpaces real-world adoption of them.

But that's changing at a 26% clip. More revealingly, such DevOps adoption translates into a 75% jump in job listings: 

Why is DevOps reshaping enterprise IT? Quite simply, because it works. Because IT operations and development are better in collaboration than in competition. As the survey uncovered, high-performing, DevOps-savvy organizations deploy code 30 times faster with 50% fewer failures. And, strikingly, the longer DevOps practices are followed within an organization, the lower that organization's app failure rate and the faster its recovery from failure:

Source: Puppet Labs

While some of DevOps' rise can rightly be credited to a startup ethos and the evangelism of companies like Puppet Labs, it also gets a boost from IBM, ostensibly the most fuddy-duddy company on the planet. RedMonk analyst Donnie Berkholtz notes:

IBM’s people really get it. They understand trends that are happening at the frontlines of tech today in startups and in open-source development. IBM is way out in front on enabling DevOps in big enterprises....A lot of my experience with enterprises is that they’re slow-moving and often lagging trends by years, to the point where it’s nearly laughable, but in this case IBM is definitely a front-runner.

Embedded in that comment is perhaps the biggest reason for the rise of DevOps within the enterprise: it mirrors the rise of open source. Or, rather, follows it. Open source puts developers in charge of their IT. Hence, as we find a massive spike in demand for Linux professionals, and as we see the top job trends dominated by open-source technologies, we should expect DevOps to rise along with it, both in the startup and in the enterprise, with significant benefits for all.

Image courtesy of Shutterstock

For decades the enterprise software industry has grown fat on outsized, upfront license fees coupled with ongoing, high-margin maintenance streams. Cracks in the model have threatened  to dismantle the system for years, as reported by The Wall Street Journal back in 2009, with CIOs chafing at paying for low-value, high-cost maintenance.

But if Oracle's big earnings miss last week is any indication, one of three disappointing quarters over the past two years, the cracks have widened to a chasm. As bellwether for the enterprise software incumbents, Oracle's miss suggests that the legacy vendors may struggle to adapt to the world of open-source software and Software as a Service (SaaS) and, in particular, the subscription revenue models that drive both.

It isn't going to be pretty.

Changing How Vendors Get Paid

This isn't just a matter of improving legacy software products. It's a matter of fundamentally changing how these legacy vendors deploy and charge for software. For example, Oracle's entire cost structure is built around the premise of a hefty upfront license and high-margin maintenance (Over 20% of the license fee). Ever read The Innovator's Dilemma? Clayton Christensen's classic addresses just this sort of inability for established companies to change. It turns out to be brutally hard, and often impossible.

Small wonder, then, that SAP has been raising its maintenance fees, trying to milk more money from its customer base as it faces serious headwinds maintaining its license model against upstart competitors like Workday:

Such actions basically force customers to start looking elsewhere, if they weren't already.

If this were just a matter of technology, Oracle, Microsoft et al. would likely weather the storm quite well. Oracle makes great software. There's a reason it's the enterprise database leader, and by a wide margin (though smaller rivals are gaining in popularity).

But building great technology is not enough. Oracle's peers, from SAP to IBM to Microsoft, also charge for software in this way, and across the industry they've been taking a beating as enterprises look to the improved productivity and OpEx of open source and SaaS. Oracle, for its part, blamed its miss on "sales execution," but as Cowen & Co. analyst Peter Goldmacher points out,

...[W]e have a hard time believing that almost all the legacy software names are suffering from poor sales execution at the same time. We believe the primary issue is a fundamental shift in the technology landscape away from legacy systems towards a new breed of better products at a lower cost both in Apps and in Data Management. Virtually every emerging software trend is having a deflationary impact on spend.

Not everyone sees it this way. Wells Fargo senior analyst Jason Maynard urges investors in Oracle to "keep calm and carry on," and expects Oracle's license revenue to grow 5% year over year. 

Good luck with that. 

Developers Rise In Importance

The problem isn't that Oracle and the mega-vendors have lost their hold on CIO affections. They haven't. The problem is that they have little to offer enterprise developers, who increasingly are the gateway to software adoption. Explaining this shift in his excellent The New Kingmakers, Redmonk analyst Stephen O'Grady argues:

With the rise of open source...developers could for the first time assemble an infrastructure from the same pieces that industry titans like Google used to build their businesses -- only at no cost, without seeking permission from anyone. For the first time, developers could route around traditional procurement with ease. With usage thus effectively decoupled from commercial licensing, patterns of technology adoption began to shift....

Open source is increasingly the default mode of software development....In new market categories, open source is the rule, proprietary software the exception.

The top-down approach, in other words, is losing its currency within the enterprise, as both open source and cloud enable developers (not to mention line of business executives) to get work done without getting permission.

The effect on the mega-vendors is overwhelmingly negative, as Oppenheimer analyst Brian Schwartz posits:

We believe something more secular is occurring as cloud computing increasingly entices CIOs to refresh their legacy IT systems with cloud services rather than infrastructure. Additionally, software purchasing is becoming more decentralized with decision-making power shifting away from IT and weakening the selling advantage as a "one-shop supplier." These trends dampen big-ticket on-premise software purchasing and remain a headwind for the infrastructure vendors.

None of which means the big vendors are going out of business anytime soon. In my years at Novell, for example, I witnessed a serious decline in the company's fortunes, even as revenue remained above $1 billion.

Time To Change?

In fact, Novell is a great example of what might happen to the mega-vendors. Ultimately, Novell had to be bought out and then split into pieces in order for its SUSE business unit, now an independent company, to thrive. SUSE can now support its subscription model without all the overhead Novell's legacy business imposed on it.

The same may well prove true for the other enterprise mega-vendors.

Not all enterprises will be affected equally, of course. Years ago IBM reshaped its business to be more services driven, which allows it to embrace new trends like open source enthusiastically. And even Oracle has built out a considerable cloud business (despite starting years later than it arguably should have), to which it can move current customers. Microsoft has been doing the same, transitioning customers to Office 365 rather than lose out on customers moving to Google Docs.

But the revenue profile for these businesses differs significantly from the traditional license/maintenance business, and it's an open question whether any of these companies will be able to turn the corner in their current form.

The Wall Street Journal echoed this sentiment, suggesting that Oracle's "business is being eroded at the edges by smaller, more focused companies offering newer technology," and, I would add, by the very different business models these firms employ. It's a great time to be in enterprise technology...so long as you're not selling a legacy business model. 

Image courtesy of Shutterstock.

Apple may be that kid who never learned to share, but Microsoft over the years hasn't been much better. While the company has long had a healthy partner ecosystem, if you really wanted tight integration with one of Microsoft's products, you pretty much had to work in Redmond. Microsoft Office worked seamlessly with Microsoft Windows worked seamlessly with Microsoft SQL Server worked seamlessly with Microsoft Sharepoint worked seamlessly with... you get the picture.

Of late, however, Microsoft's underdog status in key markets has made it more amenable to a truly open partner ecosystem, perhaps best exemplified by its open arms to open source.

Nowhere is this more evident than with Windows Azure.

While the old world of Azure looked much like the image above, with Microsoft technology as far as the eye could see, the new Azure looks much different. For one thing, much of the best technology being served up on Azure wasn't written by Microsoft. Really! I'm not joking.

For example, in partnership with Hortonworks, Microsoft has released its first public preview of Windows Azure HDInsight Service, Microsoft's cloud-based distribution of Hadoop, the popular open-source Big Data processing tool. Another example of Microsoft's classic embrace and extend strategy? Nope. This time around, Microsoft promises that HDInsight will be "100%  Apache Hadoop compatible now and in the future."

But Hadoop isn't the only open-source technology included by the Azure team.

More than Just Hadoop

In the olden days, Microsoft would have put all its engineering into supporting its own technologies on a first-class basis. Others might try to catch the Microsoft train, but they'd reverse engineer their way onto the back of the caboose, with just a slight API tweak away from incompatibility. Now it's Microsoft Azure that is adding support for Android, not to mention PhoneGap. All of which follows the Azure team's long-time support for Drupal, various open-source databases, Linux virtual machines, and a range of other open-source software.

"Of course Microsoft supports open-source software on Azure because it's a platform," you argue, "and so Microsoft must support third-party technology as a platform provider."

But that "of course" was lost on Microsoft for years. Through a personal agreement between Bill Gates and Steve Jobs, Microsoft Office came to Mac OS X, but it still hasn't touched Linux. Same with SQL Server. You can get the popular database to run on Linux, but not as a first-class citizen. That's reserved for Windows. 

Windows Azure's Open Community

Beyond directly supporting open-source software on Azure, Microsoft has also opened up its Windows Azure Community Portal to make it easy for partners to add third-party services to Azure, both open and closed. This is a big deal for SMBs and departments within enterprises that have traditionally been Microsoft's mainstay, as BitNami founder and CTO Daniel Lopez told me:

"For customers who are looking to the cloud to run department or workgroup level apps... and who are already customers of Microsoft, the transition to Azure may be simpler and more cost-effective than moving to Amazon. 

"Microsoft has traditionally dominated the SMB market. As SMBs move to the cloud, SaaS cannot meet their customization needs. They need to run their own apps - they just don't want the hassle of running their own servers. Nobody has figured out the 'Application layer' in the cloud yet, but Microsoft is actually in a better starting position than its competitors (Amazon, Google) because it already has a huge installed based and an ecosystem of partners." 

Microsoft, in other words, finally groks "open." In part Microsoft shows this by embracing leading open-source technology like Hadoop or Android, but it's just as clear by its willingness to let partners embrace and extend Azure with other offerings. Yes, Microsoft has long done this with Windows, but it was never a level playing field for some kinds of technology, like open source.

Which is not to say Microsoft has won the public cloud. Today that distinction clearly goes to Amazon Web Services. But while AWS is sexy with the Silicon Valley set, the horde of SMBs and enterprises that have traditionally gone with Microsoft will be looking closely at Azure. Microsoft remains the CIO's top vendor, according to a Piper Jaffray survey. By embracing open source, it stands a chance of being the enterprise developer's top vendor, too. 

The Hadoop market is getting interesting. Last year it was a death match between startups vying to own the heart of the project. Today it's a veritable smorgasbord of big-brand vendors getting involved to ensure they claim a big piece of the Big Data pie. Unlike American youth athletics, not everyone will get to take home a trophy.

Hadoop plays a key role in the burgeoning Big Data market, and represents a $13 billion market by 2017, according to Markets and Markets. (IDC pegs the market much, much lower at $812.8 million in 2016, but its numbers don't seem credible to me as they don't even seem to include Cloudera's sales.) Given that Big Data is hot, and Hadoop's data processing engine sits at its core, there's going to be a lot of money trading hands for Hadoop-related products and services.

Not everyone is going to collect.

SiliconAngle's John Furrier has challenged me on this, arguing that Hadoop is "not a winner take all market." While I, too, can see multiple winners in Hadoop, just as there have been in Linux (e.g., Red Hat dominates license/services revenue, but IBM, HP, and others make arguably more with related hardware, complementary software products, and professional services), markets don't tend toward entropy. They trend toward consolidation.

Today, the Hadoop ecosystem increasingly represents entropy:

• Cloudera, Hortonworks, and MapR remain the early favorites, but with very different approaches. Hortonworks positions itself as the 100% open source player; Cloudera somewhat does the same, but adds in complementary, proprietary bits, mostly around managing Hadoop, to add value to Hadoop (and its top line revenue); and MapR provides a hybrid open source/proprietary Hadoop distribution that swaps out HDFS for its proprietary NFS storage layer.
• EMC Greenplum has been involved with Hadoop for several years, and is set to release a new distribution of Hadoop called Pivotal HD. I've labeled Pivotal HD proprietary, but EMC's Hadoop team has taken issue with this characterization, arguing that PivotalHD is 100% open source, with complementary functionality (like HAWQ) available as add-ons. Point well taken, and I apologize for my misunderstanding. I was wrong, perhaps not surprisingly getting confused by Pivotal HD's product page, which says little about open source. But what seems clear is that customers won't be confused by EMC's value proposition: Hadoop with an advanced SQL query engine to make it easier and more powerful to use.
• Intel just got into the game with its own Hadoop distribution. Basically, you can think of it as Hadoop on (Intel Xeon™ processor, Intel SSD, and Intel 10GbE networking.hardware) steroids.
• For those who don't want to run Hadoop within the datacenter, Amazon offers Amazon Elastic MapReduce (EMR). As of April 2012, EMR was powering over 1 million Hadoop clusters. Presumably this number is much bigger today.
• Many, many others including IBM BigInsights, a range of startups, and more.

Will all of these companies make serious bank on Hadoop? No. Will some of them? Sure.

Ultimately, the winners in Hadoop will be those that invest most heavily in its success, as they will be perceived as the companies best positioned to help would-be customers succeed with Hadoop's complexities. But how they invest is up for discussion. Code to Apache Hadoop? Value-adding extensions?

Success isn't about open source purity, as Gartner's Merv Adrian posits: it's about making customers successful. As we saw with Linux, where Red Hat is both the top contributor to the Linux kernel and the company that harvests the most revenue from distributing Linux, contributing code is a great way to signal to the market that you're a leader and capable of getting code fixes to support customers. Code matters.

But code contributions are not the only way to demonstrate leadership and attract customers. Ultimately, companies that make it easier to get value from Hadoop will win big. There may be more than one such company. Indeed, there almost certainly will be. 

But there won't be 20 of them. Or even 10. Enterprise IT is simply not going to be able to manage a polyglot Hadoop distribution ecosystem. That's not the way markets work. No one wants to be "long tail" vendor, and customers don't want to buy from them, either, as Hugh MacLeod humorously points out on Gaping Void:

Source: GapingVoidArt. Used with permission.

The Hadoop market over the next year is going to be hugely interesting. And bloody.

Image courtesy of Ehab Othman / Shutterstock.

Hadoop, nearly synonymous with Big Data, has many failings. But open source is not one of them. In fact, Hadoop's open-source license remains one of its biggest draws, giving enterprises plenty of reasons to persevere in using it despite its shortcomings. It's therefore hard to see how EMC's new Pivotal HD, essentially a proprietary distribution of Hadoop, can hope to succeed.

Not that everyone agrees with this statement.

Dan Woods, CTO and editor of CITO Research and a contributor to Forbes, argues that embedding Hadoop into EMC Greenplum's massively parallel processing (MPP) database (HAWQ) offers CIOs and CTOs the simplicity they need to be successful with Hadoop. He has a point: Hadoop is complex and somewhat hard to use, which is why Cloudera CEO Mike Olson has argued that most of the world will experience the power of Hadoop through applications, nearly all of which will be proprietary, I might add.

But Olson's argument differs from Woods' argument in at at least one major way: Pivotal HD is enterprise infrastructure, not an application, and enterprise infrastructure is increasingly open source.

There are plenty of reasons for this, but RethinkDB's Alex Popescu nails one critical factor:

Hadoop is so successful despite its complexity [because i]t allows experimenting and trying out new ideas, while continuing to accumulate and storing your data. It removes the pressure from the developers. That’s agility. It’s highly appreciated.

In other words, a big reason for Hadoop's success is its open-source license, which permits a hefty amount of experimentation without having to get an enterprise license from EMC, Oracle, or any of the other incumbent infrastructure vendors.  

EMC's Scott Yara tries to deflect criticism of its proprietary foray into Hadoop by declaring "We're all in on Hadoop, period," but as 451 Research analyst Matt Aslett counters, "I have no doubt that EMC Greenplum is 'all in' on Pivotal HD, but that’s not the same thing at all."

Take this away by building a proprietary Hadoop distribution, and EMC has basically erased the very thing that made Hadoop workloads proliferate in the first place. EMC also cuts itself out of the standard adoption cycle for Hadoop, as Redmonk analyst Stephen O'Grady suggests, "Certainly there will be customers whose needs will dictate the adoption of a unique solution like Pivotal HD, but how many will that be relative to the segment whose adoption cycle begins with the download of one of the free Hadoop distributions?"

Today, Hadoop is one of the industry's hottest job trends. Even in absolute job numbers, it's about to pass EMC-related job posts:

Hadoop,emc Job Trends

Hadoop jobs - EMC jobs

Enterprises aren't hiring for EMC's brand of Hadoop. They're hiring for the open source Hadoop. This matters.

Perhaps EMC feels that Hadoop's brand is big enough now that enterprises essentially understand it and are ready to move on from experimentation to full-scale adoption. In this EMC is likely to be disappointed. According to recent IBM survey data, only 6% of enterprises have two or more Big Data projects underway (likely, though not explicitly, involving Hadoop in some way), and a mere 22% are running pilots to test the efficacy of their Big Data strategies. Everyone else is in full-on planning mode.

By creating a proprietary Hadoop distribution, EMC just dramatically limited its access to the 94% that are still in Big Data education and trial mode. Yes, it has a gargantuan sales force. No, they're simply not going to be able to reach would-be customers as efficiently as an open-source distribution model does.

But maybe EMC hasn't gone proprietary to more effectively monetize Hadoop interest, and instead sincerely believes, like Woods ("open source development has its limits"), that complex infrastructure problems are a poor match for open source. History has not been kind to such thinking, as Aslett sarcastically implies:

"Enterprise products" always prevail over open source. onforb.es/VgQ0Cq That's why Linux has been such an abject failure versus Unix.

— Matt Aslett (@maslett) March 1, 2013

EMC has seemingly bottomless resources to throw at Hadoop, and every incentive to do so. It's a smart, highly successful company and no doubt will prove successful with Pivotal HD. However, I can't see it ever dominating an open-source infrastructure market with a proprietary distribution. Open source is the foundation for today's most interesting markets, from Big Data to mobile to cloud computing. It's unlikely that EMC will somehow stem this tide with a proprietary product, no matter its short-term performance or functionality advantages.

It's hard to believe, but just 10 years ago open source was considered "communist," an anti-American cancer and the terror of corporate legal departments everywhere. Now, as a recent Harvard Business Review article reports, it's business as usual. Mundane, even. Developers are to blame. Getting projects done proved far easier with open-source software than protracted licensing negotiations for proprietary software.

Now this same human desire to evade bureaucracy and foster productivity is driving broad adoption of cloud services, and this time it's not just developers driving adoption. Lines of business executives are also encouraging Shadow IT to skirt enterprise policies. The catch is that it is also introducing all sorts of security and management quandaries. Skyhigh Networks, a new startup launching today and funded by Greylock, aims to give CIOs insight and some control over these cloud services without becoming a bottleneck on productivity.

Data Security And Shadow IT

Today, services like Dropbox, for storage, and Remember The Milk, for task tracking, are used by employees in most Global 2000 enterprises, but rarely with the approval, or even knowledge, of corporate IT departments. Indeed, the use of cloud-based services is by now so widespread that corporate IT really has little insight into the pervasiveness of these cloud services within their organizations, as cloud pundit Ben Kepes has blogged.

Cloud services help achieve a goal which companies strive for, efficiency. However, cloud services can also be problematic, as they bypass the corporate infrastructure. No big deal, right? Wrong. For example, services may have user agreements that, clicked through quickly by impatient users, transfer IP-ownership over any data stored through their services. Cloud services may also store data in a way that allows information to be hacked by dangerous third parties.

Like the enterprise IT reaction against open-source software 10 years ago, there is a reflexive urge to just say no to cloud services and try to shut down all but a short list of approved ones. Good luck with that. It’s too late to try and cap cloud service adoption. A better approach is to get visibility into all the services running today, put in policies for their safe use, and let employees choose the services that work best for them. Just as happened with open source. Once IT got open-source software blessed by the legal department, with the risks understood and managed safely, there was no looking back. It’s everywhere now. 

Giving The CIO (Some) Cloud Control

Enter Skyhigh Networks. Skyhigh lets IT say "yes" by giving an enterprise visibility into all the cloud services employees are already using (usually 10X what IT has already formally approved), and providing tools to manage any risks and enable adoption.  

Skyhigh offers a cloud-based service that is frictionless to central IT and user experience. Give it 30 minutes and it will cleverly discover all third-party cloud services employees have running (e.g., Dropbox, Box, MindtheMilk, Salesforce.com, etc.), and show you a list of the most dangerous. Skyhigh already has risk profiles on more than 2,000 of the most popular enterprise services using risk metrics from the Cloud Security Alliance, which rates 50+ parameters for both security and legal risk. When IT sees everything running, it can put in place governance policies for employees to safely use all but the most dangerous cloud services. Instead of the CI"no" you have the CIO as guide, enabler, trusted adviser. 

Want a free pass to try Skyhigh? Here you go.

Going forward the service will also analyze use of these services for anomalies, powered by a Hadoop cluster engine. If a service or user behaves outside their norm, Skyhigh can alert IT. Finally, Skyhigh can also meter usage to compare to one's subscription agreements. One Skyhigh customer found that it was using 5,000 fewer Salesforce.com seats than it had purchased. That quickly translated into money in the customer's bank.

Skyhigh is currently being piloted by a handful of Fortune 20 companies, half-a-dozen Fortune 100 companies and another dozen 'normal' size companies like Netflix and Lucile Packard Children's Hospital. In each case, CIOs thought they had 25 to 70 cloud services running. An educated guess, but wrong. The fewest they found was 200+ and in two instances 2,000+, one of which includes a company known for its buttoned-down approach to security.

There's no going back on cloud computing. Like open source, it's here to stay. But also like open source, the best way for enterprise IT to confront the cloud is to understand and enable its sensible use. Skyhigh may offer a serious step toward that goal.

Image courtesy of Shutterstock.

If you're like most enterprise IT professionals, you have serious concerns about cloud computing. According to a new Lieberman Software 2012 Cloud Security Survey, sponsored by the Cloud Security Alliance, 88% of the 300 IT professionals surveyed believe that some of their data hosted in the cloud could be lost, corrupted or accessed by unauthorized individuals. That's likely why 86% keep their most sensitive data behind-the-firewall.

Despite those concernse, though, an equally whopping 86% believe their cloud deployment has been a success. It's therefore not unreasonable to suspect that 100% will be back at the cloud computing trough, again and again and again.

That is really good news for Amazon, which is looking to double down on selling its cloud services to the enterprise.

Amazon Hires an Army

In a bid to drive enterprise adoption, the cloud leader looks set to nearly double its AWS salesforce, as Business Insider discovered. No doubt these salespeople will focus on moving enterprises to use Amazon Web Services beyond the test and development workloads currently in the cloud, currently the dominant type of workload enterprises cede to AWS. The goal, clearly, is to get them to move mission-critical applications to AWS.

This could prove harder than it first appears, given enterprise insistence on tight Service-Level Agreements (SLAs) and the big differences between enterprise and consumer cloud requirements, as Wikibon's Kristen Feledy posits:

IT organizations are under tremendous pressure to cut costs and the “Amazon Effect” increases that pressure. The reality is many of the successful public cloud examples are characterized by a single application accessed by millions of people; whereas the traditional enterprise is made up of hundreds or even thousands of apps accessed by thousands or maybe tens of thousands of users. These are different worlds where the former is all about scale and simplicity and the latter emphasizes service levels, reliability and security.

Amazon, after all, has mostly taken a somewhat blasé approach to SLAs, which Wikibon describes as "we’ll do our best – if we don’t please send us an email."

Not exactly a confidence booster for the cloud-wary CIO.

Shadow IT: First Open Source, Now Cloud

This is changing. Amazon recently rolled out premium support plans for the enterprise, including "white glove case routing." To sell a premium AWS experience, Amazon's job profiles scout for sales professionals who "possess both a sales and technical background that enables them to drive an engagement at the CXO level as well as with software developers and IT architects." In other words, Amazon recognizes that it is the developers and architects that pull cloud computing into the enterprise, but it is the CIO who will bless this "shadow IT."

After all, it is shadow IT that has been selling the enterprise on Amazon for years. By now, just every enterprise is using the cloud, be it from Amazon, Microsoft, Rackspace or others, as progressive IT professionals have looked to the cloud to get things done despite friction from internal bureaucracy. For those paying attention, this is precisely how open source succeeded: currying favor with developers until its spread was so pervasive within the enterprise that CIOs were forced to accept it, and signed sales contracts with Red Hat and others to mitigate legal risk and improve service.

Hence, Cloudscaling's Michael Grant is arguably correct to suggest that rather than fight shadow IT and its inexorable march to the cloud, CIOs should recognize shadow IT as "a forward thinking testbed for IT innovation." If Grant is right, that testbed suggests a future in the cloud, both for dev/test and mission-critical workloads, driven by the promise of higher convenience and lower costs, but really about increased innovation.

But it also suggests that Amazon has been right to first focus on the enterprise's new kingmakers: developers. This is how open source won. It's how the cloud is winning, too.