Home Study: Open Source Delivers Superior Quality… Up To A Point

Study: Open Source Delivers Superior Quality… Up To A Point

For years open source and proprietary software camps have fought over which model produces better software. According to Coverity‘s annual Scan report, released today, both sides are right. And wrong. Depending on how big the code base is.

Coverity’s Scan report has long served as the state of the union for open-source software quality, though Coverity analyzes proprietary software, too. In Coverity’s 2012 report, which analyzed over 450,000,000 lines of code, both open-source and proprietary software saw an increase in quality, as measured by average defect density (errors found per 1,000 lines of code tested). According to Coverity, this can be attributed in part to an overall increase in organizations that have implemented formal development testing processes for their software code. 

The most interesting part of the report, however, is its analysis of the impact of project size on code quality.  

Both open source and proprietary software had roughly equivalent average defect density rates: .69 for open source and .68 for proprietary software. Open source projects had the highest quality when there were between 500,000 – 1,000,000 lines of code: 70% fewer defects, yielding a .44 average defect density. Proprietary software?  It had the best quality (or, lowest defect density) in projects over one million lines of code, registering a .33 average defect density in larger projects.

For smaller code bases, then, open source shows dramatically better quality. In larger code bases, open source has more defects, but isn’t far off from proprietary software: .75 vs. .66.

While there’s no single factor that can explain this phenomenon, it’s likely due at least in part to the fact that open-source projects are often purpose-specific, and maintained by a core group of committed developers. As the projects grow in size and scope, and more developers come on board, there’s a greater hesitancy to make changes to the core kernel for fear of a ripple effect that could adversely impact the larger project.

Conversely, proprietary software projects usually need to get to a certain point of critical mass – somewhere around one million lines of code, if the results of Coverity’s report are to be trusted – before an organization implements formal development testing processes to ensure quality software code. 

Here’s an infographic that encapsulates the main findings of this year’s Scan report:

Lead image courtesy of Shutterstock.

About ReadWrite’s Editorial Process

The ReadWrite Editorial policy involves closely monitoring the tech industry for major developments, new product launches, AI breakthroughs, video game releases and other newsworthy events. Editors assign relevant stories to staff writers or freelance contributors with expertise in each particular topic area. Before publication, articles go through a rigorous round of editing for accuracy, clarity, and to ensure adherence to ReadWrite's style guidelines.

Get the biggest tech headlines of the day delivered to your inbox

    By signing up, you agree to our Terms and Privacy Policy. Unsubscribe anytime.

    Tech News

    Explore the latest in tech with our Tech News. We cut through the noise for concise, relevant updates, keeping you informed about the rapidly evolving tech landscape with curated content that separates signal from noise.

    In-Depth Tech Stories

    Explore tech impact in In-Depth Stories. Narrative data journalism offers comprehensive analyses, revealing stories behind data. Understand industry trends for a deeper perspective on tech's intricate relationships with society.

    Expert Reviews

    Empower decisions with Expert Reviews, merging industry expertise and insightful analysis. Delve into tech intricacies, get the best deals, and stay ahead with our trustworthy guide to navigating the ever-changing tech market.