Home Distributed Development: You’re Still Doing It Wrong

Distributed Development: You’re Still Doing It Wrong

Distributed computing is the future of technology, but you’d never know it from how many tech organizations continue to operate. While our systems run across multiple servers and even data centers, our development teams too often sit within the same office. While there are real benefits to a co-located development team, the costs of competing for local talent may well outweigh them.

Ironically, open source has taught us how to effectively work in a distributed fashion, but far too few organizations have heeded its lessons. 

The Costs Of Homogeneity

Facebook is the world’s largest open source company, as I wrote recently. If any company groks open source, it’s Facebook. 

See also: How Facebook Will Fuel The Next Wave Of Open-Source Business

And yet Facebook has a very 20th century approach to development. When my company was acqui-hired by Facebook a few years back, Facebook had little interest in any engineer that wasn’t located within Silicon Valley. Move to the Valley or move on, was the mantra.

There are good reasons for this mindset. Studies have found that co-located engineering teams can be more productive, with less time needed to coordinate resources, make decisions, etc. Embedded in that decision, however, is a tradeoff, which Jeff Waugh uncovers in reference to why companies elect the co-located or distributed development approach:

Not only does distributed development yield a greater range of people, but it also provides access to talent at far lower cost. This is a big deal given the escalating cost of recruiting and hiring engineers. Seen the price of an engineer in Silicon Valley these days?

In an email conversation, former MySQL and ZenDesk executive Zack Urlocker pushed the advantages of distributed development. Not only is it “much easier to hire when you have distributed teams,” he told me, but it turns remote locations into real offices and “not just sales outposts,” which is good for both productivity and morale beyond the engineering team. 

But set aside the costs for a moment. Can distributed development actually work?

What Open Source Teaches Us

For those in the open source world, this is almost a silly question. Some of the world’s greatest software, from Linux to Hadoop, gets written by disparate engineers huddled over their laptops in far-flung locations. By its very nature, open-source development favors online collaboration on code, with all communication happening on mailing lists, chat channels and more. 

In the open source world, you can collaborate with your colleague sitting next to you, but any relevant communication needs to happen online, or the model falls apart. If anyone is privileged by sitting next to their peers in the office, the model breaks. 

See also: GitHub For Beginners: Don’t Get Scared, Get Started

GitHub can help. Increasingly, development happens on GitHub, brilliantly explained in non-geek speak by Lauren Orsini. GitHub works so well because it allows developers to work on their own time, forking and merging code asynchronously, either collaboratively or competitively. 

Your Mileage May Vary

Of course, the subtext here is that distributed development works better when the software architecture allows it, not to mention company culture. As Charles Wise posits, “Tightly integrated apps are much tougher” to pull off than systems that involve modular architectures. Given the increasing prevalence of open-source code emerging from tech titans like Facebook and Google, perhaps the default should be modular architectures that are a natural fit for distributed development?

Even with tightly integrated systems, companies have shown that it can work, and well. Marten Mickos, a veteran of MySQL, which had a very distributed workforce, and current CEO of open-source cloud company, Eucalyptus, emailed me some thoughts on how Eucalyptus works:

We have software developers in China, Bangladesh, India and all over the US. A third are located in Santa Barbara where we have an office. The rest work from their homes or collaborative workspaces.

Everything we do is online. We use Jira, Confluence, Github, wiki pages, email, IRC and so on.

Our developers have small private (and personal) clouds in order to be more productive.

We do regular online meetings for specific teams, specific features and specific releases. We follow an agile model, shipping major feature releases twice a year and maintenance releases every second month.

A lot of discussions happen on email. We have an alias for fun@ and one for life@ and discuss@ and biz.intel@. On these lists, people can exchange information and debate various topics. And if they are busy, they can of course ignore those emails.

Because of our model and our commitment to openness, we have been able to recruit some of the absolutely best engineers in the field of distributed systems. The fact that they don’t have to move to wherever the HQ are is a huge plus from them.

This has been my experience as well, having worked at highly distributed teams at Canonical and Nodeable and more tightly coupled teams at Alfresco and MongoDB. Both work, though in different ways. 

What Works For You

Ultimately, though, whether distributed development can work for your company comes down to—surprise!—the particular nuances of your company, as Twitter’s open source guru, Chris Aniszczyk suggests:

And:

All true. But my modest proposal is that many companies haven’t even tried to experimented with distributed development, and they should. As Urlocker insists, co-located development is “good but it’s limiting. Over time there is tremendous advantage to having multiple Dev teams in multiple cities.”

Better, more diverse talent at less cost. That sounds like a winning formula.

About ReadWrite’s Editorial Process

The ReadWrite Editorial policy involves closely monitoring the tech industry for major developments, new product launches, AI breakthroughs, video game releases and other newsworthy events. Editors assign relevant stories to staff writers or freelance contributors with expertise in each particular topic area. Before publication, articles go through a rigorous round of editing for accuracy, clarity, and to ensure adherence to ReadWrite's style guidelines.

Get the biggest tech headlines of the day delivered to your inbox

    By signing up, you agree to our Terms and Privacy Policy. Unsubscribe anytime.

    Tech News

    Explore the latest in tech with our Tech News. We cut through the noise for concise, relevant updates, keeping you informed about the rapidly evolving tech landscape with curated content that separates signal from noise.

    In-Depth Tech Stories

    Explore tech impact in In-Depth Stories. Narrative data journalism offers comprehensive analyses, revealing stories behind data. Understand industry trends for a deeper perspective on tech's intricate relationships with society.

    Expert Reviews

    Empower decisions with Expert Reviews, merging industry expertise and insightful analysis. Delve into tech intricacies, get the best deals, and stay ahead with our trustworthy guide to navigating the ever-changing tech market.