Companies and projects focusing on large-scale collaboration might want to start thinking about collaboration in a new way. Clay Shirky, author of Here Comes Everybody closed out the second day of LinuxCon North America 2011 with a contrarian look at collaboration. While many treat collaboration as a “love fest” or harmonious interaction, Shirky put forward the idea that productive methods of fighting are the most successful, particularly in open source.
Shirky, who also teaches at New York University, started talking about his “favorite bug report ever.” The bug report, for Firefox (#330884), was a corner case where Firefox would show any user what sites that should never save passwords even if selected by another user.
In other words, if one user chose not to save passwords for a dating site, their fiancé could also see what sites Firefox should not save passwords for. The bug was filed by a woman who discovered (through the Firefox bug) that her significant other had been perusing dating sites and consequently ended a five-year-old relationship. The bug report not only contains the technical issue, but also the social impact of the bug, and Shirky describes the rest of the comments on the bug report as a mix of technical discussion and relationship advice, and in some cases relationship advice given as one might give techincal advice.
The bug report, says Shirky, demonstrates that you can never disentangle the technical from the “soft human squishy stuff.”
Then Shirky tied in the birth of GNU and Linux to the idea that “the whole world is your talent pool,” (a quote from Eric Raymond’s “Cathedral and Bazaar”) and his main ideas about collaboration and cognative surplus.
Linux not only produced an important technical work, but promoted the idea of mass collaboration taken up by things like Wikipedia. “Tiny changes of behavior by small fractions of the population” can make a huge difference, says Shirky. Why? Because we have a “cognative surplus” — when the whole world is your potential talent pool, you can see incredible successes. And silliness, of course, such as LOLCats. Shirky noted that the Internet is efficient at creating LOLCats and other silliness, and it takes dedication and interest to produce useful results. This is nothing new, says Shirky. The printing press produced erotica more than 100 years before the first scientific journal.
The printing press, and other modes of communication, are not only not immediately and mostly used for work – they also don’t lead to world peace. Says Shirky, “here’s a hint about what happens with media revolutions. Not world peace. When you vastly increase people’s ability to communicate with one another… means more fighting.”
The question, says Shirky, is how to structure that fighting to lead to productive results. We’re not good at subjecting our own beliefs to scrutiny, says Shirky, but we’re very good at scrutinizing others’ beliefs and work. He used the examples of chemistry versus alchemy. Alchemists hid their work, chemists published and discussed their work – a form of structured fighting. We know which field has led to better results.
Managing Collaboration
Here Shirky started talking about the importance of managing collaboration effectively. Large collaborative projects aren’t, in fact, large collaborative projects according to Shirky. They’re small collaborative projects with tight groups, that integrate very large amounts of small participatory effort. To put it another way, projects like Wikipedia and the Linux kernel may have thousands of contributors — but it’s a small core of contributors who do the bulk of the work and integrate the work from others who only contribute a small amount.
It’s also important, says Shirky, that people cannot join the project too easily. Even given the presumption that all the participants have goodwill towards the project, he says that it shouldn’t be too easy to change every aspect of a project. Some parts of the system should be easy to change, some parts should be hard.
As an example of putting barriers to contribution, Shirky pointed to StackOverflow. In order to test whether a new site would work well, they don’t just let people try anything to see what sticks. People have to propose a site, demonstrate the type of questions, and then recruit users. Sites that don’t make the criteria don’t launch, but the ones that do launch tend to be more successful.
New users can’t participate easily either. Shirky talked about the karma system for StackOverflow, which requires users to show up and listen at the beginning – and evolve to an admin-level status that can manage the site.
Another pattern, Shirky says, is getting the community to care. When you can do that, good code is a side effect. Here he returned to the Linux kernel. Kernel management shifted to tarballs sent by email to code tracked in BitKeeper, and then back to tarballs by email when BitKeeper turned out to be unsuitable. Eventually, Torvalds got tired of working that way, but rejected other code repository systems because of the influence they impose on the politics of the kernel. (Remember the problem of separating technical problems from human ones?) Torvalds wanted a system that “lived up to the promise of the GPL,” where it was easy to fork code, but made merging code a community decision.
Git was another revolution. Shirky pointed out that source control systems had been around for decades, but Git exploded in popularity and found uses outside of simply tracking code. Git spawned GitHub and a number of clones, and attempts to use Git to track things that were not code. For instance, Shirky pointed to Open-Knesset an attempt to scrape Israel’s parliament for proposed laws, start a branch, show how it should be re-written and debate the changes.
Will it work? Shirky says probably not. “It’s audacious enough it probably won’t work… but audacity beats predictablity over the long haul.” Even if Open-Knesset doesn’t work, Shirky says he thinks we’re going to see more of these attempts. “If the Internet means more fighting… we’re going to have to get good at it to get the value out of the system. What we’re seeing now with projects like StackExchange and non-code uses of Git… apply fighting in open source to the rest of the world.”
Shirky’s talk was one of many at LinuxCon to suggest that the impact of Linux is far beyond that of the kernel or operating system. The key impact is not simply what Linux is, but what it enables and has inspired – innovation and new forms of collaboration for IT and beyond.
(I’d like to thank the Linux Foundation for paying for my travel to LinuxCon North America 2011.)
