Picture if you will, a collaborative site that runs on two servers, is managed by four people, and has attracted a third of its target demographic within six months of launch. A site that has had 800,000 posts submitted by its users in its short lifetime and has 16 million pageviews/month - and growing.

This is the story of Stack Overflow, a free question and answer site built by developers for developers that has fostered a strong and committed online community in under one year. How? Easy, according to founder Joel Spolsky; all it takes is an understanding of anthropology and a lot of determination.

"As we move from the era of computing into the era of the Internet, we no longer need to worry about computer-human interaction." Joel Spolsky told a group of programmers at Google last month. "What we do have to think about [in the era of social networking] is human to human interaction," he said. And according to Spolsky, to do that, you have to think as an anthropologist does.

Anthropology and the Social Web

"In anthropology it's very clear that the environment that you create influences people and how they behave", Spolsky explained. "People will come into the environment and behave according to what you built in certain subtle ways; ways that you probably didn't think about."

He points to the Scalinata della Trinita dei Monti, or the Spanish Steps to further his point. "They were built to be stairs," he said, "from the Spanish Embassy at the base, up to the Trinita dei Monti at the top." Instead, they've become the "living room" for backpackers in Rome. "Partially it has to do with the steps being the perfect comfortable height to sit on," he said, but also, they provide a fantastic view of the Piazza di Spagna at the bottom.

"This was completely non intentional," he explained. "Similarly, the user interface you create for your applications will influence how people behave."

So what is Stack Overflow and why Does it Matter?

Founded by Joel Spolsky and Jeff Atwood, Stack Overflow is a free question and answer site designed to help developers get the most relevant answers from peers - and fast. Collaboratively built and maintained by a legion of committed developers, it is OS and language agnostic.

Stack Overflow came about because search engines are failing in a particular realm said Spolsky: "Expert Q&A; where you can ask an expert and the expert can give you a true and correct answer." And while he points out that a lot of the companies organized around search have tried to make question and answer type portals; no one site provides value.

And according to Spolsky, there are a variety of reasons why these sites are ineffective: Yahoo! Answers attracts too many adolescents seeking answers to questions about "reproductive sciences;" Mahalo Answers, the brainchild of Jason Calacanis, creates an environment where questions appear to be "scams;" AskVille, Amazon's creation emphasizes the question rather than the answer with its oversize version of a search box. Of course, there is one more; the one that must not be named, and perhaps the unofficial Raison d'etre for Stack Overflow, is basically just a big sneaky tricky mess.

Why Search Engines are Failing when it Comes to Collaborative Sites

According to Spolsky, there are certain reasons why search engines are failing when it comes to Q&A sites, and they are the same issues Stack Overflow is trying to solve.

  1. Sign-up scams: Sites that a search engine may send you to where you must first sign up and pay, if you want an answer.
  2. Register: A "road bump" that many sites have, and one Spolsky thinks reduces participation dramatically
  3. Wrong answers: When searching for highly technical questions, a search engine may send you to a forum that has multiple answers. If you are unsure which answer is the correct one, you waste too much time working through the wrong ones.
  4. Obsolete results: Google, for instance, will oftentimes give an older page priority. In turn, the page you are served is often outdated and no longer relevant.

The Nine Building Blocks of Social Engineering

To work around these problems, Stack Overflow was built on what Spolsky calls the nine "building blocks" in an effort to create a site that was anthropologically correct and would encourage people to behave in a way that would work. He also pointed out that every single one is copied from somewhere else.

  1. Voting: Copied from Reddit, via Digg, voting allows people to vote up answers they think are good. Stack Overflow tweaked its voting algorithm, giving the person who asked the question special power to select one answer as the official answer that will rise to the top regardless of what the community voted. The second answer, of course is always the highest ranked community answer.
  2. Tags: Tags allow users to specify perspective. For instance, Spolsky explained, "you can add that I'm asking this from a VB perspective, not a C# perspective." Stack Overflow is also customizable with tags, allowing users to specify which technology they are interested in, and typical of most social sites. What is not typical however, is the ability to ignore tags that Stack Overflow has built in.
  3. Editing: Taking a page out of Wikipedia, Stack Overflow allows users to edit both questions and answers; so answers could get better, rather than becoming "this frozen artifact on the Internet until the end of time," which is typical of most forum threads.
  4. Badges: "A soldier will fight long and hard for a bit of colored ribbon," said Napoleon once upon a time, and so Stack Overflow made the decision to reward its users with badges. Over time, the badges show credibility.
  5. Karma: People are willing to do for free what they're not willing to do for small amounts of money according to Spolsky and by offering karma, Stack Overflow encourages its users to do more. More Karma equals more privileges on the site.
  6. Pre-search: Once you begin typing your question, Stack Overflow's pre-search will do a quick search to see whether the question has been asked before and display the result for easy access and to prevent duplication issues.
  7. Google is UI: Stack Overflow was built around the assumption that people will go to Google which will send them to the right page. Each URL has the name of the question; each URL is permanent and clean, Metatags, sitemaps; anything and everything was done to ensure Stack Overflow's pages looked "reasonable to search engines."
  8. Performance: Ensuring answers are provided super fast was imperative. As a result, Stack Overflow is built on a Microsoft stack. "This entire site is serving 16 million pages a month and we're doing it off of two servers which are almost completely unloaded," said Spolsky. One server is a Web server, the other is running Microsoft SQL Server 2008 and both are 8 core Xeon's. While many may assume using an open source stack would be more efficient, Spolsky explained that while SQL Server licenses cost $5000 per box, the Microsoft stack is paying for itself in terms of reduced hardware.
  9. Critical Mass: It's imperative to have critical mass on day one; to ensure people are available to answer questions. "That was one of the reasons I asked Jeff Attwood to be involved in the site," Spolsky explained. Between Joel on Software (Spolsky's blog) and Coding Horror (Attwood's blog), the two had a combined visitor count of 1.3 million visitors per month. Combined with the weekly podcast the two began, they were certain to get at least 20-30 thousand programmers interested.

We've embedded Spolsky's talk below and it is well worth an hour of your time; particularly if you're interested in building, or have already created a social site; Stack Overflow's numbers speak for themselves.