Operating as a collective, an ant colony can achieve remarkable things, complete tasks, and solve problems that would be unimaginable for a single ant. Colonies are responsible for building elaborate nests, waging battles, and creating efficient highway systems to food sources. The collective intelligence of an ant colony can serve as inspiration to help us solve complex human problems. Businesses in particular are finding innovative ways to apply these lessons from nature, from routing trucks to managing plane congestion on the tarmac… to making Internet search more accurate.
The theory of swarm intelligence (or collective intelligence) relates to how the simple actions of individuals can come together to produce the sophisticated behavior of the collective. Deborah Gordon, a biologist at Stanford University who has spent decades studying harvester ants in the Arizona desert, summed up the concept this way: “Ants aren’t smart. Ant colonies are.”
Take foraging as an example. Whenever an ant finds food and carries it back to the nest, that ant leaves a chemical trail (pheromones) along the way. Other ants sniff the chemical trail and follow it toward that same food source. As more ants find the food and carry it back to the nest, the path gets a stronger chemical dose and, in turn, becomes more attractive to fellow foragers. Individually, these ants are following a simple set of rules and acting on local information: follow the pheromone clues and bring food back to the nest. However, the colony as a collective is behaving in quite a complex way: creating a sophisticated highway system that leads to the best food sources.
Collective Intelligence and the Web
So, what do ants and chemical trails have to do with the web? For starters, lessons from colony behavior can be applied to enhance the way we search for information, products, and solutions.
The Internet puts an unprecedented range of goods and information right at our fingertips. And while we now have the ability to find what we want, when we want, many are finding that more isn’t necessarily better. As Barry Schwartz explains in “The Paradox of Choice: Why More is Less,” too many options actually cause more psychological distress than good.
Think about it. How many times have you abandoned a search after hitting page 7 (or maybe 2) because you couldn’t find what you were looking for, and then ended up doing multiple searches with different search terms? The sheer enormity of results often makes searches exhaustive, tedious, and overwhelming; we’re forced to wade through pages of results before finding the product or link we want.
Of course, the answer is not to limit the available products or retrieved results. After all, access to this rich “long tail” of goods has been a key driver behind the success of many online retailers. Instead, websites try to minimize this search exhaustion by predicting what a person really wants and putting these products up front. It’s a great idea in theory, but not so easy to implement in the real world.
The Limits of Web 2.0, and the “Squeaky Wheel” Syndrome
With a strong foundation in collaboration, community, and user participation, the Web 2.0 movement seemed to solve this dilemma by factoring in the contributions of users to narrow down choices. Eager community participants can make their voices and opinions known through user reviews, recommendations, ratings, tagging, and more. Websites have tapped into these crowd-sourcing techniques to determine the relevance of search results. While these methods may help tame the tangle of options, they suffer from one major problem: bias.
Traditional crowd-sourcing demands active participation from its members. The problem here is that not everyone contributes. Only certain types of individuals are likely to make an effort, and they are driven by various motives, from a mere hope to be noticed in a crowd to an altruistic desire to help others to a need to rant about a negative experience. In short, only a subset of the population (the squeaky wheels) will participate, significantly limiting the sample pool and possibly skewing the results with personal bias and inaccuracy.
But what if there was a way to sidestep these biases and gather a perfect representation of consumer attitudes by tapping into the opinion of every single person who visited a site or conducted a search?
Back to the ant colony…
The Next Phase of Social Search: the Super-Community
Watching a trail of ants march toward crumbs of food, it’s hard to imagine that ants aren’t aware of their actions. But according to studies on swarm intelligence, what appears to be intelligent behavior actually results from nothing more than the complex interaction of simple actions.
Likewise, websites can tap into the implicit wisdom of the community to more accurately predict the most popular and relevant results of any given search. There’s a wealth of information in the everyday online activity and behavior of website visitors: every successful or failed search, every page visited, every purchase or abandoned cart represents valuable information. These natural behaviors and actions reflect the true and unbiased opinion of the community as a whole.
By listening to these implicit actions, website owners can gain new insight into the preferences of the silent majority; by leveraging the data, they can optimize results for future searchers. Just like ants that leave a chemical trail each time they bring food back to the nest, we leave real-time feedback each time we visit a page or select (or ignore) a result.
With each search, we unknowingly participate in a cooperative design that improves the search experience for all searchers to follow. Simple, self-guided actions — entering keywords and selecting results — drive the greater common good. And as more people participate, both the chemical trail and the overall system grow stronger.
This new participatory strategy gives greater power to the super-community, in which the collective intelligence of all site visitors is harnessed to create a better search and shopping experience for everyone. With each search, the community carves out a faster, more efficient pathway to desired information and products, no different than the trail of pheromones leading to food sources. And like the ants, web searchers act as a collective team (whether they know it or not), yet another example of the whole being greater than the sum of its parts.
Scott Brave is a founder and CTO of Baynote, Inc. Prior to Baynote, he was a postdoctoral scholar at Stanford University and served as lab manager for the CHIMe (Communication between Humans and Interactive Media) Lab. Scott is an inventor of six patents and co-author of over 25 publications in the areas of human-computer interaction and artificial intelligence. Scott is also an Editor of the “International Journal of Human-Computer Studies” (Amsterdam: Elsevier) and co-author of “Wired for speech: How voice activates and advances the human-computer relationship” (Cambridge, MA: MIT Press). Scott received his Ph.D. in Human-Computer Interaction, and B.S. in Computer Systems Engineering from Stanford University, and his Master’s from the MIT Media Lab.
(Photo by Il conte di Luna.)
