Two Wharton academics released an interesting paper last week that asks whether online recommendation services are a threat to the aggregate diversity of items discovered by their users. The study is titled “Blockbuster Culture’s Next Rise or Fall: The Impact of Recommender Systems on Sales Diversity” and I found it via a good summary article at PaidContent this weekend.
All indications point towards a rise in importance by recommendation engines, so this argument deserves examination. From eBay’s acquisition of StumbleUpon to the CBS acquisition of Last.fm to this weekend’s MSNBC acquisition of Newsvine – recommendation engines are big money. We’ve covered quite a few startups in this space and I’m sure it will continue to grow in prominence.
Perhaps more importantly, the “Long Tail” of diverse discovery is an important part of the meritocratic and democratic promise of the new web.
Good recommendation engines are also just plain fun.
After just a little consideration, the Wharton study seems more meaningful as a cautionary tale than as a critique of the inherent nature of recommendation engines. In discussing this with others I’ve found that most people swing quickly from believing the study is either obviously wrong or obviously correct. It’s a more complex question than it might seem.
Recommendation engines should strive to be smarter than simply finding that “there is a high correlation between people who liked X and people who liked Y.” I would argue, for example, that recommending other users of a system and highlighting their less popular discoveries could be a good way to solve the problem. Getting it right is probably easier said than done, but it seems there’s still plenty of potential for recommendation engines to expand the long tail. The study’s arguments are important to consider, though.
What the Study Says
A Wharton summary of the paper excerpts the following to explain the study’s conclusion: “Because common recommenders recommend products based on sales and [consumer] ratings, they cannot recommend products with limited historical data, even if they would be rated favorably,” the authors write. “This can create rich-get-richer effects for popular products and vice-versa for unpopular ones, which results in less diversity.”
There’s also some discussion of the Facebook app landscape, arguably an environment where the long tail doesn’t hold up. See also this related discussion at TechCrunch.
The authors argue that individual users may consistently be exposed to items that are new to them, but we’re all exposed to the same new items – resulting in greater individual diversity but less aggregate diversity.
Counter Arguments
The study includes a counter argument from Greg Linden, who helped develop Amazon’s recommendation engine. Linden says “recommendation algorithms easily can be tuned to favor the back catalog — the long tail — as Netflix does.”
The role played by early adopters, “cool hunters”, taste makers and advertisers relative to recommendation engines would also be interesting to look at.
My personal fantasy for recommendation engines is this: I want del.icio.us to look at my bookmarks and recommend not just other URLs I might be interested in, but also other users whose tastes are similar to mine. I’d also like to see which of those recommended users tend to find items of interest earliest, so I can prioritize following them.
Repetition, perhaps another way to describe popularity, will probably always drive consumption – but if I can see all of the things that are discovered by people recommended to me then I can use their less popular picks as guidance.
If other metrics are considered, and surely they are in any sophisticated recommendation engine, then what’s called “Attention Data” can help augment recommendations beyond merely what’s most popular among people with similar interests. (Need an intro to Attention Data? Here’s one that could work for you.)
It would be ill advised to reject recommendation engines as dumb popularity machines based on this study, but it is also important to take its arguments into consideration.
