Between self-aggrandizing FriendFeeds, bottom-feeding link baiters, and perpetual Twitter spammers, finding cool online friends can be challenging. Michael G. Noll and Ching-man Au Yeung created the SPEAR (SPamming-resistant Expertise Analysis and Ranking) algorithm in the hopes of separating the social media wheat from the chaff. This morning the two postgraduate students offered their findings to Delicious in a blog post. The project was first evaluated using data sets collected from the popular bookmarking community.
Noll and Yeung presented SPEAR in a paper entitled, Telling Experts from Spammers: Expertise Ranking in Folksonomies at July’s SIGR Conference. The solution is based on the information retrieval algorithm HITS (Hyperlink-Induced Topic Search), an algorithm best known for powering Google and Yahoo web page rankings. Rather than producing search results, SPEAR ranks and produces a list of experts and content. According to the duo, their method is more resistant to spammers for the following reasons:
1. Mutual reinforcement of user expertise and document quality: A user’s expertise in a particular topic depends on the quality of the documents she or he has found, and the quality of documents in turn depends on the expertise of the users who have found them.
2. Discoverers vs. followers: Expert users should be discoverers – they tend to be faster than others to identify new and high quality documents…SPEAR gives more credit to users the earlier they find high quality documents.
After analyzing more than 500,000 Delicious users and 2 million shared bookmarks, the solution produced a set of trustworthy users. No spammers were found in the top 200 recommendations.
While there are obvious uses for SPEAR in shopping and friend recommendation engines, says Noll, “The SPEAR algorithm itself is not restricted to the online world. We imagine to use SPEAR, for example, for estimating the expertise of researchers by analyzing scientific publications. Such publications – whether available as online versions or printed out on paper – provide all the information we need.”
Expertise may have an algorithm across all industries. Be first and be fascinating. For more information on SPEAR visit Michael G. Noll’s site.
