Using Big Data To Solve Social Discovery's Relevance Problem

Guest author Johann Schleier-Smith is co-founder and CTO of Tagged, a social-media site designed to connect people of like mind who don't already know each other.

The desire to connect with people is part of being human, with countless generations of networking and matchmaking predating our modern online world.

And yet the science behind making connections is far from being conventional wisdom. Compared to traditional social networks, social-discovery products face a unique challenge in needing to do more than simply connect folks who are already friends. Every social site faces this issue, and each one has its own ways of solving it. 

At Tagged, our "secret sauce" - hard-won learnings and critical insights for social discovery - has five guiding principles:

1. Seek Deep Understanding Of The Customer

It's easy for analytically minded people working on big data sets to focus on average performance metrics or other statistical measures. Analyzing the individual member’s experience, often proves fruitful for insights, ideas and improvements.

Mathematical tools are fantastic for generalizing or classifying behavior, but there's tremendous value in recognizing that each person is unique. To be successful, social media must be attentive to what individuals, through their descriptions and actions, show they really want or need.

2. Seek Mutual Compatibility

The best matches happen when two members want to connect with each other. The goal is to measure how members interact, and how conversations start, rather than simply measuring one-way interest.

Is a great profile picture important? Yes, it’s been proven. However, there are deeper facets of interaction that play a large role in compatibility.

3. Combine Off-The-Shelf Algorithms And Proprietary Techniques

We use a variety of textbook classification models (Naive Bayes, Support Vector Machines, Decision Trees), as well as regression techniques (logistic regression and others), with inputs including basic profile information such as age, gender, or location.

However, as members interact with others, we incorporate personal patterns into the models. Tagged also bases algorithms on large-scale social-graph structure, an approach that has been surprisingly effective.

4. Test Rigorously And Extensively

Like any large-scale social network, Tagged makes billions of match recommendations each month. It's critical to carefully compute statistical error measures, even when benefitting from large data volumes that quickly shrink sampling errors.

5. Seek Deep Understanding Of Algorithms

Sometimes one can make quick progress by applying techniques that work elsewhere. However, sustained progress or innovation requires full understanding of why each mathematical tool works. Social discovery demands a thorough understanding of each tool's strengths and weaknesses. 

And that’s just scratching the surface. This five-point standard only alludes to the nature of the work required, but you get the idea of what it takes to figure out and perfect social discovery and the science of personal connection.

 

Lead image courtesy of Shutterstock.