by    in Data, Data Science, Popular Lists

Applying Machine Learning to the Diversity within our Worst Presidents List

Ranker visitors come from a diverse array of backgrounds, perspectives and opinions.  The diversity of the visitors, however, is often lost when we look at the overall rankings of the lists, due to the fact that the rankings reflect a raw average of all the votes on a given item–regardless of how voters behave on multiple other items.  It would be useful then, to figure out more about how users are voting across a range of items, and to recreate some of the diversity inherent in how people vote on the lists.

Take for instance, one of our most popular lists: Ranking the Worst U.S. Presidents, which has been voted on by over 60,000 people, and is comprised of over a half a million votes.

In this partisan age, it is easy to imagine that such a list would create some discord. So when we look at the average voting behavior of all the voters, the list itself has some inconsistencies.  For instance, the five worst-rated presidents alternate along party lines–which is unlikely to represent a historically accurate account of which presidents are actually the worst.  The result is a list that represents our partisan opinions about our nation’s presidents:

 

ListScreenShot

 

The list itself provides an interesting glimpse of what happens when two parties collide in voting for the worst presidents, but we are missing interesting data that can inform us about how diverse our visitors are.  So how can we reconstruct the diverse groups of voters on the list such that we can see how clusters of voters might be ranking the list?

To solve this, we turn to a common machine learning technique referred to as “k-means clustering.” K-means clustering takes the voting data for each user, summarizes it into a result, and then finds other users with similar voting patterns.  The k-means algorithm is not given any information whatsoever from me as the data scientist, and has no real idea what the data mean at all.  It is just looking at each Ranker visitor’s votes and looking for people who vote similarly, then clustering the patterns according to the data itself.  K-means can be done to parse as many clusters of data as you like, and there are ways to determine how many clusters should be used.  Once the clusters are drawn, I re-rank the presidents for each cluster using Ranker’s algorithm, and the we can see how different clusters ranked the presidents.

As it happens, there are some differences in how clusters of Ranker visitors voted on the list.  In a two-cluster analysis, we find two groups of people with almost completely opposite voting behavior.

(*Note that since this is a list of voting on the worst president, the rankings are not asking voters to rank the presidents from best to worst, it is more a ranking of how much worse each president is compared to the others)

The k-means analysis found one cluster that appears to think Republican presidents are worst:

ClusterOneB

Here is the other cluster, with opposite voting behavior:

ClusterTwoB

In this two-cluster analysis, the shape of the data is pretty clear, and fits our preconceived picture of how partisan politics might be voting on the list.  But there is a bias toward recent presidents, and the lists do not mimic academic lists and polls ranking the worst presidents.

To explore the data further, I used a five cluster analysis–in other words, looking for five different types of voters in the data.

Here is what the five cluster analysis returned:

FiveClusterRankings

The results show a little more diversity in how the clusters ranked the presidents.  Again, we see some clusters that are more or less voting along party lines based on recent presidents (Clusters 5 and 4).  Cluster 1 and 3 also are interesting in that the algorithm also seems to be picking up clusters of visitors who are voting for people that have not been president (Hillary Clinton, Ben Carson), and thankfully were never president (Adolf Hitler).  Cluster 2 and 3 are most interesting to me however, as they seem to show a greater resemblance to the academic lists of worst presidents, (for reference, see wikipedia’s rankings of presidents) but the clusters tend toward a more historical bent on how we think of these presidents–I think of this as a more informed partisan-ship.

By understanding the diverse sets of users that make up our crowdranked lists, we are able to improve our overall rankings, and also provide more nuanced understanding how different group opinions compare, beyond the demographic groups we currently expose on our Ultimate Lists.  Such analyses help us determine outliers and agenda pushers in the voting patterns, as well as allowing us to rebalance our sample to make lists that more closely resemble a national average.

  • Glenn Fox

 

 

by    in Popular Lists, Trends

Everywhere Around the World…They're Coming to the List of the Day

Happy birthday, America! It’s July 4th once again, which means a day off from work. Hooray! Oh, yeah, and also time to reflect on the founding of the nation and those who bravely gave of their lives in order to declare our independence from England. That’s good too.

We know you’re probably going to spend the day with friends, at barbecues, watching fireworks and other, wholesome outdoorsy kinds of activities. Totally get that. But maybe, just maybe, you’ll have a few minutes to spare to look at some amazing Ranker lists! (Or it’s possible you’re also just pretending to have lots of social, active things to do as an excuse to lounge around in pajamas all day surfing the Internet and eating cereal straight from the box with total impunity. Don’t think it had not occurred to us.)

In either case, you’ll definitely want to check these lists out:

Horrible Fireworks Accidents

Just as integral to the July 4th holiday as hot dogs and ironic aprons are the annual warnings to children and immature adults not to play around with dangerous explosives. But, of course, some people just don’t listen. Which is why this list is both entertaining and a helpful cautionary tale, all wrapped up into one.

In this sample clip, an entire fireworks FACTORY explodes:

No one was injured, so it’s okay to find this clip amusing. And to post this: 

When are they gonna get to the FIREWORKS FACTORY?

Some day, Milhouse…Some day…

US History’s Most Important Battles

Sure, at Ranker we have a lot of fun posting animated “Simpsons” gifs and videos of exploding factories. But we’re also about LEARNING, people!

Dozens of community members so far have helped us narrow down this thorough list of the most significant battles in US military history.

Interestingly, battles of the American Revolution – the reason for the season, as it were – don’t seem to rate all that high on the list. The Battles of Lexington and Concord, which served to kick off the conflict, currently don’t appear on the list until #8. Think this is wrong? Let us know by going through to the list and voting for the battles YOU think matter the most. It’s not too late to have your opinion change around the order of the list.

Transformers!

What could possibly be more American than a Michael Bay movie about 3D robots from space killing each other while simultaneously taking out large sectors of Downtown Chicago? NOTHING! (Except maybe Tom Hanks going to community college, I suppose…)

Now you can relive all your favorite moments from one of the summer’s loudest films with our list of the best quotes from “Transformers: Dark of the Moon.” Yes, there is actually dialog in the film. I swear. Even aside from Tyrese grunting. You may have gone to the restroom at that part. Understandable, as the movie is, after all, approximately 11 hours long.

Yes, my face is somewhat cold and expressionless for a lead character in an extended film franchise. So what? Wanna 3D robot fight about it?

The Greatest US President…Ever…

Votes are still being tallied for our massive CrowdRanked list that will settle – once and for all – who was the greatest American president to date. Unsurprisingly, lanky stove-pipe hat enthusiast Abraham Lincoln is out in front. Much to the consternation of this man…

Oh, Franklin Pierce…will you ever win?

Some of our most unpopular recent presidents – your Tricky Dick Nixons, for example – are still faring rather well, actuall. It really is still anybody’s game. Unless you’re Warren G. Harding, that is. I guess we’re all still kind of upset about that Teapot Dome Scandal thing.

This one’s going to be a nail-biter, so be sure to get in there and make your own ranking, which is the best way to influence the final result.

Plus-Size Video Game Heroes

Finally, a list about our uniquely American love of things that are BIG. Ranker user Adam Thomas threw together a salute to the overweight Americans of video game history that we think is worth a look, even if you’re one of those gamers who prefers a vegan option.

The comments thread already has some provocative challenges to the list. Though Mario is clearly of Italian descent (and speaks with an Italian accent, even), are the accepted parts of the Mario canon implying he lives in Brooklyn enough to quality him for the list? And what of King Hippo? Surely he was from some place called “Hippo Island”! Also, whose brilliant idea was it to put an enormous cartoon band-aid over the sole weak spot on his entire body? Let’s think ahead, folks…