by    in Data, Data Science, Popular Lists

Applying Machine Learning to the Diversity within our Worst Presidents List

Ranker visitors come from a diverse array of backgrounds, perspectives and opinions.  The diversity of the visitors, however, is often lost when we look at the overall rankings of the lists, due to the fact that the rankings reflect a raw average of all the votes on a given item–regardless of how voters behave on multiple other items.  It would be useful then, to figure out more about how users are voting across a range of items, and to recreate some of the diversity inherent in how people vote on the lists.

Take for instance, one of our most popular lists: Ranking the Worst U.S. Presidents, which has been voted on by over 60,000 people, and is comprised of over a half a million votes.

In this partisan age, it is easy to imagine that such a list would create some discord. So when we look at the average voting behavior of all the voters, the list itself has some inconsistencies.  For instance, the five worst-rated presidents alternate along party lines–which is unlikely to represent a historically accurate account of which presidents are actually the worst.  The result is a list that represents our partisan opinions about our nation’s presidents:

 

ListScreenShot

 

The list itself provides an interesting glimpse of what happens when two parties collide in voting for the worst presidents, but we are missing interesting data that can inform us about how diverse our visitors are.  So how can we reconstruct the diverse groups of voters on the list such that we can see how clusters of voters might be ranking the list?

To solve this, we turn to a common machine learning technique referred to as “k-means clustering.” K-means clustering takes the voting data for each user, summarizes it into a result, and then finds other users with similar voting patterns.  The k-means algorithm is not given any information whatsoever from me as the data scientist, and has no real idea what the data mean at all.  It is just looking at each Ranker visitor’s votes and looking for people who vote similarly, then clustering the patterns according to the data itself.  K-means can be done to parse as many clusters of data as you like, and there are ways to determine how many clusters should be used.  Once the clusters are drawn, I re-rank the presidents for each cluster using Ranker’s algorithm, and the we can see how different clusters ranked the presidents.

As it happens, there are some differences in how clusters of Ranker visitors voted on the list.  In a two-cluster analysis, we find two groups of people with almost completely opposite voting behavior.

(*Note that since this is a list of voting on the worst president, the rankings are not asking voters to rank the presidents from best to worst, it is more a ranking of how much worse each president is compared to the others)

The k-means analysis found one cluster that appears to think Republican presidents are worst:

ClusterOneB

Here is the other cluster, with opposite voting behavior:

ClusterTwoB

In this two-cluster analysis, the shape of the data is pretty clear, and fits our preconceived picture of how partisan politics might be voting on the list.  But there is a bias toward recent presidents, and the lists do not mimic academic lists and polls ranking the worst presidents.

To explore the data further, I used a five cluster analysis–in other words, looking for five different types of voters in the data.

Here is what the five cluster analysis returned:

FiveClusterRankings

The results show a little more diversity in how the clusters ranked the presidents.  Again, we see some clusters that are more or less voting along party lines based on recent presidents (Clusters 5 and 4).  Cluster 1 and 3 also are interesting in that the algorithm also seems to be picking up clusters of visitors who are voting for people that have not been president (Hillary Clinton, Ben Carson), and thankfully were never president (Adolf Hitler).  Cluster 2 and 3 are most interesting to me however, as they seem to show a greater resemblance to the academic lists of worst presidents, (for reference, see wikipedia’s rankings of presidents) but the clusters tend toward a more historical bent on how we think of these presidents–I think of this as a more informed partisan-ship.

By understanding the diverse sets of users that make up our crowdranked lists, we are able to improve our overall rankings, and also provide more nuanced understanding how different group opinions compare, beyond the demographic groups we currently expose on our Ultimate Lists.  Such analyses help us determine outliers and agenda pushers in the voting patterns, as well as allowing us to rebalance our sample to make lists that more closely resemble a national average.

  • Glenn Fox

 

 

A Ranker World of Comedy Opinion Graph: Who Connects the Funny Universe?

In the previous post, we showed how a Gephi layout algorithm was able to capture different domains in the world of comedy across all of the Ranker lists tagged with the word “funny”.  However, these algorithms also give us information about the roles that individuals play within clusters. The size of the node indicates that node’s ability to connect other nodes, so bigger nodes indicate individuals who serve as a gateway between different nodes and categories.  These are the nodes that you would want to target if you wanted to reach the broadest audience, as people who like these comedic individuals also like many others.  Sort of like having that one friend who knows everyone send out the event invite instead of having to send it to a smaller group of friends in your own social network and hoping it gets around. So who connects the comedic universe?

The short answer: Dave Chappelle (click to enlarge)

Chappelle

Dave Chappelle is the superconnector. He has both the largest number of direct connections and the largest number of overall connections. If you want to reach the most people, go to him. If you want to connect people between different kinds of comedy, go to him.  He is the center of the comedic universe. He’s not the only one with connections though.

Top 10 Overall Connectors

  1. Dave Chappelle 
  2. Eddie Izzard 
  3. John Cleese 
  4. Ricky Gervais
  5. Rowan Atkinson
  6. Eric Idle
  7. Billy Connolly
  8. Bill Hicks
  9. It’s Always Sunny In Philadelphia
  10. Sarah Silverman

 

We can also look at who the biggest connectors are between different comedy domains.

  • Contemporary TV Shows: It’s Always Sunny in Philadelphia, ALF, and The Daily Show are the strongest connectors. They provide bridges to all 6 other comedy domains.
  • Contemporary Comedians on American Television: Dave Chappelle, Eddie Izzard and Ricky Gervais are the strongest connectors. They provide bridges to all 6 other comedy domains.
  •  Classic Comedians: John Cleese and Eric Idle are the strongest connectors. They provide bridges to all 6 other comedy domains.
  • Classic TV Shows: The Muppet Show and Monty Python’s Flying Circus are the strongest connectors. They provide bridges to Classic TV Comedians, Animated TV shows, and Classic Comedy Movies.
  • British Comedians: Rowan Atkinson is the strongest connector. He serves as a bridge to all of the other 6 comedy domains.
  • Animated TV Shows: South Park is the strongest connector. It serves as a bridge to Classic Comedians, Classic TV Shows, and British Comedians.
  • Classic Comedy Movies: None of the nodes in this domain were strong connectors to other domains, though National Lampoon’s Christmas Vacation was the strongest node in this network.

 

 

A Ranker Opinion Graph of the Domains of the World of Comedy

One unique aspect of Ranker data is that people rank a wide variety of lists, allowing us to look at connections beyond the scope of any individual topic.  We compiled data from all of the lists on Ranker with the word “funny” to get a bigger picture of the interconnected world of comedy.  Using Gephi layout algorithms, we were able to create an Opinion Graph which categorizes comedy domains and identify points of intersection between them (click to make larger).

all3sm

In the following graphs, colors indicate different comedic categories that emerged from a cluster analysis, and the connecting lines indicate correlations between different nodes with thicker lines indicating stronger relationships.  Circles (or nodes) that are closest together are most similar.  The classification algorithm produced 7 comedy domains:

 

CurrentTVwAmerican TV Shows and Characters: 26% of comedy, central nodes =  It’s Always Sunny in Philadelphia, ALF, The Daily Show, Chappelle’s Show, and Friends.

NowComedianwContemporary Comedians on American Television: 25% of nodes, includes Dave Chappelle, Eddie Izzard, Ricky Gervais, Billy Connolly, and Bill Hicks.

 

ClassicComedianswClassic Comedians: 15% of comedy, central nodes = John Cleese, Eric Idle, Michael Palin, Charlie Chaplin, and George Carlin.

ClassicTVClassic TV Shows and Characters: 14% of comedy, central nodes = The Muppet Show, Monty Python’s Flying Circus, In Living Color, WKRP in Cincinnati, and The Carol Burnett Show.

BritComwBritish Comedians: 9% of comedy, central nodes = Rowan Atkinson, Jennifer Saunders, Stephen Fry, Hugh Laurie, and Dawn French.

AnimwAnimated TV Shows and Characters: 9% of comedy, central nodes = South Park, Family Guy, Futurama, The Simpsons, and Moe Szyslak.

MovieswClassic Comedy Movies: 1.5% of comedy, central nodes = National Lampoon’s Christmas Vacation, Ghostbusters, Airplane!, Vacation, and Caddyshack.

 

 

Clusters that are the most similar (most overlap/closest together):

  • Classic TV Shows and Contemporary TV Shows
  • British Comedians and Classic TV shows
  • British Comedians and Contemporary Comedians on American Television
  • Animated TV Shows and Contemporary TV Shows

Clusters that are the most distinct (lest overlap/furthest apart):

  • Classic Comedy Movies do not overlap with any other comedy domains
  • Animated TV Shows and British Comedians
  • Contemporary Comedians on American Television and Classic TV Shows

 

Take a look at our follow-up post on the individuals who connect the comedic universe.

– Kate Johnson

 

by    in New Features

New Features on Ranker

As usual, we are hard at work here in the Internet factory trying to make our site better and better. Pretty soon, Ranker.com is going to be able to walk your dog and drop your kids off at soccer practice. Those jet-pack days are not here yet, but we do have some other fun, new stuff for you to gaze upon. And maybe also use.

Do you sometimes see a list and think, “I want to rank that, but who has the TIME?” Not to fear, good people. We’ve actually reduced the amount of time it takes to register your opinion in list-form. With science!

Say you go to a list, any list. You start voting, all casual-like… and suddenly this tab pops out of the right side of your browser. Every time you vote something up, the number on the counter goes up, too! If you click on the green button that says “Your Votes,” a starter-list will come sliding in from the right side. You can literally re-order items, delete items, add items, write copy, and even add images and/or videos RIGHT THERE. And then publish your re-rank. RIGHT THERE. I mean, think of the time you just saved. Now you finally have time to cut your toenails. You’re welcome.

This isn’t technically new… but it’s something that looks new, so we’re counting it. The trigger buttons that switch how you view lists have moved! Not that exciting, I guess.

But in case you wondered why your ability to change ‘blog view’ to ‘info view’ or ‘info view’ to ‘slideshow’ wasn’t where it was supposed to be, just move your eyeballs to the left side of the screen and look right at the top of the list. There they are! Everything is still as it should be, only different.

We’ve gone and moved into the fine year of 2010 by finally making a functioning mobile version of our site. You can browse our lists more easily now, vote more easily and now you can actually re-rank a list… on your PHONE!

Unfortunately, we don’t yet support creating your own, brand-new lists on mobile, and there are some other user features of the site that we still won’t support on mobile… BUT it should be a lot easier to find, read, and vote on all your favorite lists.

Have you ever just wanted to copy and paste a list you see on Ranker? Maybe to a blog post, email, or just to Facebook? Well, now you can!

If you go into the ‘more options’ dropdown at the top of every list and select “Paste to Clipboard,” a popup with the list as it appears in text format will appear, allowing you to copy it and paste it wherever you want, free of formatting. If you want to paste it to Facebook, make sure to select the checkbox that says, “Check here to copy for Facebook.”

Enjoy. Check back soon for more fun new features!

by    in New Features

Latest Features on Ranker

There are a lot of neat little things we’ve been working on around here in the lab. Things that make it easier and more fun to make the lists you want to make. Take a peek:

Send A Note

We were sitting around the other day in the conference room and someone said ‘hey, wouldn’t it be cool if users could talk to each other? Like email, sorta?”

So we decided that you guys should totally get in on this whole “electronic” form of communication. Now, If you read a list you like, or are intrigued by the genius behind “5 Ways To Make Homemade Spam”, you can go to their profile page and send the list-maker a note and let them know that there are actually 6 ways! And, because we believe in the goodness of the human spirit, we are sure you guys won’t use this new power for evil.

Send A Note

PS. You need to be logged in to see this new feature!

 

Adding Items

Remember that time you made your favorite movie list? But you couldn’t remember ALL your favorite movies, because you’re not a damned robot, right? And then you were looking at someone else’s favorite movie list – or maybe perusing the Best Movies of All Time list – and you saw Piranha II: The Spawning listed there. That is totally one of your favorite movies, but you forgot until just now! Well, we have a way for you to add it to your own list with a single click. If you click that blue ‘+’ button, you will get a dropdown with any relevant lists of yours that Piranha II might be good to add to. Select your favorite movie list from the dropdown and POW, that James Cameron classic is now on your own list, too!

Adding Items

PS. You need to be logged in to see this new feature!

 

SlideShow View

You already know that you have two choices for how your list displays on Ranker. you can write lots of lovely words for the internet to read with big pictures… or you can just create easily digestable stacked lists with small images. Now we give you a third option… Slideshow! Build your list like normal in Edit, put in nice pretty images that will look good big — this view supports any commentary you might want to add, too! Choose the ‘slideshow view’ option from your ‘list options’ popup, and when you publish your list will display one beautiful item at a time!

SlideShow View

 

Filtering Lists

We have so many lists on Ranker. So. Many. And sometimes it’s overwhelming, we know. God, we know. But we’ve been tagging lists (and so have you) for the last few years and we finally went ahead and made use of them. Now, when you go into any of the big category tabs on ranker (film, tv, people, etc) you will see a little array of blue buttons on the top of the right sidebar. You can use these little buttons to sort and filter the content of that category in a million different ways! Each new filter button will narrow down your results until you find the exact lists you are looking for. Go try it!

Filtering Lists

 

Stylish Copy

One of the things we’ve never really had so much around here is the ability to dress up the things you guys are writing on your blog view lists. Bolding, italics, stuff like that. Well, fret no more! We now support a simple text styling interface in Edit.

When you are building your lists, and you want to write stuff… just click on the text field for your item. There is a whole little string of new tools there that allows you to make your text a lot fancier! And easy! Always easy!

Stylish Copy

by    in Opinion Graph

The Best Possible Answers To Opinion-Based Questions

Ranker, as an openended platform for ranking people/places/things, is a lot of different (awesome) things to different people. But the overarching goal for Ranker has always been to provide the best possible answer to opinion-based questions like “What are the best _____?”

Popular sports and entertainment vote lists often grow into being a great answer within 12-72 hours as they get lots of traffic quickly, but the majority of Ranker lists take 1 – 3 months to build to full credibility as visitors on Ranker and from search engines find them and shape them with votes and re-ranks.

I thought it would be fun to showcase some Ultimate Lists and Vote Lists in other categories that haven’t gone viral, but through the participation of lots of Rankers over a few months have indeed become “the best possible answer” to this question.

 

Food: you all clearly love to weigh in on the start of the day, and the 5 o’clock hour:

Best Breakfast Cereals

The Best Cocktails

But you also have strong opinions on hydration during the day:

Best Sodas (and for the more calorie-conscious among you The Best Diet Sodas)

And even specific Gatorade flavors (thanks for the list Lucas)

Snacking, whether it be on a particular type of cheese, candy bar, or even as granular as a specific Jelly Belly flavor (thanks for the list Samantha but what’s with all the chocolate pudding haters?)

Dining out, specifically at Italian chain restaurants

A list I am not authorized to vote on, pregnancy cravings

And hundreds more, including perhaps a new category entirely – food nostalgia (I do miss those Crispy M&Ms myself)

 

Fashion/Beauty: not categories that I personally check up on much, so I was psyched to see quite a few solid rankings here, some of them high-end but mostly stuff you can find at the mall:

Best women’s shoe brands

Best denim brands

Top handbag designers

Fashion Blogs

Sulfate-free shampoos

And even a men’s facial moisturizers list (have only tried 3 or 4 myself, but agree with their relative positions on the list)

 

Travel: Rankers, I know from a number of you that as we’ve been adding datasets of “rank-able objects” over the last year, one of the most-requested ones that we don’t yet have is hotels/resorts. Trust me, it’s still on the list. But in the meantime, it’s been heartening to see how many of you have participated in these great resources for travel destinations and attractions, like these:

Best US cities for vacations

Honeymoon destinations

Coolest cities in America

Theme parks for roller coaster addicts

And my personal faves, “bucket lists” of the world’s most beautiful natural wonders and historical landmarks.

Great stuff – these lists and 1000s more like them are true testimonials to the “wisdom of crowds”.  Thanks, crowds!

by    in Popular Lists

It's The List of The Day's First Day of School

Summer is coming to a (social) end. And by this I mean, of course, that it’s back to school time. The summer is by no means over, we’re still going to have awesome beach weather for a little while longer (and in LA, it’ll continue until we’re very well close to Christmas). But as far as the younguns are concerned, summer is basically over. The summer retail jobs are ending, video games are being finished and teenage summer romances are reaching their awkward “how the hell do I get out of this?” stage (unless they creepily follow you back to the high school you attend)

 

“So, uh, yeah, I’ll give you a call sometime…”

For those of you starting college in this fine fall season of 2011 (the class of 2015), there are some sage words of advice the internet has to give you. And those words of advice is “learn from our mistakes”, “don’t be an idiot” and “don’t fall for the oldest tricks in the book”.

So, thankfully, this generation has a meme looking out for them. It’s a meme that gives them great advice by ruthlessly mocking all the dumbest, most cliche, idiotic or just plain naive things we all did as college freshmen. 

We present to you, the College Freshman Meme — chronicling the best in what everybody does (or thinks) as a young, dumb college freshman. 

Have fun, make the most of it, study but don’t study too hard, and most importantly, meet as many attractive people that like you as you possibly can. They’ll come in handy later… ifyouknowwhatimsayin.

The Very Best of the College Freshman Meme 

by    in Popular Lists

Today's List of the Day is Full of Band Geeks

Remember that movie Drumline and how unmistakingly badass it made marching bands look?

Yeah, neither do we. 

BUT, turns out that a lot of the band geeks you might have picked on in high school may actually turn out to be quite the famous celebrities, so watch who you pick on.

Please.

This comes from experience. Being a band geek wasn’t like in Glee. One assembly didn’t lead to people singing along with you, a huge fanfare and you somehow getting the girl/guy; it most often led to your parents buying you some ice cream on the way home before you tried to fit in as much Nintendo as you could before bed. 

No, this is not the way of marching band nowadays. Geeks are taking over the world and everything that used to be weird or nerdy is now okay and downright cool. So if you were in marching band, you were most likely missing out. Although it seems lately that we’ve gone from this…

to this…

although nowadays you do get a fair amount of this…

So, just to prove that people in band CAN be cool, here’s a list of celebrities that, at some point in their school careers, were in marching band. From Halle Berry, Gwen Stefani and Woody Allen, to Julia Roberts, Steven Spielberg and even Bill Clinton.

Wear your band geekdom proud today, nerds, and let your flag fly (and please, try not to drop it).

30 Famous People Who Were Once in Marching Band

by    in Popular Lists

Today's Earth-Shattering List of the Day

Sometimes, a natural disaster hits and it brings us all together. Nothing could be more true for the West Coast when on August 23, 2011, the East coast felt a 5.9 earthquake from Virginia to Toronto and went into a fit of panic. Since the West Coast usually sleeps through earthquakes of this size, the West Coast got together (San Francisco, LA, Portland, Seattle, all working in perfect harmony) to relentlessly make fun of the East Coast “Earthquake Virgins” for about 12 hours. Because nothing happened.

The Devastation Was Horrifying…

Twitter then erupted with a barrage of snarky Tweets from funny people…

and the rest of the internet, including Reddit, didn’t hold back either. 

Today’s list of the day is a round-up by the hilarious Ranker user Robert Wabash, chronicling the best of the West Coast making fun of the East Coast for being at all worried about an earthquake. 

So, here’s an earth-shatteringly hilarious list about something that shook up quite a bit of the East Coast. 

The Funniest Internet Reactions to the East Coast Earthquake of 2011

by    in Pop Culture, Popular Lists, Trends

Meme Monday: Oblivious Mothers and Daughters Edition

Hello and welcome to your first helping of meme Mondays! Why meme Mondays? Because there’s nothing better to start you off on your week than knowing you’re behind on something. 

For those of you not in the know, an internet meme is sort of an “inside” joke of the internet — kind of like the kind of inside jokes you have to deal with when you hang out with your significant other and their friends, only you (yes, you) can actually participate. Memes are born when multiple people contribute to an image, video or phrase, making it evolve into something that everyone can laugh at. 

And since Ranker users have really taken a shine to rounding up memes, we have enough to do meme Mondays until “advice memes” stop being popular.

Starting us off is The Very Best of the Oblivious Suburban Mom Meme. Reddit’s AdviceAnimals section (where a lot of these memes are born, well, where a lot of these memes are raised) took a picture of a pretty attractive almost-middle-aged woman and started making her say the kind of stuff that your normal, in-denial (or really that oblivious) suburban mom would think about her kids doing “adult” or “illegal” things. 

For example

So enjoy the very best of this meme that makes fun of clueless, idealistic parents. 

Speaking of clueless idealism, when’s the last time you talked to a child about music? Nothing on earth will make you feel older.

“Paul McCartney! You don’t know Paul McCartney?!”

“Oh, you mean Jesse McCartney’s dad? Is that him? Is he an actor or something?”

This causes much distress. Especially when the kid thinks he/she knows everything already, even though they’re in 8th grade, which means they’re just starting to like stuff. At all. 

And then it hits a peak when they say things like this:

So for 39 more range-inducing hits from this annoyingly happy, yet ignorant little girl, check out The Very Best of the Musically Oblivious 8th Grade Meme. And bring a stress ball. 

And that’s your meme Monday for this pre-Comic-Con July week.