by    in Data Science, prediction, Rankings

Cognitive Models for the Intelligent Aggregation of Lists

Ranker is constantly working to improve our crowdsourced list algorithms, in order to surface the best possible answers to the questions on our site.  As part of this effort, we work with leading academics who research the “wisdom of crowds”, and below is a poster we recently presented at the annual meeting for the Association for Psychological Science (led by Ravi Selker at the University of Amsterdam and in collaboration with Michael Lee from the University of California-Irvine).

While the math behind the aggregation model may be complex (a paper describing it in detail will hopefully be published shortly), the principle being demonstrated is relatively simple.  Specifically, aggregating lists using models that take into account the inferred expertise of the list maker outperform simple averages, when compared to real-world ground truths (e.g. box office revenue).  While Ranker’s algorithms for determining our crowdsourced rankings may be similarly complex, they are similarly designed to produce the best answers possible.

 

cognitive_model_aggregating_lists

 

– Ravi Iyer

by    in Data, Data Science, Opinion Graph

A Ranker Opinion Graph of Important Life Goals

What does it mean to be successful, and what life goals should we be setting in order to get there? Is spending time with family most important? What about your career?  We asked people to rank their life goals in order of importance on Ranker, and using a layout algorithm (force atlas in Gephi), we were able to determine goal categories and organized these goals into a layout which placed goals most closely related nearer to each other.

The connecting lines in the graph represent significant correlations or relationships between different life goals, with thicker lines indicating stronger relationships.  The colors in the graph differentiate between unique groups that emerged from a cluster analysis.  Click on the below graph to expand it.

all_black

The classification algorithm produced 5 main life goal clusters:
(1) Religion/Spirituality (e.g., Christian values, achieving Religion & Spirituality),
(2) Achievement and Material Goods (e.g., being a leader, avoiding failure, having money/wealth),
(3) Interpersonal Involvement/Moral Values (e.g., sharing life, doing the right thing, being inspiring),
(4) Personal Growth (e.g., achieving wisdom & serenity, pursuing ideals and passions, peace of mind), and
(5) Emotional/Physical Well-Being (e.g., being healthy, enjoying life, being happy).

These clusters are well matched to those identified by Robert Emmon’s (1999) psychological research on goal pursuit and well-being. Emmon’s found that life goals form 4 primary categories: work and achievement, relationships and intimacy, religion and spirituality, and generativity (leaving legacy/contributing to society).

However, not all goals are created equal.  While success related goals may be able to help us get ahead in life, they also have downsides.   People who focus on zero-sum goals such as work and achievement tend to report less happiness and life satisfaction compared to people who pursue goals. Our data also show a large divide between Well-being and Work/Achievement goals with relatively no overlap between these two groups.

Other interesting relationships in our graph:

  • Goals related to moral values (e.g., doing the right thing) were clustered with (and therefore more closely related to) interpersonal goals than they were to religious goals.
  • Sexuality was related to goals from opposite ends of the space in unique ways. Well-being goals were related to sexual intimacy whereas Achievement goals were related to promiscuity.
  • While most goal clusters were primarily made up of goals for pursuing positive outcomes, the Achievement/Material Goods goal cluster also included the most goals related to avoiding negative consequences (e.g., avoiding failure, avoiding effort, never going to jail).
  • Our Personal Growth goal cluster is unique from many of the traditional goal taxonomies in the psychological literature, and our data did not find the typical goal cluster related to Generativity. This may show a shift in goal striving from community growth to personal growth.

– Kate Johnson

Citation: Emmons, R. A. (1999). The psychology of ultimate concerns: Motivation and spirituality in personality. New York: Guilford Press.

 

Changes in Opinion for House of Cards, The Walking Dead, Mad Men, & Workaholics

One of the coolest things about Ranker is the fact that Ranker votes are recorded in real time as they happen, allowing the potential for it to track changes in people’s opinions. A list like, “The Best Shows Currently on Air” generates heavy traffic due to the popularity of television shows on air and online. A certain television show can amass an impressive, almost cult-like, following and it’s interesting to see how public opinions change over time, why, and if it corresponds to changes happening in the real-world.

The figure below shows the pattern of change in the proportion of up-votes for the TV shows in this list, and highlights four shows: House of Cards, The Walking Dead, Mad Men, and Workaholics.

tv_show_house_of_cards_change

There is a steep decline in the proportion of up votes in December of 2013 for the House of Cards. Interestingly, this was during an interim period between seasons where seemingly nothing significant relating to the show was occurring. A plausible explanation could be due to a ceiling effect as there were few up votes and no down votes until that time. When a show first gets on a Ranker list, it often is only voted on by the fans of that show. As the show is only accessible through Netflix, the viewing audience is significantly smaller than cable or network Television shows, so that may further skew the number of people who knew enough about the show to consider downvoting it. Fascinatingly enough, in the same month, during a televised meeting with tech industry CEOs on NSA surveillance, President Obama expressed his love for the show stating “I wish things were that ruthlessly efficient,” adding that Rep. Frank Underwood, played by Kevin Spacey, “is getting a lot of stuff done”. Could the increase in downvotes be due to certain members of the public expressing their opinions about the President through the voting patterns on The House of Cards on Ranker?
The entire second season of The House of Cards was released on February 14th on Netflix in the same binge-watching format as the first season, which garnered positive reviews. Interestingly, there is a significant decline in proportion of up votes for The House of Cards from February 2014 to April 2014, however viewership of season two was much higher than season one based on early reviews. The show also garnered critical acclaim for season two earning thirteen Primetime Emmy Award nominations for the 66th Primetime Emmy Awards, and three nominations at both the 72nd Golden Globe Awards and the 21st Screen Actors Guild Awards. Given the viewership ratings and critical success, it may seem surprising to see such a steep drop in votes. But in looking at Ranker data, it is often common for shows to get more downvotes over time as they get better known, as people rarely downvote things they haven’t heard of, even as a show also receives more upvotes. This is why our algorithms take into account both the volume and proportion of upvotes vs. downvotes.
Shows that are more readily accessible may exhibit less of a ceiling effect early on, as there is a greater likelihood of people watching the show who aren’t specifically looking for it. Looking at Mad Men and The Walking Dead, there is a steady increase in up-vote proportion over the span that votes were submitted from June 2013 to last month, April 2015. The Walking Dead is the most watched drama series telecast in basic cable history, making it reasonable to assume that the reason for the continual increases are due to the increasing number of fans of the show who vote for it as the “Best Show Currently on Air”. Mad Men fans had similar voting patterns.

For a show like Workaholic, which airs on Comedy Central, there is a significantly smaller viewing audience compared to national networks, and they do not have the fanbase power of House of Cards or The Walking Dead. However, it is a show with positive reviews and a steady following of loyal fans. Though it is not as popular as other shows airing, it’s proven to be a show with comedic talent that generates positive sentiments amongst its viewers and a growing proportion of up-votes.
While these examples are only suggestive, the enormous number of votes made by Ranker uses, and the variety of topics they cover, makes the possibility of measuring opinions, and detecting and understanding change in opinions, an intriguing one that is worth continuing to expand upon.
-Emily Liu

by    in Opinion Graph, Ranker Comics

A Cluster Analysis of the Superpower Opinion Graph produces 5 Superhero types

If you could have one superpower, which would you choose?  Data from the Ranker list “Badass Superpowers We’d Give Anything to Have” improves on the age-old classroom ice breaker question by letting people rank all of the superpowers in order of how much they would want them.  Because really, unless you’re one of the X-men, you probably would have more than one power. So, if you could have a collection of superpowers, what kind of superhero would you be?

Using Gephi and data from Ranker’s Opinion Graph, we ran a cluster analysis on people’s votes on the superpowers list to determine what groupings of superpowers different people wanted.

This analysis grouped superpowers into 5 clusters, which we interpreted to represent unique superhero types.

 

The Overall Superpower Opinion Graph

Allpowers

 

 

The 5 Types of Superheroes

    god

1. The Creationist God: This superhero type is characterized by creation and destruction, Old-Testament Christian God-style. Notable superpowers: the ability to create/destroy worlds, die and come back to life, have gods’ weapons (Thor’s Hammer, Zeus’ Thunderbolt), remove others’ senses, and resurrect the dead.

timelord

2. The Time Lord: This superhero type is basically The Doctor from Dr. Who. Notable superpowers: omnipotence, travel to other dimensions, open portals to anywhere, and travel beyond the omniverse.

elementalist

3. The Elementalist: This superhero type has the ability to manipulate the elements and use them as weapons to their advantage. Notable superpowers: manipulation of water, fire, weather, and plants, ability to shapeshift, shoot ice, and lightning and fire.

superman

4. The Superhuman: This superhero type is humans+, with enhanced human senses and decreased human limitations. Notable superpowers: sense danger, x-ray vision, walk through walls, super speed, mind reading, flight, super strength, and enhanced flexibility.

zen

5. The Zen Master: This superhero type sounds a bit like being permanently on mind-altering psychoactive substances crossed with Gandhi. Notable superpowers: speech empowerment, spiritual enlightenment, and infinite appetite!!.

 

-Kate Johnson

by    in Opinion Graph

Characteristics of people who are not annoyed by Bill O’Reilly

On today’s The O’Reilly Factor (video below), Bill O’Reilly lamented the fact that he was only #10 on Ranker’s Most Annoying TV Hosts list and decided that he would make it his New Year’s Resolution to become the #1 most annoying person on our list. While I may not share O’Reilly’s politics, I like him as a person, even as he does annoy me from time to time, and would like to help him reach his goals. I enjoy working with the Ranker dataset as it lets me answer very specific questions, like whether people who think the show 24 is overrated are also convinced that George W. Bush was a terrible person—or, in this case, I can study the people who specifically disagree that O’Reilly is annoying, in the hopes that O’Reilly can find these people and work to annoy them more.

Who does O’Reilly need to work harder to annoy? From our opinion graph of 20+ million edges, (so named because we can connect not only vague “likes” or “interests,” but specifically whether someone thinks something is best, worst, hot, annoying, overrated, etc.), we have hundreds of specific opinions that characterize people who don’t find O’Reilly annoying. Here are a chosen few findings about these people:

People who are NOT annoyed by O’Reilly tend to…
– find liberals like Jon Stewart, Rachel Maddow, and Bill Maher annoying.
– believe that John Wayne and Humphrey Bogart are among the Best Actors in Film History.
– enjoy movies like The Sound of Music and Toy Story.
– watch America’s Got Talent, Cops, Dirty Jobs, Deadliest Catch, Home Improvement, and Extreme Makeover: Home Edition.
– listen to Lynyrd Skynyrd, Boston, and Elvis.
– enjoy comedians like Bob Hope, Jeff Foxworthy, Joan Rivers, and Billy Crystal.
– be attracted to  Carrie Underwood, Jessica Simpson, Brooklyn Decker, and Sarah Palin.

Thanks to big data, these audiences are all readily targetable online—and if O’Reilly really wants to annoy these people, he might want to study our biggest pet peeves list for ideas (e.g. chewing with his mouth open might work on TV). We hope this list will help O’Reilly with his ambitions for 2015, and please do reach out to us if you need more market research on how to annoy people more.

– Ravi Iyer

by    in Data, Opinion Graph

The Opinion Graph Connections between 24, George W. Bush, Jack Bauer, and Rachel Maddow.

As someone whose roots are in political psychology, I’m always interested in seeing how the Ranker dataset shows how our values are reflected in our entertainment choices.  We’ve seen many instances where politicians have cited 24 in the case for or against torture, but are politics reflected in attitudes toward 24 amongst the public?  Using data from users who have voted on multiple Ranker lists, including our lists polling for The Worst Person in History, the Greatest TV Characters of All-Time, the most Overrated TV shows and The Biggest Hollywood Douchebags, the clear answer is yes.

People who think George W. Bush is one of the worst people in history, also tend to think that 24 is one of the most overrated TV shows of all-time.

People who think Bush is a terrible person also think 24 is overrated.
People who think Bush is a terrible person also think 24 is overrated.

…and people who think Jack Bauer is one of the best TV Characters of All-Time also think that Rachel Maddow is one of Hollywood’s Biggest Douchebags.

maddowvsjackbauer
People who think Jack Bauer is a great TV character also think Rachel Maddow is a douchebag.

– Ravi Iyer

ps. …and these are just a few of the relationships between 24 and politicians in our opinion graph, which all tell the same basic story.

by    in Opinion Graph

The Clear Split Between AMD and Intel CPU Fans

Recently, Tom’s Hardware used the Ranker widget to poll for their Reader’s Choice awards.  Among the topics they polled was the best CPUs and while I knew that there would likely be a preference for AMD or Intel, the two largest manufacturers, I didn’t realize that the choice would be as stark.  I’m a relative novice compared to most of the people who voted in this poll, so perhaps this would not surprise them, but voting for an AMD CPU, made one, on average, 80% less likely to vote for an Intel CPU, and vice versa.  Below is a taxonomy of votes, with items that are voted on similarly closer together, based on a hierarchical cluster analysis of the votes on this list, so you can visualize the split for yourself.

TomsHardwareCPUsTaxonomy

 

– Ravi Iyer

by    in Data Science, Pop Culture, prediction

Ranker Predicts Spurs to beat Cavaliers for 2015 NBA Championship

The NBA Season starts tonight and building on the proven success of our World Cup and movie box office predictions, as well as the preliminary success of our NFL predictions, Ranker is happy to announce our 2015 NBA Championship Predictions, based upon the aggregated data from basketball fans who have weighed in on our NBA and basketball lists.

Ranker's 2015 NBA Championship Predictions as Compared to ESPN and FiveThirtyEight
Ranker’s 2015 NBA Championship Predictions as Compared to ESPN and FiveThirtyEight

For comparison’s sake, I included the current ESPN power rankings as well as FiveThirtyEight’s teams that have the most percentage chance of winning the championship.  As with any sporting event, chance will play a large role in the outcome, but the premise of producing our predictions regularly is to validate our belief that the aggregated opinions of many will generally outperform expert opinions (ESPN) or models based on non-opinion data (e.g. player performance data plays a large role in FiveThirtyEight’s predictions).  Our ultimate goal is to prove the utility of crowdsourced data, as while something like NBA predictions is a crowded space where many people attempt to answer this question, Ranker produces the world’s only significant data model for equally important questions, such as determining the world’s best DJseveryone’s biggest turn-ons or the best cheeses for a grilled cheese sandwich.

– Ravi Iyer

by    in Opinion Graph, Rankings

Characteristics of People who are less Afraid of Ebola

Ebola is everywhere in the news these days, even as Ebola trails other causes of death by wide margins.  Clearly the risks are great, so some amount of fear is certainly justified, but many have taken it to levels that do not make sense scientifically, making back of the envelope projections for its spread based on anecdotal evidence and/or positing that its only a matter of time before the virus evolves into an airborne disease, as diseases regularly mutate to enable more killing in movies.  Regardless of whether Ebola warrants fear or outright panic, the consensus is that it is scary, as also evidenced by its clear #1 ranking on Ranker‘s Scariest Diseases of All Time list.  Yet, among those who are fearful, I couldn’t help but wonder, what are the characteristics of people who tend to be less afraid than others?  Using the metadata associated with users who voted and reranked this list, in combination with their other activity on the site, here are a few things I found.

– Ebola fear appears to be slightly less prevalent in the Northeast, as compared to other regions of the US, and slightly more prevalent in the South.

– Older people tend to be slightly less afraid of Ebola, often expressing more fear of Alzheimer’s.

– International visitors to this list are half as likely to vote for Ebola, as compared to Americans.

– People who are afraid of Ebola are 4.4x as likely to be afraid of Dengue Fever.

– People who are afraid of Strokes, Parkinson’s Disease, Muscular Distrophy, Influenza, and/or Depression are about half as likely to believe that Ebola is one of the world’s scariest diseases.

Bear in mind that these results are based on degree of fear and ALL people are afraid of Ebola.  The fear in some groups is simply less pronounced and only the last 3 results are statistically significant based on classical statistical methods.  There are plausible explanations for all of the above, ranging from the fact that conservative areas of the country are likely more responsive to potential threats, to the fact that losing one’s mind over time to Alzheimer’s really may be much scarier for older people versus a quick death, to the fact that people who are afraid of foreign diseases prevalent in tropical areas likely fear other foreign diseases prevalent in tropical areas.

To me the most interesting fact is that people who are afraid of more common everyday diseases, including Influenza, which kills thousands every year, appear to be less afraid of Ebola than others.  Human beings are wired to be more afraid of the new and spectacular, as much psychological research has shown.  That fear kept many of our ancestors alive, so I wouldn’t dismiss it as wrong.  But it is interesting to observe that perhaps some of us are less wired in this way than others.

– Ravi Iyer

by    in Opinion Graph, Rankings

Ranky Goes to Washington?

Something pretty cool happened last week here at Ranker, and it had nothing to do with the season premiere of the “Big Bang Theory”, which we’re also really excited about. Cincinnati’s number one digital paper used our widget to create a votable list of ideas mentioned in Cincinnati Mayor John Cranley’s first State of the City. As of right now, 1,958 voters cast 5,586 votes on the list of proposals from Mayor Cranley (not surprisingly, “fixing streets” ranks higher than the “German-style beer garden” that’s apparently also an option).

Now, our widget is used by thousands of websites to either take one of our votable lists or create their own and embed it on their site, but this was the very first time Ranker was used to directly poll people on public policy initiatives.

Here’s why we’re loving this idea: we feel confident that Ranker lists are the most fun and reliable way to poll people at scale about a list of items within a specific context. That’s what we’ve been obsessing about for the past 6 years. But we also think this could lead to a whole new way for people to weigh in in fairly  large numbers on complex public policy issues on an ongoing basis, from municipal budgets to foreign policy. That’s because Ranker is very good at getting a large number of people to cast their opinion about complex issues in ways that can’t be achieved at this scale through regular polling methods (nobody’s going to call you at dinner time to ask you to rank 10 or 20 municipal budget items … and what is “dinner time” these days, anyway?).  It may not be a representative sample, but it may be the only sample that matters, given that the average citizen of Cincinnati will have no idea about the details within the Mayor’s speech and likely will give any opinion simply to move a phone survey conversation along about a topic they know little about.

Of course, the democratic process is the best way to get the best sample (there’s little bias when it’s the whole friggin voting population!) to weigh in on public policy as a whole. But elections are very expensive, infrequent, and the focus of their policy debates is the broadest possible relative to their geographical units, meaning that micro-issues like these will often get lost in same the tired partisan debates.

Meanwhile, society, technology, and the economy no longer operate on cycles consistent with elections cycles: the rate and breadth of societal change is such that the public policy environment specific to an election quickly becomes obsolete, and new issues quickly need sorting out as they emerge, something our increasingly polarized legislative processes have a hard time doing.

Online polls are an imperfect, but necessary, way to evaluate public policy choices on an ongoing basis. Yes, they are susceptible to bias, but good statistical models can overcome a lot of such bias and in a world where the response rates for telephone polls continue to drop, there simply isn’t an alternative.  All polling is becoming a function of statistical modeling applied to imperfect datasets.  Offline polls are also expensive, and that cost is climbing as rapidly as response rates are dropping. A poll with a sample size of 800 can cost anywhere between $25,000 and $50,000 depending on the type of sample and the response rate.  Social media is, well, very approximate. As we’ve covered elsewhere in this blog, social media sentiment is noisy, biased, and overall very difficult to measure accurately.

In comes Ranker. The cost of that Cincinnati.com Ranker widget? $0. Its sample size? Nearly 2,000 people, or anywhere between 2 to 4x the average sample size of current political polls. Ranker is also the best way to get people to quickly and efficiently express a meaningful opinion about a complex set of issues, and we have collected thousands of precise opinions about conceptually complex topics like the scariest diseases and the most important life goals by making providing opinions entertaining within a context that makes simple actions meaningful.

Politics is the art of the possible, and we shouldn’t let the impossibility of perfect survey precision preclude the possibility of using technology to improve civic engagement at scale.  If you are an organization seeking to poll public opinion about a particular set of issues that may work well in a list format, we’d invite you to contact us.

– Ravi Iyer

Page 3 of 812345...Last »