Your Local Weather Is an Indication of Your Local Weather...And Only Your Local Weather

Categories: TV Ratings

Written By

November 23rd, 2012

This morning in reference to some Thanksgiving night programming/ratings I saw this comment from Bill:

If you can't read that because it's too tiny the quoted text is "If Twitter last night is any indication – and maybe it isn’t," to which Bill responded "Twitter is an indication of, Twitter."

He is absolutely correct. But, things are slow today (it's the day after Thanksgiving!) and I figured this was worth a quick post because the topic is the first cousin of "but I like it and everyone I know likes it..." and "I don't like it and nobody I know likes it..."  And 2nd cousin to  things like "I have an iPhone and everyone I know has an iPhone..."  where the "..." become "therefore everybody likes it," "therefore nobody likes it" and "therefore everyone has an iPhone.

Of course those are not not  reasonable conclusions. Indeed, as Bill notes, taking the temperature on Twitter is only indicative of the temperature on Twitter and that is an overstatement unless you follow everybody. Otherwise it is merely the temperature of a small subset of Twitter.

Nobody has a problem figuring out that jumping to conclusions about the weather around the world based on their own local weather is a bad idea. I doubt even the craziest of our commenters would ever say "But it's sunny and 65 where I am, how could it possibly be cold and rainy anywhere else in the world!??" And yes, yes, I know people do actually say that kind of thing in real life, but it's only to gloat over their own good weather at the expense of the less weather fortunate.

Yet when it comes to TV shows otherwise reasonable people are seemingly very often willing to jump to global conclusions based on small bits of local data - themselves, their friends, their twitter feed, etc. That flawed reasoning produces wacky comments like "'The Walking Dead' and "NCIS" are horrible shows, I don't like them and nobody I know likes them! This is just further proof about how flawed the Nielsen system is."

But nope, it's just a flaw in those folks' logic.

 
  • Common Anomaly

    If nobody watches Community, how can Nielsen continue to state that based on their measurements that millions of Americans watch Community?

  • DTravel

    Because I watch Community and there are millions of me.

    Agent Smith

  • Jeff (Canada)

    The Nielson system in itself is a small subset of the much larger community, so they themselves are saying “since this small group is watching something, that means a HUGE portion is also watching that same thing!”.

  • richard

    @ Jeff,

    yes that is a good point. idk, we really do need a better system, im just not willing to donate a cent to funding it :)

  • http://tvbythenumbers.com Robert Seidman

    @Jeff:

    I don’t know about the Nielson system, but the Nielsen system reports that huge portion (that’s why it’s reported that ~20 million people watch NCIS even though there are only about 50,000 people in ~25,000 households in the Nielsen panel).

    What the Nielsen panel does is attempt to create a statistical panel that represents *everybody* based on their universe estimates (instead of just a panel of “people like you” or people who like “Community” or people who like science fiction, etc. While I can’t speak to how well they succeed at mapping their panel to their universe estimates, I’d guess they do a much, much better job of representing everybody than anyone’s twitter feed.

  • http://tvbythenumbers.com Bill Gorman

    “@Jeff:

    I don’t know about the Nielson system, but the Nielsen system reports that huge portion (that’s why it’s reported that ~20 million people watch NCIS even though there are only about 50,000 people in ~25,000 households in the Nielsen panel).

    What the Nielsen panel does is attempt to create statistical a panel that represents *everybody* based on their universe estimates (instead of just a panel of “people like you” or people who like “Community” or people who like science fiction, etc. While I can’t speak to how well they succeed at mapping their panel to their universe estimates, I’d guess they do a much, much better job of representing everybody than anyone’s twitter feed.”

    Likelihood of convincing a Nielsen doubter through reasoned explanation: 0%!

  • http://tvbythenumbers.com Robert Seidman

    On the heels of Thanksgiving I will charitably (and perhaps “ridiculously,” “foolishly” and “worthlessly”) hold out 1% hope!

  • Steve

    Presidential polling was thought to be as inaccurate as the Nielsen system too, but it was actually verified through the election to be true. (Whether you wanted it to be or not to be). If you really want to go crazy, try being a Neilsen household and save a show all by yourself. It doesn’t work (I know, I was one), Baywatch Nights still got cancelled.

  • CrimTV

    so Nielson ratings aren’t entirely accurate? They say Revolution has 7 million viewers when it actually might have 9 million, or is it not that inaccurate?

  • Justin121

    Jeff (Canada) seems like a reasonable guy. Therefore, here’s my response:

    Nielsen is a credible statistical institution, while the “everybody I know” peeps aren’t.

    Case closed.

  • Doug

    Wow, it must be a reaaal slow slow. It was the third paragraph before I even got the point of that.

  • Paul

    @CrimTV, Nielsen ratings are not entirely accurate. They are based on a sample of the total population. If you know the sample size you can use statistical formula to calculate how accurate the ratings are. Basically, you would then calculate that there is an x% probability that the ratings are within a certain margin of error.

    If you get a larger sample, the probability gets closer to 100% and the margin of error gets smaller. The only way to get 100% accurate ratings would be to make every American household a Nielsen household.

  • richard

    @ Paul

    Then I say we should! In theory, if that were to happen how much would it cost?

  • Doug

    @Richard – Yes, somebody could totally devise a system that tracks what every single American watches. A black box in everybody’s home, recording everything. There would be no opposition to that whatsoever ;)

  • BOTR

    Am I the only one that thought this was going to be an article making fun of the Weather Channel?

  • Paul

    @ richard, there are more than 100,000,000 households in the US. I don’t know how much a Nielsen box costs, but manufacturing them and installing them in every household would probably cost several billion dollars.

  • Kitsune

    If you know anything about statistics, then it’s reasonable to assume that 50,000 is a decent enough sample size, as the distribution approaches normality long before that.

    Basically, the difference between 50,000 samples and 100 million samples is the difference between 99% accuracy and 100% accuracy.

  • Oliver

    @Kitsune

    Two issues:

    1. Cancellation decisions are regularly made using 0.2% of the audience, which is within Nielsen’s margin of error.

    2. The sample needs to be representative and not statistically biased.

    In the past, it has been incredibly difficult for networks to check whether Nielsen was statistically biased but with Nielsen now tracking online viewing, it should become pretty clear whether there’s a discrepancy between Nielsen’s numbers and the network’s own internal online analytics.

  • eridapo

    @ CrimTV

    so Nielson ratings aren’t entirely accurate? They say Revolution has 7 million viewers when it actually might have 9 million, or is it not that inaccurate?

    The answer is yes because like all statistical models there is a measurement error. Nielsen has never reveal exactly what their measurement sampling error is (like most election day polls do), but it is estimated to be 2% to 4%.

    In 2007 the NY Times confronted Nielsen about its sampling error, but it failed to get Nielsen to reveal what it is. A Nielsen spokesperson, Gary Holmes, provided the times an estimate of +/1 3/10 as a possible margin of error.

    http://www.nytimes.com/2007/04/08/opinion/08pubed.html

    If true the article states, there is no way you can really tell the difference between a show that scored a 2.0 and a 2.1 (or 2.2).

    There is no doubt, however, that overtime and with additional measurements, the Nielsen measurement error is reduced.

    Therefore.. the weekly up and down of a tenth in the ratings could all be due to simple measurement error, but if a show over repeated airings achieves an average of 2.2 it is likely that is the number of people that are watching it. This is why Premiere numbers are not indicative of the actual performance of a show over time. You have to get maybe six or seven viewings before you achieve a number closer to reality. Unfortunately the Networks don’t have that kind a time. If a show Premiere’s weak, the probability of going up is small.

    http://www.nytimes.com/2007/04/08/opinion/08pubed.html

© 2014 Tribune Digital Ventures