'

Why I (More or Less) Trust the Nielsen Ratings

Categories: '

Written By

February 24th, 2008

We write about ratings all the time and sometimes the ratings for shows that we love aren't so good.   I love 30 Rock, you love Jericho or Moonlight (good news Moonlight fans, it  has the best ratings of all 3!).

Inevitably, we wind up getting comments and e-mail about how flawed, unreliable and untrustworthy the Nielsen counting is. 

I understand it, but after careful consideration and a little bit of research I think people who complain about Nielsen probably are going to need to get over it.

Over time, there will be better data.  Both by Nielsen and others.  More and more data will be available via set-top boxes, but in round numbers, only about 50% of the homes currently have any kind of set-top box. A  significant portion of the viewing population would be excluded by exclusive reliance on set-top box data. 

I don't think Nielsen's approach is perfect, or even close to perfect.  Not perfect by any measure, just like you and just like me.  But you know what?  Just like you and just like me, though we aren't perfect, it's good enough.  The TV networks say it's good enough, and the people buying the advertising time say it's good enough.   

That's right: the people who sell the product that pays for everything and the people who buy the product that pays for everything say it's good enough.  

Bill G. will vouch that sometimes I can be a little crazy.  If you need more references, that's easy for me to compile.  I wanted some anecdotal data so I randomly stopped people in public and pelted them with the following questions:

  • 1. Ever heard of the Super Bowl?
  • 2. What's the Super Bowl about?
  • 3. Ever heard of the Academy Awards/Oscars?
  • 4. What are the Academy Awards/Oscars About?
  • 5. Ever heard of the TV show Jericho?
  • 6. What's Jericho about?
  • 7. Ever heard of the TV show Moonlight?
  • 8. What's Moonlight About?
  • 9. Ever heard of American Idol?
  • 10. What's American Idol about?

I compiled the results.  If  you don't like my results, sorrry.  I don't like that I can't reel in Giselle Bundchen, but I got over it.   Or hopefully will someday soon.

There was 100% awareness of both the Super Bowl and the Academy Awards.  And if I accept  variants of "some sports thingy" as a correct answer, 100% of the people polled knew what the Super Bowl was.  This was not true in the case of the Oscars where although everyone had heard of it, only 88% could correctly state it was an awards show for the movie industry.

14% of the respondents said they'd heard of Jericho, but only 43% of that 14% could actually explain what Jericho was about.

Only 6% said they'd ever heard of Moonlight, of those 67% were able to correctly describe what the show is about.

But when it came to American Idol, 100% had heard of it, and if I accept "some talent show thingy" as an answer, 100% knew what the show was about.

If you want to complain about how networks decide on programming, timeslots, time and money spent on promotion - that's all fair and probably even productive for the networks if they're checking in.  

If you want to question the values of the United States of America based on what programs attract the most viewers, that's a values discussion that doesn't really have anything to do with Nielsen.  We're fine with people talking about that here, but I find that sort of discussion more enjoyable if you involve beer.

My survey was fairly random.  I didn't show up at the advance Jericho screening at Wondercon and poll the people there or Jericho would have fared dramatically better. That would have been a completely inaccurate sampling, too.  

As imperfect as Nielsen's sampling may be, with any and all  flaws, I still trust it more than my own random sampling.  But even if I didn't, it wouldn't mater.  It's good enough for the advertisers and the broadcast networks.  Until any of that changes, I bet on inertia.  Nothing will change, including the complaints about Nielsen when it comes to low-rated shows and shows on the bubble.

 
  • http://TVoholic.com fred

    Well, it's true what Peter said, you used shows/events that are to be known from people who don't even own a TV.

    You took the #1 unscripted show, but Jericho or Moonlight are far from being #1 scripted shows!
    Asking for CSI, House or Grey's Anatomy might have given different results, just like asking for American Gladiators, Dance War or Clash Of The Choirs…

    But even further than that, there's still one flaw with that survey : you asked random people. Only we don't want to hear about random people here, but TV viewers.
    Maybe 14% of people of heard of Jericho, but amongst all (regular) TV viewers, that number is much higher.

    I'm not saying you're completely wrong here, or that millions are actually watching Jericho, because (sadly) it doesn't seem to be the case, but while not everyone knows what Lost is (about), a much bigger percentage of people will know about it amongst (regular) TV viewers than the whole population, and that should matter.

    It's no mystery that the regular TV viewer has evolved, and doesn't watch TV the way he used to years ago. Question is, what about the regular Nielsen TV viewers ?

  • http://TVoholic.com fred

    Well, it's true what Peter said, you used shows/events that are to be known from people who don't even own a TV.

    You took the #1 unscripted show, but Jericho or Moonlight are far from being #1 scripted shows!
    Asking for CSI, House or Grey's Anatomy might have given different results, just like asking for American Gladiators, Dance War or Clash Of The Choirs…

    But even further than that, there's still one flaw with that survey : you asked random people. Only we don't want to hear about random people here, but TV viewers.
    Maybe 14% of people of heard of Jericho, but amongst all (regular) TV viewers, that number is much higher.

    I'm not saying you're completely wrong here, or that millions are actually watching Jericho, because (sadly) it doesn't seem to be the case, but while not everyone knows what Lost is (about), a much bigger percentage of people will know about it amongst (regular) TV viewers than the whole population, and that should matter.

    It's no mystery that the regular TV viewer has evolved, and doesn't watch TV the way he used to years ago. Question is, what about the regular Nielsen TV viewers ?

  • Seymour

    Did you know that 50 percent of all statistics are made up on the spot?

  • Seymour

    Did you know that 50 percent of all statistics are made up on the spot?

  • Seymour

    Sue, even if everyone had a little black box attached to their TV set, it still wouldn’t count all the iTunes, Amazon Unbox, Netflix, and station direct viewing on computers. Viewing habits are changing. Neilsen, broadcasters, and advertisers need to change with the times.

  • Steve

    I think I understand the whole representative thing – the viewing habits of a similar (demographics) Nielsen viewer is said to represent X number of viewiers (someone “like” me is a Nielsen viewer and I’m like X number of other people).

    This is what I can’t grasp – if my personal/specific viewing is NOT being counted by Nielsen (which it isn’t since I’m not connected to Nielsen), then how does my watching (or not watching) a show affect anything. After CBS renewed it, they made a big PR push for fans to watch Jericho when it aired (presumably so it would get higher ratings). OK, so now I watch it. But no one in the TV industry (CBS, advertisers, Nielsen) knows that.

    So while a Nielsen viewer may be representing me, why does it matter if I watch a show? Cause my watching the show has no effect on the ratings since I have no power to influence the Nielsen viewers.

    Furthermore, even if my personal demographics are adequately represented by one or more Nielsen viewers, why does it follow that my tastes match their tastes?

  • http://tvbythenumbers.com Robert Seidman

    Seymour, I'm 85% sure I knew that. By the way, just announced by Nielsen: a $3.5 million study to track “all-day media usage”. They're actually going to follow people around!

  • http://tvbythenumbers.com Robert Seidman

    Seymour, I'm 85% sure I knew that. By the way, just announced by Nielsen: a $3.5 million study to track “all-day media usage”. They're actually going to follow people around!

  • Saving$$$

    Did you know with an S-video cord everything you download can be viewed on your tv. I got rid of cable a long time ago. I save over $1000/yr because of it. I spend about $6-8 a week for the shows I like. It is superior quality and I can watch the video file over and over. I will never go back to conventional viewing.

  • Saving$$$

    Did you know with an S-video cord everything you download can be viewed on your tv. I got rid of cable a long time ago. I save over $1000/yr because of it. I spend about $6-8 a week for the shows I like. It is superior quality and I can watch the video file over and over. I will never go back to conventional viewing.

  • http://jerichomonster.blogspot.com/2008/01/bringing-guests-to-jerichon.html Jane

    If you want to know how bad Nielsen’s are read this:

    http://copywriteink.blogspot.com/2008/02/counting-beans-journeyman-vs-friday.html

    Advertisers are even beginning to change their minds.

  • http://tvbythenumbers.com Robert Seidman

    Steve, my understanding is that if you’re NOT a Nielsen family what you watch (and don’t) doesn’t affect anything.

    The theory behind that sort of sampling is if the sample is any good your tastes would be proportionally represented.

    Seymour, Nielsen has big plans with its anytime/anywhere measuring which is aimed at capturing all the stuff you’re talking about, even what you watch when you’re at a friend’s house.

    But to call a spade a spade, if the millions of Jericho viewers who seemed to have abandonded the show since last year’s finale were watching the show on CBS.com and iTunes, CBS would be pelting us with press releases talking about those millions of streams/downloads. CBS isn’t doing that.

    Jane, my opinion (only that) is that the Nielsens are bad for Journeyman and Friday Night Lights because not enough people watch.

  • Seymour

    That will be a very useful study. I know I spent over 3 hours watching “Better know a…” on the Colbert Report online last weekend. I stopped because I got tired of the same 10 second commercials.

  • Seymour

    That will be a very useful study. I know I spent over 3 hours watching “Better know a…” on the Colbert Report online last weekend. I stopped because I got tired of the same 10 second commercials.

  • Gary

    Sue, sorry but the numbers are accurate within an acceptible margin of error. They do represent most viewers and what facts do you have to say that they are not accurate? Sounds like you just like throwing out objections. Every TV will never be mesured, because of the cost. Programmers, markerters and agencies could never afford the cost for a sample that large. You should take some comfort that we will likely see the national sample more than double in the coming years as each local market moves to meters. Nielsen is also testing how to capture video that is viewed on your computer.

  • Steve

    “if you’re NOT a Nielsen family what you watch (and don’t) doesn’t affect anything”. I agree and I accept that my specific tastes are picked up by a good sampling.

    And I get that even if a billion or trillion people watch a particular show, it still may be cancelled cause it may have the wrong demographic for advertisers – no ads means no show.

    So how does a letter writing or nut campaign save a show? How does 100,000 or 1 million cans of nuts sent to CBS tell them that the senders are the right demographic in the right numbers? And if CBS thinks it is the right demo with sufficient viewers, then what does it say about Nielsen “undercounting” this show. (This has nothing specific to Jericho – I’m just using them as an example.)

  • http://TVoholic.com fred

    Well, it’s true what Peter said, you used shows/events that are to be known from people who don’t even own a TV.

    You took the #1 unscripted show, but Jericho or Moonlight are far from being #1 scripted shows!
    Asking for CSI, House or Grey’s Anatomy might have given different results, just like asking for American Gladiators, Dance War or Clash Of The Choirs…

    But even further than that, there’s still one flaw with that survey : you asked random people. Only we don’t want to hear about random people here, but TV viewers.
    Maybe 14% of people of heard of Jericho, but amongst all (regular) TV viewers, that number is much higher.

    I’m not saying you’re completely wrong here, or that millions are actually watching Jericho, because (sadly) it doesn’t seem to be the case, but while not everyone knows what Lost is (about), a much bigger percentage of people will know about it amongst (regular) TV viewers than the whole population, and that should matter.

    It’s no mystery that the regular TV viewer has evolved, and doesn’t watch TV the way he used to years ago. Question is, what about the regular Nielsen TV viewers ?

  • Seymour

    Did you know that 50 percent of all statistics are made up on the spot?

  • http://peter.jemts.com Peter

    Ok first off, that junk post is hilarious. Next, I think it's important not to ask about House or Lost (these have also jumped above TV itself) and use NCIS. Personally, I love NCIS, but I don't know a single other person that watches it, or even has heard of it and knows what NCIS stands for. No one uses “NCIS” in pop culture the way “CSI” is used (which comes up in movies like Superbad).

    Everyone should really stop complaining about bad sampling data. You have two choices, either you can get a little clicker in which you can tell “Big Brother” what you're watching all day. Or they can bug and monitor every piece of your life constantly. I'm pretty sure people don't like the latter, and the former… well people always lie, so I don't think we can trust that data.

    For all I know every person in Texas turns on CSI and every senior turns on CBS at 8pm and falls asleep so every crime drama on CBS gets big ratings while they snooze. No, Nielson isn't perfectly accurate, but there are over 300 million people in the US, and your 15 friends that all started watching Jericho (I'm one of those friends FYI) may not have the same impact you thought.

  • http://peter.jemts.com Peter

    Ok first off, that junk post is hilarious. Next, I think it's important not to ask about House or Lost (these have also jumped above TV itself) and use NCIS. Personally, I love NCIS, but I don't know a single other person that watches it, or even has heard of it and knows what NCIS stands for. No one uses “NCIS” in pop culture the way “CSI” is used (which comes up in movies like Superbad).

    Everyone should really stop complaining about bad sampling data. You have two choices, either you can get a little clicker in which you can tell “Big Brother” what you're watching all day. Or they can bug and monitor every piece of your life constantly. I'm pretty sure people don't like the latter, and the former… well people always lie, so I don't think we can trust that data.

    For all I know every person in Texas turns on CSI and every senior turns on CBS at 8pm and falls asleep so every crime drama on CBS gets big ratings while they snooze. No, Nielson isn't perfectly accurate, but there are over 300 million people in the US, and your 15 friends that all started watching Jericho (I'm one of those friends FYI) may not have the same impact you thought.

© 2014 Tribune Digital Ventures