'

Why I (More or Less) Trust the Nielsen Ratings

Categories: '

Written By

February 24th, 2008

We write about ratings all the time and sometimes the ratings for shows that we love aren't so good.   I love 30 Rock, you love Jericho or Moonlight (good news Moonlight fans, it  has the best ratings of all 3!).

Inevitably, we wind up getting comments and e-mail about how flawed, unreliable and untrustworthy the Nielsen counting is. 

I understand it, but after careful consideration and a little bit of research I think people who complain about Nielsen probably are going to need to get over it.

Over time, there will be better data.  Both by Nielsen and others.  More and more data will be available via set-top boxes, but in round numbers, only about 50% of the homes currently have any kind of set-top box. A  significant portion of the viewing population would be excluded by exclusive reliance on set-top box data. 

I don't think Nielsen's approach is perfect, or even close to perfect.  Not perfect by any measure, just like you and just like me.  But you know what?  Just like you and just like me, though we aren't perfect, it's good enough.  The TV networks say it's good enough, and the people buying the advertising time say it's good enough.   

That's right: the people who sell the product that pays for everything and the people who buy the product that pays for everything say it's good enough.  

Bill G. will vouch that sometimes I can be a little crazy.  If you need more references, that's easy for me to compile.  I wanted some anecdotal data so I randomly stopped people in public and pelted them with the following questions:

  • 1. Ever heard of the Super Bowl?
  • 2. What's the Super Bowl about?
  • 3. Ever heard of the Academy Awards/Oscars?
  • 4. What are the Academy Awards/Oscars About?
  • 5. Ever heard of the TV show Jericho?
  • 6. What's Jericho about?
  • 7. Ever heard of the TV show Moonlight?
  • 8. What's Moonlight About?
  • 9. Ever heard of American Idol?
  • 10. What's American Idol about?

I compiled the results.  If  you don't like my results, sorrry.  I don't like that I can't reel in Giselle Bundchen, but I got over it.   Or hopefully will someday soon.

There was 100% awareness of both the Super Bowl and the Academy Awards.  And if I accept  variants of "some sports thingy" as a correct answer, 100% of the people polled knew what the Super Bowl was.  This was not true in the case of the Oscars where although everyone had heard of it, only 88% could correctly state it was an awards show for the movie industry.

14% of the respondents said they'd heard of Jericho, but only 43% of that 14% could actually explain what Jericho was about.

Only 6% said they'd ever heard of Moonlight, of those 67% were able to correctly describe what the show is about.

But when it came to American Idol, 100% had heard of it, and if I accept "some talent show thingy" as an answer, 100% knew what the show was about.

If you want to complain about how networks decide on programming, timeslots, time and money spent on promotion - that's all fair and probably even productive for the networks if they're checking in.  

If you want to question the values of the United States of America based on what programs attract the most viewers, that's a values discussion that doesn't really have anything to do with Nielsen.  We're fine with people talking about that here, but I find that sort of discussion more enjoyable if you involve beer.

My survey was fairly random.  I didn't show up at the advance Jericho screening at Wondercon and poll the people there or Jericho would have fared dramatically better. That would have been a completely inaccurate sampling, too.  

As imperfect as Nielsen's sampling may be, with any and all  flaws, I still trust it more than my own random sampling.  But even if I didn't, it wouldn't mater.  It's good enough for the advertisers and the broadcast networks.  Until any of that changes, I bet on inertia.  Nothing will change, including the complaints about Nielsen when it comes to low-rated shows and shows on the bubble.

 
  • http://peter.jemts.com Peter

    I think more than anything, it's about people watching scripted television in general. I wonder what would've happened if you used NCIS (lesser known to CSI but still extremely popular) or even Two and a Half Men. I think that the Super Bowl, Academy Awards and AI have all completely transcended TV and become entities unto themselves. Even people who don't own a TV will have heard of them. This isn't to lower effectiveness of your random survey… because I think it's quite accurate, just to add as a note.

  • http://peter.jemts.com Peter

    I think more than anything, it’s about people watching scripted television in general. I wonder what would’ve happened if you used NCIS (lesser known to CSI but still extremely popular) or even Two and a Half Men. I think that the Super Bowl, Academy Awards and AI have all completely transcended TV and become entities unto themselves. Even people who don’t own a TV will have heard of them. This isn’t to lower effectiveness of your random survey… because I think it’s quite accurate, just to add as a note.

  • http://tvbythenumbers.com Robert Seidman

    I should've asked about CSI and Two and a Half Men. I'll ask around. It's a great point. Perhaps near 100% know of CSI and yet it's not watched by as many as AI.

    While the higher rated telecasts are generally highly correlated with telecasts with higher awareness, some correlations are higher than others. I'm thinking Miss America.

  • http://tvbythenumbers.com Robert Seidman

    I should've asked about CSI and Two and a Half Men. I'll ask around. It's a great point. Perhaps near 100% know of CSI and yet it's not watched by as many as AI.

    While the higher rated telecasts are generally highly correlated with telecasts with higher awareness, some correlations are higher than others. I'm thinking Miss America.

  • http://tvbythenumbers.com Robert Seidman

    I should’ve asked about CSI and Two and a Half Men. I’ll ask around. It’s a great point. Perhaps near 100% know of CSI and yet it’s not watched by as many as AI.

    While the higher rated telecasts are generally highly correlated with telecasts with higher awareness, some correlations are higher than others. I’m thinking Miss America.

  • Sue

    Well, I don't trust the Nielsen numbers because they aren't even close to being accurate! They don't represent the majority of viewers…….they are not a fair representation and until they give everyone a little black box, they never will be a fair or accurate representation of the actual viewers.
    And…….I'm not going to get over it!

  • Sue

    Well, I don't trust the Nielsen numbers because they aren't even close to being accurate! They don't represent the majority of viewers…….they are not a fair representation and until they give everyone a little black box, they never will be a fair or accurate representation of the actual viewers.
    And…….I'm not going to get over it!

  • Sue

    Well, I don’t trust the Nielsen numbers because they aren’t even close to being accurate! They don’t represent the majority of viewers…….they are not a fair representation and until they give everyone a little black box, they never will be a fair or accurate representation of the actual viewers.
    And…….I’m not going to get over it!

  • Seymour

    Sue, even if everyone had a little black box attached to their TV set, it still wouldn't count all the iTunes, Amazon Unbox, Netflix, and station direct viewing on computers. Viewing habits are changing. Neilsen, broadcasters, and advertisers need to change with the times.

  • Seymour

    Sue, even if everyone had a little black box attached to their TV set, it still wouldn't count all the iTunes, Amazon Unbox, Netflix, and station direct viewing on computers. Viewing habits are changing. Neilsen, broadcasters, and advertisers need to change with the times.

  • Steve

    I think I understand the whole representative thing – the viewing habits of a similar (demographics) Nielsen viewer is said to represent X number of viewiers (someone “like” me is a Nielsen viewer and I'm like X number of other people).

    This is what I can't grasp – if my personal/specific viewing is NOT being counted by Nielsen (which it isn't since I'm not connected to Nielsen), then how does my watching (or not watching) a show affect anything. After CBS renewed it, they made a big PR push for fans to watch Jericho when it aired (presumably so it would get higher ratings). OK, so now I watch it. But no one in the TV industry (CBS, advertisers, Nielsen) knows that.

    So while a Nielsen viewer may be representing me, why does it matter if I watch a show? Cause my watching the show has no effect on the ratings since I have no power to influence the Nielsen viewers.

    Furthermore, even if my personal demographics are adequately represented by one or more Nielsen viewers, why does it follow that my tastes match their tastes?

  • Steve

    I think I understand the whole representative thing – the viewing habits of a similar (demographics) Nielsen viewer is said to represent X number of viewiers (someone “like” me is a Nielsen viewer and I'm like X number of other people).

    This is what I can't grasp – if my personal/specific viewing is NOT being counted by Nielsen (which it isn't since I'm not connected to Nielsen), then how does my watching (or not watching) a show affect anything. After CBS renewed it, they made a big PR push for fans to watch Jericho when it aired (presumably so it would get higher ratings). OK, so now I watch it. But no one in the TV industry (CBS, advertisers, Nielsen) knows that.

    So while a Nielsen viewer may be representing me, why does it matter if I watch a show? Cause my watching the show has no effect on the ratings since I have no power to influence the Nielsen viewers.

    Furthermore, even if my personal demographics are adequately represented by one or more Nielsen viewers, why does it follow that my tastes match their tastes?

  • http://jerichomonster.blogspot.com/2008/01/bringing-guests-to-jerichon.html Jane

    If you want to know how bad Nielsen's are read this:

    http://copywriteink.blogspot.com/2008/02/counti

    Advertisers are even beginning to change their minds.

  • http://jerichomonster.blogspot.com/2008/01/bringing-guests-to-jerichon.html Jane

    If you want to know how bad Nielsen's are read this:

    http://copywriteink.blogspot.com/2008/02/counti

    Advertisers are even beginning to change their minds.

  • http://tvbythenumbers.com Robert Seidman

    Steve, my understanding is that if you're NOT a Nielsen family what you watch (and don't) doesn't affect anything.

    The theory behind that sort of sampling is if the sample is any good your tastes would be proportionally represented.

    Seymour, Nielsen has big plans with its anytime/anywhere measuring which is aimed at capturing all the stuff you're talking about, even what you watch when you're at a friend's house.

    But to call a spade a spade, if the millions of Jericho viewers who seemed to have abandonded the show since last year's finale were watching the show on CBS.com and iTunes, CBS would be pelting us with press releases talking about those millions of streams/downloads. CBS isn't doing that.

    Jane, my opinion (only that) is that the Nielsens are bad for Journeyman and Friday Night Lights because not enough people watch.

  • http://tvbythenumbers.com Robert Seidman

    Steve, my understanding is that if you're NOT a Nielsen family what you watch (and don't) doesn't affect anything.

    The theory behind that sort of sampling is if the sample is any good your tastes would be proportionally represented.

    Seymour, Nielsen has big plans with its anytime/anywhere measuring which is aimed at capturing all the stuff you're talking about, even what you watch when you're at a friend's house.

    But to call a spade a spade, if the millions of Jericho viewers who seemed to have abandonded the show since last year's finale were watching the show on CBS.com and iTunes, CBS would be pelting us with press releases talking about those millions of streams/downloads. CBS isn't doing that.

    Jane, my opinion (only that) is that the Nielsens are bad for Journeyman and Friday Night Lights because not enough people watch.

  • Gary

    Sue, sorry but the numbers are accurate within an acceptible margin of error. They do represent most viewers and what facts do you have to say that they are not accurate? Sounds like you just like throwing out objections. Every TV will never be mesured, because of the cost. Programmers, markerters and agencies could never afford the cost for a sample that large. You should take some comfort that we will likely see the national sample more than double in the coming years as each local market moves to meters. Nielsen is also testing how to capture video that is viewed on your computer.

  • Gary

    Sue, sorry but the numbers are accurate within an acceptible margin of error. They do represent most viewers and what facts do you have to say that they are not accurate? Sounds like you just like throwing out objections. Every TV will never be mesured, because of the cost. Programmers, markerters and agencies could never afford the cost for a sample that large. You should take some comfort that we will likely see the national sample more than double in the coming years as each local market moves to meters. Nielsen is also testing how to capture video that is viewed on your computer.

  • Steve

    “if you’re NOT a Nielsen family what you watch (and don’t) doesn’t affect anything”. I agree and I accept that my specific tastes are picked up by a good sampling.

    And I get that even if a billion or trillion people watch a particular show, it still may be cancelled cause it may have the wrong demographic for advertisers – no ads means no show.

    So how does a letter writing or nut campaign save a show? How does 100,000 or 1 million cans of nuts sent to CBS tell them that the senders are the right demographic in the right numbers? And if CBS thinks it is the right demo with sufficient viewers, then what does it say about Nielsen “undercounting” this show. (This has nothing specific to Jericho – I'm just using them as an example.)

  • Steve

    “if you’re NOT a Nielsen family what you watch (and don’t) doesn’t affect anything”. I agree and I accept that my specific tastes are picked up by a good sampling.

    And I get that even if a billion or trillion people watch a particular show, it still may be cancelled cause it may have the wrong demographic for advertisers – no ads means no show.

    So how does a letter writing or nut campaign save a show? How does 100,000 or 1 million cans of nuts sent to CBS tell them that the senders are the right demographic in the right numbers? And if CBS thinks it is the right demo with sufficient viewers, then what does it say about Nielsen “undercounting” this show. (This has nothing specific to Jericho – I'm just using them as an example.)

© 2014 Tribune Digital Ventures