Dear TV Fans: There Is No TV Ratings Measurement System That Will Make Everyone Happy

Categories: 1-Featured,Broadcast TV,Cable TV,Help

Written By

July 8th, 2012

Whenever there are Twitter Q&A's from  Syfy's Craig Engler (@Syfy) or USA's Ted Linhart (@TedOnTV) I just can't look away. Having seen many of the Twitter Q&A sessions it's well into repeat questions and different variants of the same questions. I still can't look away! There are always lots of questions about Nielsen and measurement. Many fans have a quarrel with Nielsen, but I've come to the conclusion that most of them would have a quarrel with TV ratings measurement no matter what the system of measurement was.

As long as there are fans of TV shows, there will be disappointed fans when shows get canceled. No measurement system can change that and whatever the measurement system, low-rated shows will typically be canceled.

There are several types of complaints/improvement suggestions that come up, here are a few of the most frequent:

We have the technology! We could measure everybody! Why don't they measure everybody!

The technological capabilities to achieve that probably exist but that technology doesn't actually exist yet. My bet is it never exists, even if the technology vastly improves. There are a lot of reasons why I think that but the top two are:

  1. lots of people don't want their viewing measured (they view it as an invasion of privacy)
  2. $$$$$ - It would cost too much money

There are other reasons besides those, but both of those are show stoppers. Would a complete census be more accurate than Nielsen? If you could get it, it would, without a doubt, be more accurate. But TV ratings measurement exists for the purpose of buying/selling TV advertising. The networks and advertisers aren't going to be willing to pay for it and as expensive as Nielsen is (and it's very expensive) the census style system would be multiple orders of magnitude more expensive to maintain and manage. The networks and advertisers aren't going to pay for something like that for a system that might only be a little more accurate.

On top of that you'd still probably need Nielsen or something like it because the census system would have so much data to crunch it wouldn't likely be able to produce fast national ratings the next morning and final ratings the next afternoon. The networks need the information fast so they can react and make scheduling decisions.

But what about data from all those set top boxes - that's a lot bigger sample than Nielsen!

It is indeed a bigger sample than Nielsen. But in at least a couple of ways it's not what advertisers (at least buyers of national advertising) want.

  1. it only measures people with set top boxes. Advertisers want to measure everyone, even the people still using rabbit ears
  2. it measures household viewing but doesn't tell you *who* in the household watched. Advertisers (at least buyers of national advertising) want to know who watched, what their gender is, what their age is, etc. set top box data isn't that granular.

the good news is the data is available and is being used.  Usually it is in addition to Nielsen and not a replacement though recently there have been reports that a few local stations are switching to using set top box data to sell local advertising spots. My understanding is that those numbers aren't churned out as rapidly as Nielsen reports ratings either.

Neftlix, On Demand, DVR, iTunes and Online Viewing Aren't Counted and they should be!

This is a fairly common one and not just with fans of TV shows. I've seen producers of shows with similar laments. First, let's talk about the "not counted" piece. DVD, Netflix, and iTunes certainly aren't counted in the Nielsen ratings. That's because the purpose of the Nielsen ratings is to measure in order to broker the sale of TV advertising, not to come up with some type of "total popularity" metric. Since Netflix, iTunes and DVDs don't have advertising, they're not counted in the Nielsen ratings. But that's not the same thing as "they're not counted." Of course sales of DVDs and iTunes episodes are counted and the people who need to know, know!  You can pretty much count on any deal where how much Netflix pays to license content depends on how many people watched on Netflix, it gets counted.  There are several reasons why months and months and months after the fact nobody reports a total popularity metric - but you can start with months and months and months after the fact it's not in anyone's interest to pay money to figure all that out. The people who need the information to make decisions have the information.

DVR viewing *is* measured by Nielsen.  Currently advertisers pay for commercials watched up to 3 days after the program initially aired on television (these ratings are commonly called C3 or C+3 ratings).  That was the compromise reached between the networks and the advertisers though the networks are reportedly still pushing for 7 days instead of 3.  In any event Nielsen measures commercial viewing live and up to three days after. On the program ratings side Nielsen measures DVR viewing up to 7 days after and can slice that in a variety of ways (the common slices are the same day DVR viewing included in the Live+SD daily ratings, L+3 which includes live plus 3 days of viewing and L+7 which is live plus seven days).

On Demand and Online viewing can be tracked by Nielsen but only up to 3 days after the initial telecast (for purposes of the C3 ratings advertisers use) and only if they contain the same national commercials that aired on TV. Some networks are playing around with this but particularly on the online side it looks like for now most networks are shying away from full commercial loads for online viewing. I'm not sure if that's because the networks don't want to put off online viewers by adding (LOTS) more commercials or whether the reality is that despite the measurement capabilities advertisers don't want to pay the same rates for online viewing or a combination of both.


Another thing to keep in mind is that whatever the measurement system, many of the things you (and the networks, for that matter) don't like aren't a function of measurement but rather what the advertisers are willing to pay for. So, for example, a new measurement system wouldn't change that broadcast primetime advertising is sold on the basis of adults 18-49 and subsets of adults 18-49. That's not about measurement, that's about what the folks with the big briefcases full of money want to spend it on.

  • Max Vrany

    With regards to the sample size, don’t forget that most national public opinion polls (politics and such) can have a sample size as low as 500, with most falling in the 1000-2000 range. Nielsen has about 26000 homes. Do the math.

  • Jon23812

    They should put a pop up thing when a show is on and people click on a button to let the TV station know they are looking at the program. It’s that simple.

  • panda22

    Have to agree that my main issue os the sample size, it’s too small!
    You need at least 100,000 homes to get an accurate sample imo, i also belive Neilsen target specific demogaphic homes with certain incomes, i bet they vast majority are white middle class searners too!

  • Holly


    Is there a scientific/statistical basis for your minimum sample size?

    And the Nielsen sample is designed to mirror the general population as much as is possible, so the percentage of white, middle class viewers in the sample will be roughly the same as the percentage in the US population.

  • Jasterisk

    Am I the only one that thinks that knowing the watching habits of ever person in America could actually hurt some shows. Instead of a 2.0 rating you find out that your show on the bubble actually only gets a 1.5 and boom it’s cancelled. A lot of people are thinking that it would help their show. I’m not so sure. I know that a lot of my friends watch 30 Rock, but never do so in actually time. They always watch on internet. I think we will eventually see ads in the TV shows (like on youtube) so that watching something on the internet will be eventually counted no matter when you watch it.

  • Robert Seidman

    Jasterisk: no you aren’t! While there are definitely some people who think the ratings system is a big conspiracy to punish shows they love and reward shows they hate (particularly reality shows), of course it’s just as likely that any improvement to measurement would be *bad* news for some fans.

  • Ultima

    Am I the only one that thinks that knowing the watching habits of ever person in America could actually hurt some shows.

    Could you imagine screams of despair if Nielsen increased their sample size tenfold and the ratings for shows like American Idol increased while those for a handful of low-rated “smart” shows decreased?

  • Ged

    Look at these Nielsen fanboys getting so defensive about this out of date inaccurate system.


  • Holly


    Some questions:

    How exactly would the competition work? Would networks and advertisers be somehow forced to pay two different companies for numbers? If not, and each net and advertiser picked on or the other, wouldn’t they be working off of different sets of numbers? How would they agree on prices? Or, if they just started setting them by the second company, wouldn’t that simply create another de facto monopoly?

    Also, what would this other company look like? How would it fix the problems you see in Nielsen? And how would it be financed?

  • Robert Seidman

    @Holly: looks like Ged is also Fugitive and a host of other characters ( though only those two on this post.) Pretty sure he doesn’t want to engage in thoughtful discussion.

  • Holly

    I thought it unlikely, but I keep hoping we can get some reasoned discussion….

    I actually wish one of these posters who seem so positive that a new, better system would be easy would come up with good answers so maybe some improvements can be made.

  • DTravel

    Interesting that this was posted a day after this article about movie production profits: I know the two are different but I suspect there’s a significant amount of overlap in how the two industries generate revenue.

  • Kyle7

    “But with a different *better* system, there’d be a totally different list of so-called low-rated shows. The stupid reality shows would be the first to go because *everyone* says how stupid they are and who really watches those things? Shows with cult followings, with the most vocal, loyal fan base (as opposed to the coveted “casual viewers”) would be the top ten shows because loyal fans write to advertisers in support of their fave show, and a new system would prove there are more popular fan-supported shows than the dumb shows Nielsen says are the top 10. Of course, low-rated shows would still be canceled but the point of having a new ratings system would be so fans could really support – and save – their favorite shows, so they wouldn’t be unfairly “low-rated.” ~Keppel1~”

    Does this system measure the loudness of fans’ complaining? The definition of a cult is that there’s a small but devoted following, emphasis on the “small.” To advertisers, the fact that X show has super-passionate fans, but a small total number of them, is meaningless if they can find a show that hits the same general demographic but in much larger numbers. In some cases the production studio may decide that DVD/merchandise/whatever sales may make it financially feasible to lower the licensing fee to the network, and that will trickle down to advertisers having to pay less, but unless a few very vocal fans are willing to buy a ton of the advertisers’ merchandise, the fact that they’re loud doesn’t matter.

    And I laugh at this notion of cult shows being “unfairly” low-rated. Unless you can prove that Nielsen is actually grossly inaccurate, then it comes down to there not being enough people watching the shows you like, no matter how vocal they are.

  • Craig

    There would be no need to put chips in set top boxes. It can all be done in software. The subscribers would need to make profiles, each profile holds the info of each viewer so when you go sit down to watch a show you select your name or select family if everyone in the household is watching (ie superbowl) and automatically sends data for every profile. Households can opt in or out of the ratings system when they first get their cable or satellite service by on screen notice with incentive to opt in to make sure your favorite shows stay on the air.
    The technical side of this is soo easy its not even funny. The only problems are the legal and financial sides. Every company is so money hungry and they dont want to spend money on anything that doesnt guarantee profit. The cable and satellite companies would have to spend lots of money redesigning new software for their boxes and then the time and bandwidth to roll out updates to existing customers. Then the legal contracts between providers and nielsen ratings. Lots of hurdles to overcome such a simple solution… They will never be able to collect data on everyone but at least they would have a bigger piece of the pie.

  • Robert Seidman

    Craig: in addition to the huge barriers you mention there are many more, including you giving too much credit to the capabilities of a large chunk of the installed STBs.

    Creating the front end opt in button is easy but such profiles would be beyond the capabilities of many boxes. The backend software to collect the data and crunch it rapidly would not be easy or cheap. Plus there needs to be a mechanism to validate profiles to ensure that I can’t create five fictional profiles who all love Louie.

    A bigger piece of a self selecting pie is probably not so interesting to advertisers.

  • Jasterisk

    Not to mention the fact that you would have to get people (many of them couch potatoes) to participate. Good Luck with that.

  • Ray32

    Also, I wonder whether the larger sweeps samples turn in significantly different data…which could indicate whether a larger overall sample would be worth the extra cost.

    That is virtually impossible. Mathematically, the ratio of sample size to overall population does not affect accuracy. That’s counterintuitive, I know, but it’s true.

    The required size of the sample depends only on how narrow you want your margin or error. So if you’re okay with a +-1.0 rating-points error, you don’t as big a sample as if you want a +-0.1 error.

    From a producer’s point of view, the idea of a bigger sample is still not without merit, though: Back in the ’80s and ’90s, when the networks had large numbers of viewers, small errors did not enter into renewal decisions. Nowadays, where decisions might be made between a show with a 2.2 rating and another with a 2.4, the margin of error may doom a show.

    Of course, producers don’t pay Nielsen. The networks and advertisers buy Nielsen, and they may not want to pay a lot more money just to make the system more fair without making it more effective.

  • hammard

    There is a system they are trialling in France which may change this. It is essentially about trying to measure if people watch the full length of a commercial and they’re trying links to things like mobile phones (e.g. link your phone to a box which receives data on who you are and what you are watching) or the internet (dropping a cookie when the full length of a commercial is viewed). The problem is 1) how to ensure accuracy and 2) what sample to measure. If it does somehow work we’d probably see more complaints as it would discount people who fast-forward or leave the room during advertising.

  • Sally

    How does Nielsen acquire sample households? Just because they call up random households and ask for their participation in their data collection, it’s still not truly “random” if the people they call up refuses. The people who agree to do this are specific, and not truly representative of the rest of the nation. It’s like the fallacies of phone surveys, only the people willing to talk on the phone are surveyed.

  • Holly


    I’m not disagreeing, and I think if there were major differences for any of the bigger networks we’d hear about it at some point. I just think it would be interesting to actually see the data.

    Although, since the overnight numbers are from major metro areas and the sweeps diaries include smaller markets, you may see more difference than normal because of the differences in viewing habits/preferences between NYC/LA and everywhere else (you can see some of that difference in the market numbers).

© 2015 Tribune Digital Ventures