Dear TV Fans: There Is No TV Ratings Measurement System That Will Make Everyone Happy

Categories: 1-Featured,Broadcast TV,Cable TV,Help

Written By

July 8th, 2012

Whenever there are Twitter Q&A's from  Syfy's Craig Engler (@Syfy) or USA's Ted Linhart (@TedOnTV) I just can't look away. Having seen many of the Twitter Q&A sessions it's well into repeat questions and different variants of the same questions. I still can't look away! There are always lots of questions about Nielsen and measurement. Many fans have a quarrel with Nielsen, but I've come to the conclusion that most of them would have a quarrel with TV ratings measurement no matter what the system of measurement was.

As long as there are fans of TV shows, there will be disappointed fans when shows get canceled. No measurement system can change that and whatever the measurement system, low-rated shows will typically be canceled.

There are several types of complaints/improvement suggestions that come up, here are a few of the most frequent:

We have the technology! We could measure everybody! Why don't they measure everybody!

The technological capabilities to achieve that probably exist but that technology doesn't actually exist yet. My bet is it never exists, even if the technology vastly improves. There are a lot of reasons why I think that but the top two are:

  1. lots of people don't want their viewing measured (they view it as an invasion of privacy)
  2. $$$$$ - It would cost too much money

There are other reasons besides those, but both of those are show stoppers. Would a complete census be more accurate than Nielsen? If you could get it, it would, without a doubt, be more accurate. But TV ratings measurement exists for the purpose of buying/selling TV advertising. The networks and advertisers aren't going to be willing to pay for it and as expensive as Nielsen is (and it's very expensive) the census style system would be multiple orders of magnitude more expensive to maintain and manage. The networks and advertisers aren't going to pay for something like that for a system that might only be a little more accurate.

On top of that you'd still probably need Nielsen or something like it because the census system would have so much data to crunch it wouldn't likely be able to produce fast national ratings the next morning and final ratings the next afternoon. The networks need the information fast so they can react and make scheduling decisions.

But what about data from all those set top boxes - that's a lot bigger sample than Nielsen!

It is indeed a bigger sample than Nielsen. But in at least a couple of ways it's not what advertisers (at least buyers of national advertising) want.

  1. it only measures people with set top boxes. Advertisers want to measure everyone, even the people still using rabbit ears
  2. it measures household viewing but doesn't tell you *who* in the household watched. Advertisers (at least buyers of national advertising) want to know who watched, what their gender is, what their age is, etc. set top box data isn't that granular.

the good news is the data is available and is being used.  Usually it is in addition to Nielsen and not a replacement though recently there have been reports that a few local stations are switching to using set top box data to sell local advertising spots. My understanding is that those numbers aren't churned out as rapidly as Nielsen reports ratings either.

Neftlix, On Demand, DVR, iTunes and Online Viewing Aren't Counted and they should be!

This is a fairly common one and not just with fans of TV shows. I've seen producers of shows with similar laments. First, let's talk about the "not counted" piece. DVD, Netflix, and iTunes certainly aren't counted in the Nielsen ratings. That's because the purpose of the Nielsen ratings is to measure in order to broker the sale of TV advertising, not to come up with some type of "total popularity" metric. Since Netflix, iTunes and DVDs don't have advertising, they're not counted in the Nielsen ratings. But that's not the same thing as "they're not counted." Of course sales of DVDs and iTunes episodes are counted and the people who need to know, know!  You can pretty much count on any deal where how much Netflix pays to license content depends on how many people watched on Netflix, it gets counted.  There are several reasons why months and months and months after the fact nobody reports a total popularity metric - but you can start with months and months and months after the fact it's not in anyone's interest to pay money to figure all that out. The people who need the information to make decisions have the information.

DVR viewing *is* measured by Nielsen.  Currently advertisers pay for commercials watched up to 3 days after the program initially aired on television (these ratings are commonly called C3 or C+3 ratings).  That was the compromise reached between the networks and the advertisers though the networks are reportedly still pushing for 7 days instead of 3.  In any event Nielsen measures commercial viewing live and up to three days after. On the program ratings side Nielsen measures DVR viewing up to 7 days after and can slice that in a variety of ways (the common slices are the same day DVR viewing included in the Live+SD daily ratings, L+3 which includes live plus 3 days of viewing and L+7 which is live plus seven days).

On Demand and Online viewing can be tracked by Nielsen but only up to 3 days after the initial telecast (for purposes of the C3 ratings advertisers use) and only if they contain the same national commercials that aired on TV. Some networks are playing around with this but particularly on the online side it looks like for now most networks are shying away from full commercial loads for online viewing. I'm not sure if that's because the networks don't want to put off online viewers by adding (LOTS) more commercials or whether the reality is that despite the measurement capabilities advertisers don't want to pay the same rates for online viewing or a combination of both.


Another thing to keep in mind is that whatever the measurement system, many of the things you (and the networks, for that matter) don't like aren't a function of measurement but rather what the advertisers are willing to pay for. So, for example, a new measurement system wouldn't change that broadcast primetime advertising is sold on the basis of adults 18-49 and subsets of adults 18-49. That's not about measurement, that's about what the folks with the big briefcases full of money want to spend it on.

  • The End

    I had a long reply typed up but speaking about the system in general I can only say.

    Damned if you do and damned if you don’t.

    Whatever solution, even keeping the current system, an equal amount of people will still be unhappy.

    That said I would like to see Nielsen increase their sample size by a higher number over the next few years, the 13,000 homes in 2007, to 20,000 in 2012 should be something like 40,000 by 2017. At least in my opinion.

  • The End

    Meant to say 30,000, bit of a typo there.

  • Mark Wood

    It seems to come down to three groups that I hear about all the time. People who aren’t interested in the financial side of what determines a renewal, just interested in tracking a shows total audience. And of course as Live or live same day unlike just 7 years ago, doesn’t paint you an even close to accurate picture of a shows audience. Ie people that talk said show must be terrible because over the years its gone from X number of viewers and is now down to Y number.

    Then you have two groups that do concern themselves with renewal, the first that bitches and moans that every viewer doesn’t count, why I strongly say if your older then 49 purchase your media. When your 80 or when your 18 your dollar is worth the same exact amount. You’ll never get them to understand, its like talking to a wall.

    Then the next group who think just because we don’t see the various ancillary revenue generated, or the exact breakdown by gender, age, and income and how that changes just the standard 18-49 demo, that the studio and networks aren’t very aware of the total revenue that is generated by each and every show. Not to mention things we never see like License fees, and actual production cost of each show, and how that can make a sizable difference between two shows that get similar ratings.

    And of course you always have people who don’t seem to understand that shows are going to perform at different levels on different networks. Like if next year NCIS was on the CW its going to take a massive hit in the ratings, as most of the demo of NCIS isn’t going to migrate to the CW.

  • Tested

    Very good analysis.

    My view: Nielsen should buy the set-top data and have their ratings boxes in 0.1% of all homes. (115,000 households I believe) The combined information would tell a more complete picture of overall viewership than what we get now.

  • Fugitive at 17

    No Nielsen is the worst and most inaccurate system in the world. The ones in Canada and overseas are way better.

  • brutony

    DVR’s SHOULD be counted-most people have one, or will have one, as well as On-Line and On Demand, since they can be easily and accurately tallied. They still want to use the 1950’s model of calling random households, and doing a survey based on that-REALLY? What is this, the Dark Ages? WTF is going on-modernize, you Luddites!

  • One

    It’s hard not to complain when the shows w/ vocal internet fanbases seem to get the short end of the stick ratings wise, while the shows w/ quieter fanbases seem to be the big hits at least half the time.

  • Fugitive at 17

    Nielsen really needs to move forward with technology and put a chip in all set-top boxes so we can get accurate views like we get on YouTube.

  • IRA SY Valfer

    Regarding the complaint that “the sample is too small.” My wife worked in advertising 30 years ago and heard Mr. Nielsen tell an audience of TV execs and advertisers “If you don’t believe in sampling, tell that to your doctor the next time he wants to draw blood.” It has the same effect now as it did then.

  • Silvio

    TV Ratings Measurement System unhappiness Visual Interpretation

    People that are generally not happy, and because of rational reasons
    :cry: :cry: :cry:

    People that are not happy only when “their” show is cancelled
    :cry: :cry: :cry: :cry: :cry: :cry: :cry: :cry: :cry: :cry:
    :cry: :cry: :cry: :cry: :cry: :cry: :cry: :cry: :cry: :cry:
    :cry: :cry: :cry: :cry: :cry: :cry: :cry: :cry: :cry: :cry:
    :cry: :cry: :cry: :cry: :cry: :cry: :cry: :cry: :cry: :cry:
    :cry: :cry: :cry: :cry: :cry: :cry: :cry: :cry: :cry: :cry:
    :cry: :cry: :cry: :cry: :cry: :cry: :cry: :cry: :cry: :cry:
    :cry: :cry: :cry: :cry: :cry: :cry: :cry: :cry: :cry: :cry:
    :cry: :cry: :cry: :cry: :cry: :cry: :cry: :cry: :cry: :cry:
    :cry: :cry: :cry: :cry: :cry: :cry: :cry: :cry: :cry: :cry:
    :cry: :cry: :cry: :cry: :cry: :cry: :cry:

  • Fugitive at 17

    What Nielsen needs is some competition from a company that will pay for chips to be set-top boxes. I think everybody wins if competition comes.

  • Ultima

    Nielsen really needs to move forward with technology and put a chip in all set-top boxes so we can get accurate views like we get on YouTube.

    Clearly you aren’t capable of reading a short article on this subject…

    No Nielsen is the worst and most inaccurate system in the world. The ones in Canada and overseas are way better.

    … so how can you possibly be informed enough to come to this conclusion?

  • Crazy Eye Broyles (AJ)

    I like that comparison to blood samples, except for one thing. I do not believe for a second Nielsen has a truly “random” sample. They most certainly want to get those boxes into households with money to spend and more than one person in key demographics, because those are the people advertisers want to spend money on. The more houses they can get the boxes in that have children and young adult parents with disposable income the better. When they try to tell you the sample is truly random, they are lying through their teeth.

  • Robert Seidman

    @Fugitive: what you want Nielsen to do and what Nielsen “needs” to do aren’t the same thing. As addressed in the post, your solution grows the sample size but it doesn’t solve a lot of issues like:

    -advertisers want to know what people who don’t have set top boxes watch (Nielsen panel includes those folks)

    -advertisers need granular info – if you install a people meter chip into set top boxes, you could get the granular info but only if folks punched in the appropriate codes. even if you do, you still have to solve the privacy concerns

    -new set top boxes for everyone would cost billions and take years to roll out. Who’s going to pay for that? Not you. Not me. Not Nielsen or its customers either.

  • Crazy Eye Broyles (AJ)

    And for the people making the chips in set top boxes argument, you have no clue how a Nielsen box works. They need to verify that you are sitting there watching and not just leaving the box on a channel while not watching. So you have to hit a button every so often. Cable companies have no financial interest in paying to add this to their boxes as of now. And do you really want your cable company selling your tv watching habits to whomever they feel like?

  • Robert Seidman

    @Crazy Eye: Nielsen estimates the total population and then tries to mirror it in its panel so if 10% (I made that number up, though I could dig it up) of the population according to the universe estimate has household income of $100,000 or more, Nielsen tries to have only 10% of its sample as households with $100K+ annual income. I don’t know how closely they succeed in matching, but my guess is it’s not *wildly* off or you’d here chronic squawking from the advertisers about it.

  • Oliver

    There are two issues here.

    1. Is Nielsen a good way of measuring TV viewership, which is what it purports to measure?

    Does Nielsen have good methodology? Is Nielsen’s sampling representative? How large is its statistical margin of error (something that I have never seen reported)?

    I have my concerns but, ultimately, I suspect Nielsen’s methodology (via boxes and diaries) is good enough and, ultimately, this sort of measurement is always going to have its flaws and limitations. It’s also not hard to imagine the tech improving to make it more accurate over the next few years (for example, it’s not hard to imagine them replacing fiddly remotes that record which viewers are in the room with RFID tags)

    This isn’t hot air or without precedent. There’s a famous story about Arbitron introducing Portable People Meters for radio ratings, which replaced paper diaries with an electronic device which automatically tracked radio listening. This more accurate device proved that people were systemically under-reporting how much they listened to oldies stations compared to paper reports. A dying station format very quickly became relevent again.

    I do find sampling errors and MoE interesting, though. The difference between renewal and cancellation on The CW is 0.2% in 18-49, which is tiny. It’s incredibly likely that small sampling errors and MoE have already led to the cancellation of TV series.

    2. Is Nielsen a good way for advertisers to decide how to spend advertising dollars and, as a consequence, should Nielsen decide the fate of network TV shows?

    Not to the degree it currently does.

    The focus on Nielsen to the advertising industry is, in many ways, an anachronism. People are shifting their viewing habits online, especially young people, and the advertising industry hasn’t really caught up with it yet. I strongly suspect that advertisers are systemically over-paying for TV ads and under-paying for online TV ads.

    The TV business is currently based on selling ads based on US Nielsen ratings, then selling shows into off-network on US local and cable stations, with the price paid and cost of ads also based on Nielsen.

    I expect that to change over the next decade. Online viewing (whether subscription, paid or ad-supported but free) will become much more lucrative and international sales will become much more valuable, and Nielsen will become less directly relevant. The CW and The Firm are very much ahead of their time, even if their ratings are miserable.

    Of course, this series of events should worry the networks a lot, because this gives a lot more power to the studios at their expense.

  • Kyle

    If someone wants their privacy then they can’t complain when their favorite shows are canceled. Until someone actually asks me what I watch, I will continue to complain about not being represented! :)

    I know it’s unreasonable/impractical, but I would like to see Nielsen do a week where they allow everyone to submit their info and compare it to their .02% measurement to see just how accurate it is. I know everyone won’t respond, but then again, if you’re too lazy to respond, then you can’t complain.

  • Ultima

    I would like to see Nielsen do a week where they allow everyone to submit their info and compare it to their .02% measurement to see just how accurate it is.

    I can tell you right now – the information everyone sends in will not be anywhere remotely close to an accurate representation of the whole.

  • Ultima

    . I know everyone won’t respond, but then again, if you’re too lazy to respond, then you can’t complain.

    But Nielsen’s customers, the advertisers, would complain, and with good reason.

© 2015 Tribune Digital Ventures