Dear TV Fans: There Is No TV Ratings Measurement System That Will Make Everyone Happy

Categories: 1-Featured,Broadcast TV,Cable TV,Help

Written By

July 8th, 2012

Whenever there are Twitter Q&A's from  Syfy's Craig Engler (@Syfy) or USA's Ted Linhart (@TedOnTV) I just can't look away. Having seen many of the Twitter Q&A sessions it's well into repeat questions and different variants of the same questions. I still can't look away! There are always lots of questions about Nielsen and measurement. Many fans have a quarrel with Nielsen, but I've come to the conclusion that most of them would have a quarrel with TV ratings measurement no matter what the system of measurement was.

As long as there are fans of TV shows, there will be disappointed fans when shows get canceled. No measurement system can change that and whatever the measurement system, low-rated shows will typically be canceled.

There are several types of complaints/improvement suggestions that come up, here are a few of the most frequent:

We have the technology! We could measure everybody! Why don't they measure everybody!

The technological capabilities to achieve that probably exist but that technology doesn't actually exist yet. My bet is it never exists, even if the technology vastly improves. There are a lot of reasons why I think that but the top two are:

  1. lots of people don't want their viewing measured (they view it as an invasion of privacy)
  2. $$$$$ - It would cost too much money

There are other reasons besides those, but both of those are show stoppers. Would a complete census be more accurate than Nielsen? If you could get it, it would, without a doubt, be more accurate. But TV ratings measurement exists for the purpose of buying/selling TV advertising. The networks and advertisers aren't going to be willing to pay for it and as expensive as Nielsen is (and it's very expensive) the census style system would be multiple orders of magnitude more expensive to maintain and manage. The networks and advertisers aren't going to pay for something like that for a system that might only be a little more accurate.

On top of that you'd still probably need Nielsen or something like it because the census system would have so much data to crunch it wouldn't likely be able to produce fast national ratings the next morning and final ratings the next afternoon. The networks need the information fast so they can react and make scheduling decisions.

But what about data from all those set top boxes - that's a lot bigger sample than Nielsen!

It is indeed a bigger sample than Nielsen. But in at least a couple of ways it's not what advertisers (at least buyers of national advertising) want.

  1. it only measures people with set top boxes. Advertisers want to measure everyone, even the people still using rabbit ears
  2. it measures household viewing but doesn't tell you *who* in the household watched. Advertisers (at least buyers of national advertising) want to know who watched, what their gender is, what their age is, etc. set top box data isn't that granular.

the good news is the data is available and is being used.  Usually it is in addition to Nielsen and not a replacement though recently there have been reports that a few local stations are switching to using set top box data to sell local advertising spots. My understanding is that those numbers aren't churned out as rapidly as Nielsen reports ratings either.

Neftlix, On Demand, DVR, iTunes and Online Viewing Aren't Counted and they should be!

This is a fairly common one and not just with fans of TV shows. I've seen producers of shows with similar laments. First, let's talk about the "not counted" piece. DVD, Netflix, and iTunes certainly aren't counted in the Nielsen ratings. That's because the purpose of the Nielsen ratings is to measure in order to broker the sale of TV advertising, not to come up with some type of "total popularity" metric. Since Netflix, iTunes and DVDs don't have advertising, they're not counted in the Nielsen ratings. But that's not the same thing as "they're not counted." Of course sales of DVDs and iTunes episodes are counted and the people who need to know, know!  You can pretty much count on any deal where how much Netflix pays to license content depends on how many people watched on Netflix, it gets counted.  There are several reasons why months and months and months after the fact nobody reports a total popularity metric - but you can start with months and months and months after the fact it's not in anyone's interest to pay money to figure all that out. The people who need the information to make decisions have the information.

DVR viewing *is* measured by Nielsen.  Currently advertisers pay for commercials watched up to 3 days after the program initially aired on television (these ratings are commonly called C3 or C+3 ratings).  That was the compromise reached between the networks and the advertisers though the networks are reportedly still pushing for 7 days instead of 3.  In any event Nielsen measures commercial viewing live and up to three days after. On the program ratings side Nielsen measures DVR viewing up to 7 days after and can slice that in a variety of ways (the common slices are the same day DVR viewing included in the Live+SD daily ratings, L+3 which includes live plus 3 days of viewing and L+7 which is live plus seven days).

On Demand and Online viewing can be tracked by Nielsen but only up to 3 days after the initial telecast (for purposes of the C3 ratings advertisers use) and only if they contain the same national commercials that aired on TV. Some networks are playing around with this but particularly on the online side it looks like for now most networks are shying away from full commercial loads for online viewing. I'm not sure if that's because the networks don't want to put off online viewers by adding (LOTS) more commercials or whether the reality is that despite the measurement capabilities advertisers don't want to pay the same rates for online viewing or a combination of both.

-

Another thing to keep in mind is that whatever the measurement system, many of the things you (and the networks, for that matter) don't like aren't a function of measurement but rather what the advertisers are willing to pay for. So, for example, a new measurement system wouldn't change that broadcast primetime advertising is sold on the basis of adults 18-49 and subsets of adults 18-49. That's not about measurement, that's about what the folks with the big briefcases full of money want to spend it on.

 
  • Holly

    @Sally,

    Yes, participation is an issue, but not one exclusive to Nielsen. Another similar issue is that the participant know they are being monitored, which can change behavior. Both are simply realities of most social research. Short of secretly monitoring people without their permission (huge privacy issues ), there’s not much they can do to avoid that.

  • DGH

    I understand that low rated shows will be cancelled no matter what but it shouldn’t be to hard to get a more accurate count of actual viewers. Its hard for me that we can predict what tens of millions of people are watching based on 10,000 households.

  • Holly

    @DGH,

    If it wouldn’t be hard, what, exactly, would it involve? And how much more accurate would it be?

  • Carmen

    It’s good to note in reviewing the comments that some have touched upon one of the factors (almost always) ignored by the “Neilsen is flawed” crowd.

    That is,
    “If the Neilsen ratings are as severely flawed as some fans claim, then those fans have to also accept that the ratings of their favorite show can be skewed high as easily as low”

    In short, Neilsen may be overstating the ratings for their show.

    But realistically, with the sample size in place – over a season average – the (Neilsen) ratings should be within 5-15% of the actual occurrence.

    As stated by many earlier, This accuracy can be improved, but it would take literally billions to get – what? maybe an additional 5-10%.

    Sorry, it just wouldn’t be worth it.

  • RatingBook.com

    While this site focuses only on national ratings (using Nielsen’s peoplemeter technology developed in the early 80’s), more than 2/3rds of US markets still use paper diaries for all Nielsen audience measurement. Since Arbitron’s ScanAmerica expenses doomed the company’s TV measurement in 1993, Nielsen has had the small market to themselves. Rentrak is about to change all that, using set-top boxes for HH tuning and using “fuzzy logic” for demographics.

    If local television stations cannot prove to local advertisers their local newscasts are being watched by potential customers, local news will die.

    And the worst off? NBC stations in smaller markets. Visit http://www.nothingbutchange.com to find out more.

  • Ray32

    @Holly,

    you’re right, it might be interesting. On the other hand, the difference in method alone (people meter vs. diaries) will lead to somewhat different results. And that is probably going to mask any gains in accuracy from the bigger sample.

  • Holly

    @Ray32,

    True. Though on the topic of different results from different methods…. Is there any accessible data on how results change when people meters are introduced? I mean as far as what shows are being watched, not data on channel flipping and ffing through commercials?

  • Holly

    @RatingBook,

    The obvious self-promoting aside, I think smaller local markets would probably be the best “in” for a newer company. While Nielsen is starting with people meters in the larger markets and working their way to smaller, there’s an opening for someone to start in the smaller markets and work their way to the larger ones as they build infrastructure.

  • Martin

    What about HBO ratings ? (and other PayTV or commercial-free networks)

    Since Showtime and HBO don’t have advertisers and they finance production of shows with subscription revenues, the “same day” ratings only matter to steal away viewers from conventional and specialty channels. Subscribers can PVR HBO shows and watch them later, or they can watch the marathon before the finale, or they can watch the reruns scheduled after the finale, as long as they get something worth their money (Let’s skip the part where cable company makes a 100% profit for these services).

  • The Real Original Ray

    This is a great post and discussion. Thank you.

  • http://tvbythenumbers.com Robert Seidman

    @Martin: not sure about all premium cable channels, but HBO, Starz and Showtime all subscribe to the Nielsen ratings. HBO certainly looks at the long view (encore telecasts, DVR, on demand, HBO Go) but the problem with the long view is it takes a long time to accumulate all the data. And although most viewing of HBO’s scripted shows is not represented in the Live+SD numbers for the first telecast of an episode, HBO can still do a pretty job of figuring out what the long view will be from those numbers without having to wait a long time. You never see stories like “despite not having nearly as many viewers in its premiere telecasts, Boardwalk Empire winds up with as many viewers as True Blood when you factor in DVR, HBO Go, etc.” The point is for HBO or USA the live+SD numbers which are available almost immediately are very telling. Not perfectly telling, but still a very good indicator that is available almost immediately.

    Basic cable channels like USA look at DVR, encore telecasts, etc. too, but the relative performance of premiere telecasts is still very telling. I’d argue (though Ted Linhart might argue differently) that those Live+SD numbers are still an excellent proxy of relative performance i.e., how Covert Affairs compares to Suits and how Game of Thrones compares to Boardwalk Empire on a relative basis.

  • Cath

    Have you ever filled out one of those diaries? I have. There isn’t enough room to record what you watch and the biggest problem is that if you have multiple televisions it is very difficult to record viewing habits with limited space.

    They are also a pain to fill out.

  • The End

    @Robin Burks

    Its more an issue with the time of the year and appealing to your audience with the Syfy channel. There’s no problem when you air something that appeals to the audience of your channel.

    Naming Stargate SG1/Atlantis, Eureka, Warehouse 13, Battlestar Galactica and other hit shows that were popular with Nielsen viewers.

    This tech stuff is a bit of an excuse in my opinion, when you compare the sales figures of Stargate Universe to Stargate Atlantis and SG1 you’l notice there’s a huge difference. Both shows sold vastly more DVDs/Blu Rays than SGU, and both rated very highly on the network, SGU however rated poorly and the sales reflected that.

  • Fake Me Out

    The monitoring that cable/sat companies have built into their boxes has to do with the fact they have the capacity to offer 500+/- channels and there are 70 bazillion channels available … they need to know which channels aren’t pulling their weight so they can be dropped to make way for other channels that may attract more subscribers.

    The bottom line (which is almighty) is that any costs to adjust, increase, add-on or build new data gathering in order to have “better data” will not be picked up by the networks, nor the ad agencies, nor the advertisers … it will be picked up by you and me. The additional costs would all end up being built into the price of the products we buy … whether it’s a nickel on that Big Mac or $100 on that new minivan we all would feel it in the wallet.

    YMMV

  • Nadine

    I actually agree with everything you said, Robert. I’m pretty surprised that the reaction has still included so much “I wuz robbed” whining. I would add — just how accurate does the measurement have to be? Statistics are very sophisticated. Suppose there’s a 3% (or even a 5%) error rate (which of course, is pre-supposed.) Would even HALF of that error rate watching the show have saved Firefly? Of course not. It’s good enough.

  • Nadine

    @Ultima
    Could you imagine screams of despair if Nielsen increased their sample size tenfold and the ratings for shows like American Idol increased while those for a handful of low-rated “smart” shows decreased?
    You’re very clever, Ultima, and I’m pretty sure you’re absolutely right. Statistics is also a predictor of tendencies and it measures trends as they are going up or as they are going down. So it makes a lot of sense that AI would get even higher ratings. Well said.

  • The End

    @Nadine

    To be fair, if Nielsen increased their sample size ten fold, they’d adjust the value that each box is worth and there likely won’t be that much difference, though Ultima is right if this occured it’s entirely likely lower rated shows would get lower with each box being worth less viewers due to a higher number of boxes around.

  • Fleur

    I was a Nielsen viewer a long while back, as was a friend of mine who didn’t own a tv. She asked them if they were really okay with her as a sample and they said absolutely; they needed even people who don’t own a television (nor did she have any other way to watch tv). I, on the other hand, wrote in (the original)”Dr. Who” every single night hoping the reruns on PBS wouldn’t get canceled. About 2 weeks after I mailed in my diary, my local PBS canceled Dr. Who. Needless to say I didn’t think much of my Nielsen experience. lol.

  • Ray32

    @Holly,

    I’m not 100 % sure what the effect would be. Every instance I’ve ever heard about that the methode was switched from diaries to meters, media usage time went up overall. People just don’t like to admit even to themselves how much time they spend on media consumption. By the same token, I’d expect worse ratings for PBS, better ratings for reality TV… but I have no data to back that up.

  • Holly

    @Ray32,

    I’d expect worse ratings for PBS, better ratings for reality TV… but I have no data to back that up.

    I agree. I suspect that if we were able to avoid reporting bias and changes in behavior because the participants know they are being recorded we would see lower numbers for the “too-intelligent-for-the-masses” type shows and higher numbers for the shows considered low-brow like reality. People are more likely to try to make themselves look smarter and cooler than like they’re just going along with the crowd.

    I know there is the oft-repeated story about participants in the radio surveys under-reporting how much they listened to oldies stations. I just wondered if there were any similar stories for TV viewing.

© 2014 Tribune Digital Ventures