Measuring Internet Video: Virtual Panel Discussion

Categories: Internet TV,New TV Technology,TV Industry Panels

Written By

September 18th, 2007

At the recent TechCrunch40 conference in San Francisco, angel investor Ron Conway said that he thought monetization of Internet video was the single biggest growth opportunity on a percentage basis. 

While TVbytheNumbers will mostly focus on the numbers that drive the business of television, we believe that when it comes to today's television content over time some of the lines are getting blurred. People are accessing their favorite content in new and different ways. 

Further, with budgets for online video advertising expected to reach as high as $4.3 billion annually by 2011 according to some estimates, we expect the space to heat up rapidly and we'll be tracking it. 

As veterans of some of the issues that have plagued Web measurement, it seemed like a great time to check in with some of the measuring companies and thought leaders regarding Internet video metrics. 


1.  What are the biggest challenges of measuring internet video?

Ted Leonsis: Measurement of any new medium is difficult, but it is especially so in a hyper-growth market like internet video.  Internet video has a variety of challenges when it comes to measurement, the biggest of which is measuring the relative value of one video over another.

Dave Thomas: Whether you're measuring video online, on traditional television sets or on an expanding array of other delivery platforms, the ultimate objective is to be able to identify who actually is watching the content, because advertisers consider demographic measurement of audiences critical to effectively targeting their creative messages. 

Konrad Feldman: From a technology point of view, there are a number of challenges. Primarily, as with the web overall, a new methodology is required that combines panel-based data with directly measured data.

Gian Fulgoni: There is a lack of technical standards and a lack of advertising standards as well.  A variety of different technologies are being used by publishers - Flash, Windows Media and Quicktime. Many have made their own versions of the Flash Video Player.  The IAB has standards for display ads, but nothing in place for video.

Mark Cuban:  Defining unique users and getting streamers to release information to 3rd parties. It's not hard to do, but they realize that if they get fully audited a lot of published numbers are going to decline.

Ted Leonsis: There is such a wide range of video on the Internet that aggregated volume metrics don't necessarily reflect value to the consumer and the advertiser.

Dave Thomas: From a single panel perspective (where we measure both Internet and television in the same home) a major hurdle is simply getting people to allow us to measure their Internet usage.  From a privacy standpoint, people see PC's as dramatically different from TV's, and, at this point, they are much less willing to allow us to measure their online activities.

Konrad Feldman: Videos can become extremely popular overnight (for example Funny or Die's ‘The Landlord') and then flame out within a few days.  While each video may represent a terrific advertising platform, real time speed and resolution are required if advertisers and publishers are to capitalize on such transient popularity.

Gian Fulgoni: Some form of standardized coding / tagging system that helps in the identification of specific content and ads across sites would be a huge step forward. But, is it realistic to expect that this will happen? That said, I do want to stress that comScore is already providing measurement of the audience for streaming video today.

2.  The only thing I've seen actively measured currently is the number of times a video is accessed or downloaded. Do you think this is a valuable metric, especially for advertisers? 

Konrad Feldman:  Sure it's useful, but not in isolation. Metrics of access/download alone can be quite limited. Just because a video is downloaded does not mean it is watched, and items such as YouTube videos are frequently embedded in web pages, resulting in an impression without getting played.

Ted Leonsis: This is a valuable metric, but one that needs to be understood in context. 

Mark Cuban: Number of downloads by itself is a worthless measurement.

Dave Thomas:  The number of times a piece of content (or a website or banner) is requested can provide relevant context for the sake of comparison.  But accessing, or even downloading, video is not the same as watching it.  We know that from other experiences.  Thus, the metrics of greater value are those that provide real insights into actual consumer profiles and behavior. 

Gian Fulgoni: Yes, it's useful but the number of Streams and number of Streamers are also valuable metrics that are supplied by comScore today because they provide a good proxy in video to Page Views and Unique Visitors in traditional web content.  Additionally, we use the two second rule to validate that users are engaged (i.e., we don't count the stream if duration is less than 2 seconds).

Dave Thomas: Demographic information as well as how much (from a duration perspective) and how many times content is actually consumed - will become more and more important.  But we also recognize the fact that we may have to change or even create new metrics as new ad formats and executions develop.

Ted Leonsis: When comparing volume metrics with internet video, there is danger in assuming that all video ‘streams' are created equally.  They are not, so it is important to look at both volume metrics as well as engagement -- time spent viewing. Engagement is the key metric.

3.   Are we in for another round of "hits vs. page views" when it comes to measuring Internet video?

Ted Leonsis: Differing technologies will ultimately introduce different measurement solutions and opinions.  It's important that we embrace standards for how and when measurement occurs (client side vs. server side), independent of the measures themselves.  Hopefully we have learned from our experiences in the traditional banner display world and won't delay in defining those standards. 

Mark Cuban: We are already there. Views are worthless information.

Gian Fulgoni: There's no real equivalent in Video, as the definition of a Stream is perhaps more concrete than a Page View (i.e., there had been long period of time without a formal definition of a Page View).

Dave Thomas: The metrics and debates surrounding Internet measurement (or any audience measurement for that matter) will continue to evolve as new delivery and consumption technologies are introduced.  It's inevitable that we will have to develop newer solutions to provide relevant, actionable data to our clients.  The recent transition by NetRatings to "time spent" from "page views" is one example. 

Konrad Feldman: Probably, but that's just part of different constituents determining the metrics that are most useful to them. Note that for the Web, hits vs. page views has given way to reach vs. time on site, so metrics continue to evolve.

Gian Fulgoni: The debate will more likely be between streams and duration, as the market debates which is the more relevant way to measure ad delivery and inventory.

Dave Thomas: The progression from accesses, to who is accessing, to who is accessing and how much they are accessing is logical.

4.  What do you think is the most valuable metric when it comes to internet video?  What year (if ever) do you think we'll be able to start measuring it?

Ted Leonsis: In my opinion, the most valuable metric is engagement.  Right now we can't get an accurate capture of time spent viewing, but by 2008 "true" time spent viewing metrics should be available.

Dave Thomas: An essential metric is persons viewing - comprehensive information about who's actually watching the video.  Conceptually, the capacity to deliver person's viewing to Internet video represents a major stride, and one we're taking with the NetRatings' VideoCensus service. 

Gian Fulgoni: A combination of time, streams and streamers is the most appropriate way to measure the market today, but as measurement gets more advanced, both publishers and advertisers will want to see more metrics around how much of the stream was viewed, for both ads and content (e.g., 25%, 50%, 75%, 100% viewed after started).

Konrad Feldman: User-engagement - the playing of videos, amount of time watching, progress reports, interactivity, etc. - is vital to understanding video consumption and audience characteristics. At Quantcast, we have recently started supporting all of these measures for video content producers, free of charge.

Mark Cuban: Time spent viewing and simultaneous viewers. It's easy to capture now (servers can capture everything), it's just never published. This is more a function of what information the content producers want to release than a measurement issue.

Konrad Feldman: Our free Quantified Publisher program supports measuring and reporting of everything other than the simultaneous viewers Mark would like to see.

Gian Fulgoni: Fundamentally, the most important metrics are the reach and frequency of ad exposure and the response to that exposure - measured across time and channels (i.e. offline and online response).

Dave Thomas: We believe the ability to connect and analyze video content across delivery platforms (which is the basis for our A2M2 initiative) is the foundation for the most valuable metrics.  We will begin building a test panel next month to support that vision, and expect initial data in the Spring. (A2M2=Anytime Anywhere Media Measurement)

5.  Will things ultimately evolve to the point where actual viewership can be tracked via software and not require panels?

Mark Cuban: Can be tracked via software and not require panels? Servers already capture all the data, though they don't necessarily capture the information about the user viewing the streams.

Gian Fulgoni: I can't see this happening. The viewer is fundamentally in control and the delivering site can't possibly know the characteristics of who is on the computer without insisting on some kind of log-in cookie and that would generate huge negative consumer feedback and resistance.

Konrad Feldman: Yes, at Quantcast we are already working with thousands of publishers large and small to enable direct measurement. We've just recently started  directly measuring viewership of online videos and widgets. We believe in convergent models of measurement whereby panel and directly measured data are combined to provide increased accuracy and detail.

Gian Fulgoni: Panels provide a complete and consistent view of the marketplace:

  • All sites measured using a consistent method
  • Visibility into audience demographics
  • No issues with over-counting unique viewers because of cookie deletion, multiple machine/browser usage
  • Measurement of consumer response to ad exposure and the ROI from advertising spending - both of which will be demanded by advertisers

Dave Thomas: We always are hopeful that advances in technology will present us with less intrusive opportunities to interact with consumers to gain insights into their viewing.  Furthermore, creativity in how we can build hybrid panel/census solutions (similar to VideoCensus) is going to be critical to meeting the industry needs in a cost-effective manner.   But panels will continue to supply the demographic composition information, whether it is reported directly or applied to census data.

6.  Large numbers of simultaneous viewers seem to be what advertisers really get excited about when it comes to traditional television.  Does simultaneous viewing play a role when measuring Internet video?

Konrad Feldman: For live streamed events maybe, but the on-demand nature of Internet video will likely result in other metrics talking precedence. Simultaneous viewership is changing rapidly in the face of DVR and on demand distribution, even for traditional TV.

Mark Cuban: It serves the same value as it does for TV. It also is the best gauge for understanding bandwidth costs

Ted Leonsis: Advertisers and content providers care about reach and engagement, not simultaneous views. Plus, in an on-demand world, there's more audience potential. Our audiences for the 2005 Live 8 concerts were 10 times bigger in the days after the event than the day of the event itself.

Gian Fulgoni: Today, online video is fundamentally an on-demand personal experience and simultaneous viewer metrics don't apply. However, they could become important in the future if online-delivered content becomes something that is viewed on a device -- beyond a personal computer -- that makes it easy and convenient for simultaneous viewing by multiple users.

Dave Thomas: We expect the definition of simultaneous viewing to evolve as platforms advance and change.  Traditionally defined simultaneous viewing (such as a parents and children watching television simultaneously) will certainly remain an opportunity for advertisers.  But we think another interesting trend involves individual viewers consuming both television and the Internet simultaneously.  Watching traditional television while also online enables viewers to immediately drill down into anything that inspires them.

Konrad Feldman: Reach and engagement are going to be more important for Internet video. When a new video suddenly gets popular, it becomes a very valuable commodity. The fact that many users are watching it - and watching it all the way through, even repeatedly - is important.

7.  Is "number of times replayed" a valuable metric?  Some short-form Internet videos have much higher replay than say, a 42 minute episode of LOST.  I know I have watched the "Will it Blend?" short for the iPhone at least 5 times.

Gian Fulgoni: Measurement of multiple exposures to an ad is an important metric in the planning and analysis of TV advertising campaigns, and while our experience with the online video market is that this has not become an important metric as yet, that is likely to change as advertising in online video increases. 

Ted Leonsis: Any data is valuable, especially to content providers used to the limited data they currently receive on TV usage. However, number of times replayed is valuable only on a relative basis. You can't compare that metric on a full episode of CSI to that same metric on a 2 minute user generated clip or news story.

Mark Cuban: If you have good data including duration it's a valuable piece of data. Without it, it could be the result of a buffering problem.

Dave Thomas: In and of itself, frequency is not terribly important.  But when you combine it with overall reach, you then have some relevant gauges.

Konrad Feldman: Yes definitely, but it's going to be interesting to see how the advertising opportunity for watching short videos many times compares to watching longer segments. Of course, there's a lot of interest today in short videos online, and we're going to see much more innovation in non-obtrusive and short form video ads. When we find the right model to integrate advertising with short form video --  in a way that doesn't detract from the viewing experience-- then multiple plays = multiple ad impression opportunities.

8.  While many of the short videos have no advertising (or are themselves some type of advertising) regular television programming that's available for streaming over the Internet is starting to incorporate advertising/sponsorships. ABC for example has a few 30 second breaks during its streams).  How important is it to measure viewership of these spots?

Dave Thomas: That's what it's all about.  Accurately measuring audience and behavior related to these spots is critical to business development, and consistent with our traditional television business.

Gian Fulgoni: For long form, where pre-roll and mid-roll advertising might make the most sense, it is related to the discussion above about the importance of streamers, streams and time - this may be only one stream from a technical perspective, but a long form show represents multiple ad exposure opportunities.

Ted Leonsis: Measuring viewership of ad spots is critical, for three very specific reasons.  First, we have the opportunity (unlike TV) to measure the true viewership of ads and their associated demographics.  Second, with viewership metrics we can better understand the relative value of these placements.  Lastly viewership can help to truly understand "engagement" and how user behaviors are evolving.

Konrad Feldman: Just as with the recent round of TV upfronts, the currency of Internet video advertising will be based on measures of exposure to advertising. Properly measuring video advertising enables us to understand both who may be exposed to these segments as well as who actually lets the player run through them. For interactive ads like those distributed by the VideoEgg Egg Network, engagement is important too: e.g., do users click on the ads, and at what point?

Gian Fulgoni: This form of content (professionally-produced content by well-known publishers) is what's most attractive for leading brands - they are reluctant to advertise on much of the UGC because of fear of being associated with what could be questionable content.

9.   If I download an episode of LOST from iTunes and then watch 15 minutes of it via iTunes or Apple TV, when I sync my iPhone and choose that episode, it picks up wherever I left off (the reverse is true as well).  All the information is "tracked" (at least via iTunes/iPod/iPhone).  How important is this sort of information when it comes to measurement?

Ted Leonsis: In the aggregate, information on where people are watching video is very important. We need to be able to follow trends around 10 foot (PC-enabled TV), 2 foot (PC) and 1 foot (mobile) video consumption to make sure we're bringing users what they want. This is a metric that will help drive product innovation, which will in turn drive usage. Less important is whether or not a single episode was watched in every different environment.

Gian Fulgoni: As it relates to advertising, advertisers should want to know how far through their ads the consumer got before closing, and if they engaged with it. That said, it's rather interesting to note that ratings of TV commercials weren't available until this year! Also interesting to ask whether this was because the TV networks didn't want them, advertisers didn't ask for them or if Nielsen was unwilling or unable to provide them!

Mark Cuban: it can only add value, but it's not critical right now because there aren't enough people doing this relative to the universe of people who watch TV or even to those who stream video over the Internet.  Whether it becomes critical over time, we'll see.

Konrad Feldman: It would be very valuable to track this kind of integrated usage. We believe there is great value in integrating content measurement into all players, and that content owners should be able to decide what measurement services to use for interpreting the distribution of their content.

Dave Thomas: While the migration from analog to digital poses a variety of challenges, digital technologies also afford us a host of opportunities.  Dealing with time-shifting is a terrific example of that dual phenomenon.  The ability to aggregate viewing occurring at different points in time is becoming ever-more critical. That is clearly evident in the use of "live-plus-3" as the preferred measurement for this year's national television upfronts.


The panelists from the measurement companies provided us with a lot of information.  For readability the responses have been edited, but you can read the full responses here:

comScore's Gian Fulgoni, Nielsen's Dave Thomas and Quantcast's Konrad Feldman 

A special thanks to all the panelists for taking the time to participate!

© 2015 Tribune Digital Ventures