Nielsen Admits Big Problems With How Its Technology Measures Web
NEW YORK (AdAge.com) -- Last week Nielsen Co. made a stunning disclosure, announcing that a computer glitch triggered by long web addresses used by social networks was tripping up the system and causing the company to underestimate the time people spend on the web by an average of 22%.
The admission, first reported by Ad Age, confirmed some of what the online publishing industry had long suspected: that they were being sold short in a key tool used by agencies to plan media buys.
And while Nielsen said the glitches impact data going back three months, clients say they've been talking to Nielsen about apparent discrepancies in the data for nearly a year, though the company at the time couldn't isolate the cause.
As executives at Nielsen investigated that problem, they found others. The company now faces two stiff challenges: fixing the immediate problems with a system that hasn't kept up with the web and rebuilding its image as a provider of trustworthy data as marketers and media are asking for even more complicated metrics that cross TV, mobile devices and the web.
"What it does is it erodes confidence in one of the primary tools for planning," said Sherrill Mane, senior VP of products for the Interactive Advertising Bureau. "Those who are making plans and allocating dollars were using a fundamentally reduced number of users of audience to do their plans. That does ultimately translate to lost money for publishers and websites."
Initially, Nielsen thought the problem was isolated to long URLs -- 2,000 characters or more -- that are becoming increasingly common on social networks where web addresses are used during sessions to pass data back and forth between applications and users. If a Nielsen panelist encountered such a long URL by visiting a social network, the entire internet session was thrown out in some cases, leading to a broad underestimation of time spent on websites.
Additionally, Nielsen found that its meters were not counting users of Google's Chrome browser because of the long URLs it generates for secure purchases on the web.
But the even bigger problem is with the meters themselves. Nielsen depends on a panel of more than 200,000 people who voluntarily install a small bit of software, called a "PC meter," on their computers, which sends activity back to Nielsen. But those meters can crash without the knowledge of the user and stop recording data until the user restarts his computer, an increasingly infrequent occurrence.
"We believe social networks are a significant contributor to this, but not the only contributor," said Steve Hasker, Nielsen president of media products. "The silver lining for us is our team has been working around the clock and weekends to identify these problems."
Nielsen has been steadfast that it believes the problems began three months ago. But some dispute that. Lyle Schwartz, managing partner for research at WPP's Group M, said the company questioned Nielsen's internet-usage numbers in the first quarter when it released its "Three Screen Report," which reflected lower-than-expected internet and TV viewing numbers -- Nielsen said time spent on the internet fell to 25 hours and 26 minutes a month from 26 hours and 32 minutes the prior year.
"We saw that internet usage was going down, so we questioned it," Mr. Schwartz said. "If the data is telling you something you don't expect, it's something to pay attention to. We were told it was accurate."
Another person said some publishers came to Nielsen last December to question the numbers and were told the company was investigating discrepancies.
Others said they don't believe the problems start and end with the "time spent" metric. TVGuide.com saw a big drop in unique users across the entertainment category in September, which it brought to Nielsen's attention in October. "We've had an ongoing dialogue with Nielsen for over a year-and-a-half about the quality of these metrics," said General Manager Christy Tanner. "We believe strongly that our unique users are under-reported by Nielsen."
Nielsen told clients it will update them every two weeks on its progress as it roots out problems and implements fixes. Nielsen Online has hired a new team and is committed to spending "tens of millions of dollars" over the next 12 months to overhaul and modernize the system, Mr. Hasker said. "None of our competitors will do that."
Critics of Nielsen say it should be more transparent in the way it collects and analyzes data in the first place and seek the approval of an independent organization such as the Media Research Council (MRC) to audit their data.
Neither Nielsen nor its main online competitor, comScore, are "accredited" by the MRC. Mr. Hasker said Nieslen is working with the MRC to work through the problems and ultimately receive their imprimatur.