Nielsen's TV measurement first faced a major crisis of confidence more than 20 years ago, amid growing skepticism from networks and surprising ratings fluctuations. "When those numbers became unreliable, inconsistent, inaccurate, unexplainable," a TV researcher said in 1992, "they just threw into chaos almost everything we did, because they were the basis on which the business was built."

There were other challenges before and since, including the controversial introduction of "people meters" in 1987 and audience fragmentation stemming from the cable boom and the arrival of the VCR. Challengers arose, like AGB, a company that pioneered people meters in the U.S, as well as broadcasters themselves, which invested in an alternative form of measurement in the mid-90s.

Remarkably, the company survived all of those tribulations and remains the primary measure by which TV advertising is bought and sold. But this status, which it has held since the 1950s, is once again in jeopardy, and this time its reigning dominance isn't a sure thing. For the methodical research company, the accelerating pace of change in media may finally be getting out of hand.

While the decline in live TV viewership has worried networks and advertisers for decades, that shift away from Nielsen's core competency is increasingly pronounced: As Nielsen reported just last week, Americans watched 12 fewer minutes of live TV per day in the third quarter than they did a year earlier.

And by the time Nielsen develops, tests and deploys ways to measure viewing on an emerging platforms, consumers are now likely to have moved on yet again. Nielsen today is hyper-aware of the need to change -- and the inherent dilemma. Clients griped about its delay in measuring mobile, but if it had moved strongly into mobile two years ago, said Nielsen's exec VP of global digital products Megan Clarken, "We may have been measuring Blackberry today."

"I call it the burden of responsibility," Ms. Clarken added. "We are responsible for $100 billion worth of advertising, so we need to be really careful with what we do."

That may be true, but what does that say about Nielsen's ability to keep up?

Over the past 18 months Nielsen has been frantically hiring experts in digital, product architecture, programmatic and planning. Under Ms. Clarken, the product team is restructured every year to align with the organization's focus areas for the following year. And at any one time, Nielsen has dozens of programs going on behind the scenes, from hardware prototyping to developing open data delivery frameworks.

According to Ms. Clarken, the primary goal is to measure content and advertising no matter when it's watched or on which devices. She said the company will be delivering true cross-platform measurement by the end of 2015, which to her means closing the gaps on connected TVs, Rokus and platforms like Netflix.

Clients are meanwhile turning to competitors like Rentrak to shake Nielsen and potentially dismember its monopoly. They aren't taking Nielsen's promises for cross-platform measurement very seriously. Why should they?

Nielsen first announced plans to measure viewing over the internet, cell phones and TV sets outside of the home in 2006. But PC measurement wasn't deployed until around 2009, mobile TV measurement only became available in September and Nielsen still isn't measuring out-of-home viewing.

Ms. Clarken said she can only account for Nielsen's current commitments and that the company has delivered on its promises from the past year.

"Nielsen is wedded to a legacy approach," said Lyle Schwartz, managing partner-director of research and marketplace analysis at the ad-buying giant GroupM. "Nothing they have showed me gives me confidence in what they can do in the future."

Nielsen executives said the company could better communicate its strategy. "We have done a lousy job of helping the industry at-large understand what can be measured," said Steve Hasker, president-global product leadership, Nielsen.

Gaping holes
Clarifying what Nielsen can and cannot measure is certainly laborious. Depending on who you ask at the company, the explanation can even be misleading.

"There's issues with clarity -- three different groups tell me different things and each doesn't know what the other can do," Mr. Schwartz said.

It's true that Nielsen can track many devices in some capacity, but caveats are the rule.

PCs, for example, are measured using a representative panel, which is criticized as being too small to accurately gauge fragmented audiences. Nielsen is working to implement direct measurement of viewing on computers, but there's no timeline for when that will be complete.

And while Nielsen said in September it is now capable of measuring mobile devices like smartphones and tablets, industry executives said that's not exactly accurate. The software that must be implemented is filled with bugs, according to these executives, and tests of the system have been postponed several times. For some networks, implementing the software has also required hiring additional manpower.

One cable TV executive predicted there won't be a true picture of mobile ratings until summer or fall of 2015.

Connected TVs and over-the-top devices like Roku and Apple TV, are also largely missing from the picture.

Nielsen made headlines last month for plans to begin measuring streaming-video-on-demand services like Netflix and Amazon Prime. But it won't report broad viewership data from those platforms any time soon.

It is not measuring Netflix originals like "House of Cards" or "Orange in the New Black," Ms. Clarken said.

What Nielsen will do is provide participating clients data on their own shows' peformance on streaming platforms.The company's primary focus for 2015 is figuring out how to present a complete picture of the viewing that takes place on these devices and platforms across all programming.

There's also still no way to measure viewing that occurs outside of the home at places like bars, airports or gyms. Nielsen ran a test of out-of-home viewing in Chicago earlier in the year, but Ms. Clarken doesn't have a time frame on when that might be measured on a larger, national scale.

Still, she stands by the promise that most of these holes will be filled by the end of next year.

So little time
That's not fast enough for impatient TV networks, who according to analyst estimates pay $100 million annually for Nielsen's service. (Ad agencies pay a fraction of that.) The networks want Nielsen to modernize its core measurement system, and lament the amount of resources required to maintain its audience samples.

"If you were to start over you wouldn't be spending so much money on maintenance of the panel to keep it up and running, you'd be trying to build for the future," said David Poltrack, chief research officer, CBS.

"The people meter was introduced in the 1970s when people sat around one TV set in the living room," said Alan Wurtzel, president of research and media development, NBC Universal. "How relevant is that now?"

A glitch in Nielsen's TV measurement, revealed in October but dating back to March, also irritated old doubts. While it didn't have a meaningful impact on ratings, what disturbed network and agency executives was how long it took Nielsen to identify the problem, begging the question, what else isn't being caught?

The company described the glitch, moreover, in an unusual conference call with press that executives began by criticizing Rentrak, suggesting a bit of sensitivity about that company's recent moves. "They can't do program measurement," Mr. Hasker said on the call. "They can't measure digital or tablet consumption. They can't deliver overnights. And they have no tools to measure out-of-home viewers." In other words, don't think they can replace us.

Alternatives
Nielsen's panel is actually highly regarded in the industry, and its dominant position often merely makes it a go-to scapegoat. Mr. Poltrack said if there's one thing Nielsen does know, it's how to manage a research panel. "Companies have tried to challenge Nielsen, but found they haven't been able to do it better," he said.

"On a national level they are taken for granted," said Artie Bulgrin, senior VP-global research and analytics, ESPN. "Their national TV currency is among the best in the world,"

Rentrak has been trying to carve some space for itself with set-top box data from more than 14 million households, allowing marketers to more effectively target specific audiences. WPP and Publicis Groupe's Zenith Media have both recently struck deals with Rentrak to use the service in planning and buying advertising, the coverage of which sparked Nielsen's conference-call attack.

For all that, no one is expecting Rentrak to truly challenge Nielsen. "The No. 1 always gets the brunt of the complaints, but this doesn't mean others don't have issues," said Sam Armando, senior VP-director of strategic intelligence, SMGx.

Rentrak doesn't break out demographics, and there are also concerns that the data isn't wholly representative. It could be flawed, too, if set-top boxes are on but the TV is turned off, according to media buyers and TV executives.

Rentrak poses some risk to Nielsen at the local level, where Nielsen is still using paper diaries in some markets. And while it won't likely be used as a currency, Rentrak is considered a valuable selling and planning tool for TV networks and agencies. Rentrak is also looking to release overnight ratings in several markets in the next few months, and recently was granted patents for its method to calculate exact commercial ratings.

While both industry insiders and Wall Street have been skeptical on Nielsen's future growth, analysts are confident in its future. Nielsen's profit declined 30% in the third quarter to $91 million, but that decline was largely due to a refinancing of its long-term debt. Revenue climbed 13% to $1.57 billion.

Perhaps the biggest problem for Nielsen's live-TV tracking is splintering audiences, which make it less likely that its representative sample of about 22,300 households can correctly gauge viewership for smaller channels.

"Because of the fragmentation of audience and how people watch content, the current sample is being pushed to the limit," Mr. Schwartz said. "It needs a much bigger sample and I don't know if they are going to get that."

Ms. Clarken said Nielsen is working to expand the size of the panel. The panel could eventually include about 100,000 households, but exactly when that might happen is unclear.

Nielsen is also considering how to supplement the panel with direct measurement from data sources like set-top boxes, she said, but that's another effort with no announced time frame for adoption.

No one is calling for the demise of basic ratings; there's an agreement that there will always be the need to know how many eyeballs are watching a program. But TV measurement will be a combination of the panel approach and directly-ascertained "census-level" data.

The lack of clarity about when several integral pieces of the puzzle will come together has made the industry skeptical of Nielsen's ability to operate in the fast-paced media ecosystem.

Ms. Clarken argued Nielsen isn't always playing catch up. "Sometimes we are way too early, with OCR [Online Campaign Ratings] for instance," she said. "It was conceived in 2003 and we had a product in the market called Advertising Intelligence and it just fizzled. It didn't work because the advertising community didn't understand digital advertising enough to understand they needed to have accountability measurements."

Alphabet soup
Despite the plethora of demands the industry is placing on Nielsen, and the sheer amount of work that's ahead of them, when asked what the company's biggest challenge is, executives point to communication. "Because we are a research company, sometimes we get in our own way," Ms. Clarken said.

The industry wouldn't argue. The typical Nielsen PR strategy has been to announce a product before it has really been thought through, a TV executive said. "They are notorious for over-promising on things and then by the time the product comes out they have made so many changes, no one even knows what it does."

Ms. Clarken apologized for what she calls Nielsen's "alphabet soup," the string of three-letter acronyms that define many of their products, and said the company is in the midst of a naming exercise to correct that. It's also recently overhauled its communication team and brought in Katie Burke, an Edelman vet, to serve as exec VP of marketing. Ms. Burke said a year from now Nielsen will feel and sound a whole lot different.

Nielsen will roll out its first business-to-business marketing campaign in over five years in early 2015 in an effort to better communicate its strategy to the marketplace.

This isn't the first time, however, that Nielsen has blamed poor communication for its problems. In a 1995 article in Multichannel News, Nielsen's president-CEO at the time said, "In the past, frankly, we've been too busy doing the work, and have not spent enough time communicating."

Perhaps more important, there's been some significant shakeup at the top over the last year as Nielsen looks to reframe the organization as digitally focused.

Mitch Barns took over as CEO in January, succeeding David Calhoun, and this summer Mr. Hasker was promoted to global president, overseeing Nielsen's business across the media sector and leading both the audience-measurement and consumer-shopping-research segments of the company. Ms. Clarken, herself, assumed her current role and relocated to the U.S. in May 2013.

"Their corporate culture is adapting," Mr. Poltrack said. "It's become easier to work with than 10 years ago."

It's still unclear how well-vetted research and light-speed change can co-exist, but Nielsen seems determined to think further ahead than in the past. Mr. Hasker and Ms. Clarken identified programmatic advertising, wearable devices and in-car viewing as emerging areas the company is preparing to tackle.

And as advertisers look to target audiences beyond the traditional age and sex demographics that are Nielsen staples, the company is investing in its "buy" side of the business, which provides advertisers with data on consumer reactions to their campaigns, and insight into how well their marketing resonates and prompts people to make a purchase.

Nielsen has also realized the importance of partnerships and acquisitions in helping it keep pace with the changing landscape. It recently partnered with Adobe to deepen its measurement of online video, and Mr. Hasker said Nielsen is eyeing deals with big data owners and aggregators. It's also paying attention to ad tech firms that can help integrate data.

It Takes Two
While much relies on Nielsen's ability to develop new measurement tools, the problem isn't solely Nielsen's to solve. "It is as much Nielsen's problem as an industry problem," ESPN's Mr. Bulgrin said.

GroupM's Mr. Schwartz contends the industry can't rely on Nielsen. "Either we sit here and go down with Nielsen or the industry builds a better mousetrap," he said.

Ms. Clarken argued that even once Nielsen solves for the gaps in measurement, the currency still won't reflect the change in consumer behavior.

Due to rules that govern the standard C3 and C7 commercial ratings, rules set not just by Nielsen but also the industry, ads must be the same in the playback of content in the three and seven days after an episode airs as in the live broadcast in order to be credited. That means if any ads are changed from the original broadcast, that viewership does not count in the total.

"There's a big difference between what we can measure and what's in the currency," Mr. Hasker said. "The currency needs to be modified. It needs to measure total audience, duplicated and unduplicated, and total exposure of ads however they are placed."

That's something Nielsen can't change or decide on its own. The question is, who will lead that charge?