It's a stunning admission for a company who's flagship brand has
61.5 million fans, more than any other brand on Facebook. But Eric
Schmidt, senior manager-marketing strategy and insights at
Coca-Cola, isn't giving up on buzz just yet. And he cautioned
against reading too much into the research, noting that it covers
only buzz, not sharing, video views or other aspects of social
But when Coca-Cola put buzz sentiment data into the same
analytical framework it uses to evaluate other digital media, Mr.
Schmidt said, "We didn't see any statistically significant
relationship between our buzz and our short-term sales."
That was at a 95% confidence level, but even stepping back from
that high standard, he said showed buzz affecting sales by only
"Is that the end of the story?" Mr. Schmidt asked. "I would say
no. This is one study on a set of brands in a particular company
within a certain segment of the consumer-packaged-goods industry.
It is by no means a generalized result that applies to all
Now Mr. Schmidt said Coke is looking to refine how it measures
buzz, for example by getting a better idea of how many people buzz
actually reaches rather than just counting the raw publicly
available comments from such sources as Facebook, Twitter, blogs
Coke research was far more favorable for digital display
advertising, which it found on average to be 90% as effective as TV
at generating sales on a per-impression basis, Mr. Schmidt said.
Search was 50% as effective as TV – about the same as
out-of-home – with radio coming in between TV and search and
print scoring slightly more effective than TV.
One problem Coca-Cola has is determining whether buzz is
actually positive or negative in the first place. In one 2010 study
where Coke pulled out more than 1,000 social-media messages
randomly and had human raters compare them to automated sentiment
analysis by one vendor, there were widespread differences.
"When we say it's positive, the machine about 21% of the time
says it's negative," he said. "That can cause some problems in our
understanding" of how buzz impacts sales.
Machines have the most trouble judging sentiment in longer posts
such as those in blogs or Facebook and do much better on Twitter,
It's important to get digital reach and viewership data on par
with TV and to accurately measure buzz in order to get more
accurate return-on-investment analysis for all media, Mr. Schmidt
"Digital ROI has to be a financial measure that allows us a
consistent measurement of business performance over time," he said,
and also lets marketers make informed tradeoffs between media.