In July, the American Marketing Association's Journal of Marketing published an article by a group of researchers and academics challenging the claim that NPS is the most effective metric linking customer loyalty to company growth.
NPS, which was developed by customer loyalty expert Fred Reichheld, director emeritus at Bain & Co., is a simple score that is based on one question: "How likely are you to recommend our company to friends or colleagues?"
Since NPS was first introduced by Reichheld in a book published last year, "The Ultimate Question: Driving Good Profits and True Growth" (Harvard Business School Publishing Corp.), many leading b-to-b marketers, including American Express Co., Dupont, General Electric Co. and Intuit Inc., have embraced the metric as a key indicator of corporate performance.
However, researchers challenging the effectiveness of NPS claim that after replicating some of the research methodologies used by Reichheld in developing NPS, there is a lack of evidence to support the claim that NPS is a superior metric to other loyalty measures.
"Given the widespread adoption of Net Promoter [Score], we believed it appropriate to examine the claims attributed to the metric," said Timothy Keiningham, senior VP-head of consulting at IPSOS Loyalty, a loyalty consulting firm, and co-author of the report.
The other co-authors are Bruce Cooil, professor of management at the Vanderbilt Owen Graduate School of Management; Tor Wallin Andreassen, professor of marketing at Norwegian School of Management; and Lerzan Aksoy, professor at Koc University in Istanbul, Turkey.
The authors sought to replicate research used in Net Promoter Score findings with independent research across a number of industries.
"Our research clearly shows that claims of Net Promoter's superiority in predicting firm growth, or in predicting customers' future loyalty behaviors, are false," Keiningham said. "The consequences are the potential misallocation of resources as a function of erroneous strategies guided by Net Promoter on firm performance, company value and shareholder wealth."
Reichheld could not be reached for a comment on the Journal of Marketing report.
Report won't sway some
Marketers that have embraced NPS say the debate will not affect their use of the metric.
Phil Clement, CMO at Aon Corp., a risk management and reinsurance company, said he's aware of the debate but that it doesn't change his point of view about the effectiveness of NPS.
"What metric you use for customer satisfaction is something you are always thinking about," he said. "You always have to be skeptical about your data."
Aon began using NPS about a year and a half ago. It surveys every client at least once a year with an NPS-based questionnaire, as well as supplemental research including focus groups and phone interviews.
"I believe the referring question [used to calculate NPS] is one of the best [research methods]," Clement said.
Mindy Eiermann, senior product manager at PayCycle, said the online payroll company has been very pleased with the results of NPS since it started using it last year.
"It has been very effective for our organization," Eiermann said. "It has provided us with actionable data we can all rally around. Everyone in the company, from the CEO to product managers to product marketers, knows what is going on with NPS and its results."
Three times a year, PayCycle surveys a sample base of its customers and asks them four questions: What was your previous payroll method? How likely is it that you would recommend PayCycle? What is the main reason for the answer you gave above? What can we add or change to improve PayCycle?
The company then follows up on the data and makes changes when necessary.
"It has had a huge impact on the way in which we run our organization," Eiermann said.
More research needed
Other researchers said they will continue to study the effectiveness of NPS.
Ralph Oliva, executive director of the Institute for the Study of Business Markets (ISBM) and professor of marketing at Smeal College of Business Administration at Pennsylvania State University, said ISBM will do further research on NPS and other loyalty metrics.
"The whole business of customer satisfaction is controversial," Oliva said. "NPS is about as good as any other metric out there, but we are tracking other lines of research to get at a compact set of metrics that can be used to measure customer loyalty and advocacy."
Gary Slack, senior partner at b-to-b agency Slack Barshinger, Chicago, said, "Frankly, we don't know what or who to believe now. NPS loyalists dismiss the Journal of Marketing paper as the work of economically threatened customer satisfaction consultants. NPS critics say it's about time NPS is getting this level of scrutiny. We need a respected third party to sort out the controversy and tell us if and how we should be talking about NPS."