"In a year, that means 20% of all addresses change, 21% of CEOs change and 18% of telephone numbers change," said Krishna Chettayar, assistant VP-marketing strategy at D&B, one of the world's largest providers of business information. "And the average database decays at a rate of between 2% to 5% per month."
It's little wonder why data quality has become one of corporate America's biggest priorities for 2005 and beyond. "Every organization feels some pain from inaccurate and incomplete data," said Ted Friedman, principal analyst with IT research company Gartner Inc. "Bad data result in wasted efforts, unnecessary expenses and poor efficiency, especially in marketing and upfront business planning."
Not an it responsibility
But the most important thing about cleansing data is that it's not an IT responsibility-it's a business issue, Friedman said. "Although there have emerged some very sophisticated data-quality suites with far-ranging capabilities, this is not something you can fix simply with some out-of-the-box software solution," he said. "It requires a complete commitment to overhauling the way the enterprise processes and stores its data."
The industry's leading data-quality vendors agree. Most have worked hard to generate top-down buy-in of enterprisewide data-quality initiatives, as well as to provide customized implementations and consulting services for their major clients.
"Much attention has been focused on the data-cleansing aspects of data quality," said Jeff Cohen, VP-marketing for Group 1 Software, a division of Pitney Bowes. "However, this is only one part of the quality equation, which also should include data integration, consistent business rules and a strong dose of common sense.
While data quality may once have been considered a nice-to-have initiative, organizations now realize that it is an absolute necessity, especially for `bet your business,' mission-critical applications or those required in order to meet governmental reporting and disclosure requirements."
In the past, most companies did not recognize the need for data quality until it was too late, Cohen said, usually when an expected high-return initiative failed due to some faulty data. "It's all been reactive rather than proactive," he said. "A forward-thinking company should integrate data-quality checks and controls into its everyday operations. While this may not happen overnight, recent regulatory and homeland security initiatives are forcing the issue."
Bad data via suppliers
Even companies that regularly cleanse existing data may not be preventing bad data from entering via suppliers, list providers, customers and others, said Len Dubois, VP-marketing for Harte-Hanks Trillium Software.
Gartner's Friedman uses an IT term to describe this process as a data-quality "firewall." "Good data-quality systems will check incoming data and correct it before it's integrated into the enterprise," he said. "You want to do everything in your power to ensure that you don't destroy your clean, existing databases in the process."
Savvy business information providers such as D&B are helping with the process by putting their databases through intensive cleansing before they ship them off to clients. "It's important to work with data vendors that you can trust," Friedman said. "Some are doing very good jobs in providing quality data, but all can improve."
In the case of D&B, the company not only provides clients with clean data but has realized it also can share its proprietary data-cleansing expertise. "Our custom information management service has become a $100 million business for us, a 20% increase last year over 2003," D&B's Chettayar said. "We have been very successful in offering our consulting services and selling licensable technology. After all, data is our business."
Most data-quality suites today still lack the ability to analyze and cleanse data in real time, and across multiple applications and data warehouses throughout the enterprise, Friedman said. "Some software vendors-such as Group 1, Trillium and D&B-are improving that functionality, but the technology is not easy to deploy since the applications all handle data so differently," he said.
Larger companies choosing a data-quality solution should make sure it has full functionality or is easily upgradable, Friedman said, because "we're talking six figures for implementation, usually in the $100,000 to $300,000 range."
Is it worth the cost?
Is such an investment worth it? "You need to try to quantify the negative impact of bad data-dynamically, not statically," Friedman said. "But you'd be surprised at how much money a company can save with a good data-quality system in place. A 2003 Data Warehouse Institute report said bad customer data cost U.S. businesses more than $600 billion a year in postage, printing and staff overhead-not to mention lost opportunities."
Better data mean better ROI from marketing campaigns, added D&B's Chettayar.
Trillium's Dubois shared a prime example: "Most lists have 15% to 20% duplicate names. If you don't de-dupe them, you'll have a skewed response rate. So when you expect a 2% rate from the 100,000 potential customers for a given campaign, you'll actually be getting 2% from 80,000-85,000. You could scrap the campaign because you were working off erroneous assumptions."
Data quality doesn't only pertain to customer information. It's also important to maintain accurate and consistent product data for inventory and other purposes, Friedman said. "And maintaining quality financial data has become more important than ever because of the new post-Enron assortment of financial reporting regulations," he said.
Chettayar agreed, saying that such compliance is weighing heavily on everyone's mind. "In today's world, having good, clean data isn't just about reducing waste and improving returns; it's also about not going to jail," he said.