Roundtable: Taking measure of which metrics matter

By Published on .

Marketing metrics have never been more important. The weak economy, which has caused scrutiny of all budgets, has given marketers yet another reason to put in place processes to measure the impact of their efforts.

But are they acting on the volume of data flowing from their marketing channels, particularly online? Even more fundamentally, are they aligning their measurements with business goals? And if they are dedicated to improving their analytical capabilities, where are they getting people with the right skills?

BtoB Editor Ellis Booker recently asked these and other questions during a telephone roundtable composed of Pamela Jacobs, manager-worldwide developer communications at IBM Corp.; Laura Patterson, president and co-founder of VisionEdge Marketing; Boren Novakovic, director of b-to-b operations at Whirlpool Corp.; and Jim Sterne, producer of the annual eMetrics Marketing Optimization Summit (going on this week in San Francisco) and founding director and chairman of the Web Analytics Association.

BtoB: What are b-to-b marketers measuring?

Jim Sterne: Everything they can, and that ranges from how many people respond to an ad to how many sales are closed and then trying to hook up those two end pieces. And the tough part is, my goodness, we've got so much data. How do we sift through it?

Is it wise to spend money on the newer or measurable media? I think we're not seeing the movement of money over there because it's measurable. I think we're seeing the movement of money over there because it seems to be a better connector. In other words, search marketing is people reaching in and pulling information rather than having it pushed at them.

Laura Patterson: I think that's a really good observation, but the challenge that we're seeing when we work with people in this area of marketing and measurement is that they tend to be very down in the weeds of what they measure. So they're measuring every activity they can. But in terms of being able to talk about how it's impacting the business, they're pretty much still struggling.

We just completed our [annual] 2008 marketing performance measurement and management surveys. And we are continuing to see people trying to measure things like market share, purchase frequency and recency, and people keep talking about needing to measure how they are increasing business with existing customers. But when you look at what they say they're measuring, they're hardly measuring anything related to that. They don't look at things like, for example, rate of customer acquisition, or customer loyalty or customer lifetime value.

BtoB: So you're saying there is a disconnect between what's being measured and some of the objectives?

Patterson: According to everything we are seeing out there and their resource, the answer to that is yes.

Pamela Jacobs: At IBM, it is aligned. We're measuring everything we can, and we have analytic teams that work with just those measurements, and they align the measurements with our objectives so that way we are able to bridge the gulf between what we're measuring and our objectives.

BtoB: How's it at Whirlpool?

Boran Novakovic: I'm in the supply chain organization, and we work very closely with our merchandising and sales groups. So it's maybe a little bit of a different setting. Our b-to-b Web site operation [is] targeted toward our retailers and dealers, if you will. So we do really two broad categories of metrics. One is strictly focused [on], I would say, operational metrics on how much business are we driving through the Web. So we measure purchase size and cost per transaction. And we measure revenue with that and also the percent of channel adoption.

BtoB: How have your measurement priorities changed over the last year? Why have they changed, if they have?

Jacobs: I'll just take a moment to talk about what we do with our developer Works' Web site. Software developers are very, very comfortable on the Web. So primarily our marketing is through the Web and our Web site. Developers, because buying software is a complex purchase, must spend time learning about the product to make sure that it meets the needs that they are trying to achieve. So our role in marketing is very different, I would say, from other b-to-b companies in that we need to train [our audience]. We need to provide all of these resources to them so that they have what they need to make a valuation and a decision.

So we measure training, and our strategy over the last five years has been to hone in on the new Web 2.0 offerings, which are community-based, like forums, podcasts and chats. Our measurements are morphing as the technology morphs.

BtoB: Can you give us a feel for what those engagement metrics are? Is it number of posts that an individual does in a period or how often they come to the site? What sorts of things are you looking at, since you're not using the classic click-through numbers?

Jacobs: [We] have a very, very, very active electronic marketing group, and in fact I manage that group. And so we do [measure] the click-throughs. We also measure registrations, and we then can align our marketing to their behavior on the Web. We're measuring unique visitors, we're measuring who our evangelists are, we're measuring who the experts are that are in our community site and we're working toward a measurement system where the peers can rate the experts and we can find out exactly who are the leading experts in the field from that measurement.

Novakovic: Our measurement strategy has stayed fairly constant [for] the last two years. It focuses, again, on a lot of those things I talked about earlier. And I think the next frontier of measurement is how do we really drive more customer focus and understand all the stuff we can do—what is truly important to a particular segment—and just focus on that?

BtoB: Jim, you've been talking about the changes in measurement for a few years now. How do you see the forest for the trees?

Sterne: There's a convergence of the Web metrics kind of things, the Web analytics click-through, with the measuring marketing ROI in general and measuring marketing processes. The Web people discovered that they could measure stuff, and now the whole marketing department's saying, "Oh, this should be part and parcel of our global marketing return on investment analysis."

BtoB: How do you see the great unwashed in terms of organizing themselves for cross-department use of analytics?

Patterson: I want to acknowledge that IBM and Whirlpool are best- in-class companies in just about every way you can think of. So the things that they're doing, I suspect, are light years ahead of many other b-to-b organizations, particularly the midsize companies. Many [of these companies] didn't even have a measurement strategy six months ago. It's very common for us to see [companies] tracking results [for which] they've never even established performance targets. When they get to the end of an initiative or a program, they can arbitrarily declare a success because they didn't have a stake in the ground to begin with. There was never any measurement system, strategy or process.

Sterne: It's been my feeling that the whole concept of return on investment has fallen down in the process—that people do a heavy ROI analysis in order to get funding to get a project going—but at the end of that project, nobody ever goes back to revisit, "Did we reach our numbers?"

BtoB: Has the current economic scenario impacted your strategy?

Sterne: It's a matter of controlling spending. Where do you cut your spending on those things that are not demonstrably giving you return? So interest in and attention to metrics tends to blossom in tough times.

Jacobs: At IBM, the economic scenario has created more of a need for marketers to link their efforts with sales, which is really good business because we need to be able to measure our value in providing leads. So I think more effort or more energy is spent identifying our impact on sales leads.

Novakovic: We've been fairly disciplined around measuring our ROI and focusing our business cases every time we go through our annual budgeting process. These times bring even more scrutiny, and that's why having these metrics is extremely valuable because they make the business case stronger. We have a fairly structured audit process. So [metrics] really help drive even more focus, and it helps because it clarifies your ideas and mission. In times like these, everybody's fighting for priorities. Having fact-based data, I think, helps establish that priority.

BtoB: Are your online measurement strategies affecting your offline strategies?

Jacobs: Well, absolutely. Our online measurement strategies are affecting offline strategies, and here's a great example. IBM Developer Works, you know, is a Web portal. But we sponsor offline events all over the world. We are sending e-mails, and then they go to our Web site and register. When they register, we are extracting all kinds of information from how fast the class fills up—what topics are good, say, in Russia, what are not working in India—and all of that data is then evaluated for the next year on how we will direct our offline strategy.

Novakovic: What we're trying to do is not look at them in the binary way as either/or, but rather how they can complement each other. I'll give you a couple of examples. What is the cycle time to deploy a promotion by using the Web, as well as hands-on communication and in-house selling that will enhance the value of that promotion and how it's executed overall versus doing it one way or the other? We're trying to look for synergies.

BtoB: Where are you getting the people to handle analytics? Are you training them internally? Are you hiring from the outside? Where do you find good talent?

Sterne: You can grow it from inside, but it is more about character traits than it is, at the moment, about education. It's great if they have a math background. But they have to be curious. Tim Goudy at Coca-Cola, in preparation for his keynote for the eMetrics Summit, said they've made a decision to outsource the analysis because the tools are changing so fast. The business needs don't change very quickly, but what you can measure, and how to correlate it and what do the data mean is something that takes a trained person who's got experience at a number of different companies.

Patterson: Many [product marketing people], particularly on the marketing communications side, don't exercise a lot of analytical muscle. So there has been a real challenge or gap in skill sets. Our recent research said that only about a third of the organizations feel that their people have the skill-sets to be able to do the kind of analytical work required of them, and they're trying to get them trained.

But the challenge isn't just there; it's even at the university level. I find that in many of the schools, [they're] not pushing the analytics as hard as they should be.

Jacobs: IBM, I guess, can be differentiated in that we train and hire from within. What happens is people learn about the databases, and they learn about the entire process and so they are always in training.

Novakovic: [Whirlpool is] very similar to IBM in terms of we do it from within. We're looking for people with engineering backgrounds or people who are really process- oriented. You have to be smart, but you have to have some affinity to work with technology such as databases. You have to understand the business process because you've got to put it in the right context. Then we give them additional analytical training by sending them to our in-house Six Sigma program. After they come out of four months of that rigorous analytical training, they're pretty much ready; you can just unleash them, and they're amazing.

BtoB: Is it critical that your senior management get a summary of this measurement data?

Jacobs: Yes, our senior management does see the data. The measurements are created to justify your return on investment and what you're doing with your campaigns. So, yes, the measurements and the data are raised up to the senior level, and it's absolutely important. And actually, to get back to the original question, I would say that most everyone in our organization from the Web—the bottom-level Web person all the way up to senior management—is looking at it.

Novakovic: Data at a certain process level goes to the appropriate people. So people that are engaged in the activity day-to-day, they look at, let's say, most of the data. Our senior managers and executives just look at more aggregate and more, I would say, business unit-level metrics.

Sterne: It's critically important. At the very bottom level, we've got people who are optimizing specific processes, a direct mail campaign that is driving leads to the sales department. And at the upper level, we've got people who are responsible for overall strategy. This is an incredible tool for, you know, when you look at the ability for Web analytics to measure behavior on a Web site.

BtoB: Yet there are marketers who are frantically nervous about having senior management who may not be educated on how to read these reports.

Sterne: Reports are the terror because people love reports, and they immediately fall into the weeds of the data instead of looking at the big trends. And that's why it's very important for us to only give the right amount of information to the right level in the organization.

BtoB: Laura, are there any rules of thumb on figuring that out?

Patterson: Most people believe that they should have a minimum of three levels. There's a level that's the executive level that looks at it from the strategic perspective that Jim is mentioning. Then there's an operational dashboard that the CMO and the VP of marketing should be using to look at processes, and efficiencies and effectiveness at a level down that they're probably using with their staff. And then there's a third level often used at the tactical level that is used by either the individuals, or program managers or departments, depending on how the organization's set up, that looks at those activities, those tasks and the results of those against performance and the returns of those.

Most Popular
In this article: