That's the question at the heart of Microsoft's "engagement mapping" project, a new approach currently in beta testing with publishers and marketers, that aims to assign values to different online touch points, not just the last-ad-clicked before a sale.
Microsoft isn't alone in trying to measure engagement. Engagement may indeed be the online industry's holy grail. DoubleClick, now owned by Google, helps customers measure engagement by integrating reporting results from many different media, in part with help from its Performics search engine optimization (SEO) unit. Web analytics vendors such as Coremetrics and Omniture Inc. increasingly focus on measuring engagement.
Most aggressive effort
But Microsoft's effort may be the most aggressive as well as the most closely watched attempt to measure the true impact of online marketing on users.
"Customers are exposed to one hundred times the online media today" than was common in the past, when a last-ad-clicked model became the de facto standard for ad reporting, said John Chandler, an analyst in Microsoft's Digital Atlas ad serving group. That means it's highly unlikely that a single display banner or search engine ad deserves full credit for delivering a sale, Chandler said. "In the offline world, the idea of a last-ad model doesn't really exist," he said. "TV, radio and billboards aren't about closing the deal. In the online world, we ended up in this place without really meaning to."
It's no coincidence that lessening the impact of search ads (Google's strength) and display ads (where DoubleClick rules) would have a major impact on Microsoft's ad rivals. But it could also have a significant impact on ad buyers and publishers if credit for an ad's success moves beyond pay-per-click and pay-per-action.
"This is definitely going to change the way we evaluate online media. It has the potential to be very controversial," said Raphael Rivilla, VP-interactive media director for direct response-focused interactive agency Bennett Kuhn Varner, a participant in the Microsoft project.
"[With] an engagement model, we'll now be able to see how a user was first encountered. We won't just pay the network where the last ad was clicked; we should be paying the previous network as well."
Varner added that the problem today is that ads aren't bought that way. "The last-ad-clicked gets all the credit," he said, adding, "But we've always been interested in optimizing the upper [end of the] funnel of the conversion process."
For publishers, changes in measuring engagement cut both ways, said Microsoft's Chandler. "Publishers haven't always been getting a fair shake" for their role in converting site visitors to buyers, he said. At the same time, he added, "there's anxiety about having the role of the last site [in a customer path] diminished."
Some publishers, however, could benefit from the engagement approach, in particular if they run a network of related sites, said Sam Whitmore, consultant and editor of Sam Whitmore's Media Survey.
"Engagement is likely to prove impossible to distill across the enormity of the Web. But within a given site or family of sites, publishers will be able to track multiple exposures to brands and their content," he said. "And that may be the reason that big advertisers consolidate their spend with a tiny number of mega-portal sites or blog networks, such as Federated Media or the new one from Forbes."
Under that scenario, a large publisher such as Dow Jones & Co. could sell an integrated ad package across Barron's, Marketwire and WSJ.com and deliver such metrics to ad buyers as time-on-site and pathing that "can prove that most `buyers' grew more engaged when they went from here to here to here," Whitmore said.
Tracking engagement is tricky for marketers, too. At a high level, Microsoft's engagement mapping measures the reach of an ad and how it leads to conversions, as well as the quality of that reach, Chandler said. Quality metrics include frequency, recency, ad size and media richness.
All those data lead to a conversion attribution, which measures the role a particular ad played in a customer's online path to action. All that is part of Microsoft's Atlas Media Console.
While still testing in beta, Microsoft "is learning the types of questions advertisers need answered, as well as how credit should be shared and the relative weight that should be given to different types of media," Chandler said.
Determining the right way to measure engagement can indeed be challenging. Beta tester Bennett Kuhn Varner worked with two months of historical data and had to build and test nine different engagement scenarios before finding one that worked, Rivilla said.
"Now we're seeing a really nice redistribution [of credit] with one of the models we're using," he said. "It's tricky. You really have to work to set the variables and percent weights correctly."