Big advertisers have for years been trying to trim what they pay agencies and production companies as a means to spend less on creative, or what finance executives call "nonworking media," to put more toward paid "working" media.

But is cutting the nonworking budget working for marketers? Some new research and data suggest no, though the results remain at best unclear for two of the largest companies involved, Unilever and Procter & Gamble Co.

The old arguments against the movement remain. The "nonworking" term implies creatives and production creatives and production houses don't really work, despite data from the Advertising Research Foundation that 75% of advertising return on investment comes from creative quality, not media placement. Then there's the rise of digital media, which simultaneously creates free or cheaper media while increasing demand for content to fill the giant media void.

But the newer arguments include a report earlier this year by the research group Warc, which said efforts by Unilever to cut agency and production costs may have hurt its ad effectiveness, at least as measured by cumulative results in 79 global ad competitions where effectiveness or strategy were among the award criteria. Unilever, which embarked on its creative cost-cutting efforts in 2009, saw its results in these effectiveness rankings slip significantly last year, putting it behind P&G.

Unilever slipped from No. 1 to No. 2 in the Warc rankings, with its number of top 100 campaigns falling from six to three. P&G rose from No. 2 to No. 1, with its number of top campaigns rising from three to nine.

Warc's head of content, David Tiltman, noted that Unilever had a head start, targeting creative costs as far back as 2009, whereas P&G announced its plans to cut agency and production costs first in 2012 and has focused most extensively on that
initiative only in the past two years.

"What we found interesting this year was that P&G has done very well and Unilever has done less well than previous years," Mr. Tiltman said. "Unilever is further down the road in all these cuts in nonworking media. Agencies have been saying repeatedly this is going to come back to hurt you long term in terms of results."

Unilever declined to comment. But P&G Global Brand Officer Marc Pritchard said he doesn't believe creative cost containment hurts creative quality. "We've seen just the opposite," he said. "We've been at this for the last few years, and while we have more to do, we're getting better creative and better execution. When we set out on this effort, we were deliberate in the cost savings we wanted to achieve and equally deliberate in our intention to work with only the best partners, to upgrade the creative quality and capability, all aimed at driving growth for the business."

Despite cost-cutting efforts by both marketers, Unilever and P&G remain the two biggest award
winners globally. Of course, they're also the two biggest ad spenders, so by the law of averages, they should be.

Another look at effectiveness data for both big marketers shows a different trend. Advertising Benchmark Index, which has been tracking consumer survey responses to every U.S. ad it can find from every advertiser since 2012, shows Unilever's overall ad effectiveness climbing fairly steadily throughout all those years of cost containment. P&G's effectiveness scores have remained fairly flat to slightly down, but haven't suffered amid recent creative cost cuts.

Overall, Unilever's index scores rose to 108 early this year from an average of 100 in early 2012, where 100 is the average for all ads from all advertisers. P&G's score fell to 108 from 109 during the period. But scores for both companies actually have been rising since the fourth quarter of 2014, a time when both were in full creative cost-cutting mode.

Warc is basing its findings on ad competitions judged by agency and marketer executives globally. ABX is basing its results on consumer surveys in the U.S. In the ultimate contest, Unilever has generally had better organic sales growth than P&G globally and in the U.S. since 2012, though other factors besides marketing play a role. Those include Unilever's greater concentration in emerging markets and spending power fueled by depreciation of the euro versus the dollar.

ABX data, drawn from partner Competitrack, also points to another development likely linked to nonworking-cost containment. U.S. marketers for the first time last year actually made fewer ads than the year before. Creative executions across all media found by ABX and rated by its survey panelists fell 1% to 588,000 last year. That follows increases of 7% to 10% in 2013 and 2014.

Some of that may owe to marketers moving budgets into such things as social media posts, recipes, mobile apps or other content that wouldn't be tracked as ads, said ABX President Gary Getto.

Yet one more problem developing for the nonworking-working ratio is accounting ambiguity.

One of the key growth areas for print and digital publishers, from BuzzFeed to The New York Times, has been production studios for video and other advertiser content. At the International Festival of Creativity in Cannes last year, media companies such as Snapchat, Pinterest, Vice and the Daily Mail announced efforts to produce creative for advertisers. Facebook and Google have likewise had growing creative and research departments for marketers.

Some agency executives privately believe the trend is fueled at least partly by the working-nonworking issue. Marketers under pressure from their finance and procurement departments to improve the ratio can have their creative produced by a media company that folds the costs into an invoice that falls conveniently into the "working media" side of the ratio.

As one agency executive notes, the allure of native content isn't just that it straddles the line between editorial and advertising, but also the line between working and nonworking media.

Regardless of the complexities, marketer focus on creative costs is here to stay, and agencies need to avoid looking like they're "whining," said Bryan Wiener, executive chairman of 360i. "It's perfectly reasonable for marketers to try to make their agency and production fees as low as possible, and to make their marketing dollars go as far as possible."

But the working-nonworking ratio isn't the way to do that, he said. "These ratios incentivize behavior that can't possibly be good for marketing effectiveness," he said, including "standardized media placements and cookie-cutter variations."