Many years ago, I was the art director on a TV spot that the client decided to "focus group" because the content seemed "risky." So we asked a few dozen 35 years-old women what they thought.
To my delight the ad tested really well. So well in fact, the client decided to test it again. The re-test scored even higher. I'll never forget the feeling of euphoria I felt when the client's director of research stated that in his 30+-year career he had never had anything test as well as my little 30-second spot. Yes! Finally testing creative actually came out in favor of the work! My heart was filled with joy.
It was short-lived.
The client asked the agency to wait in the hall while the marketing team discussed what the next steps should be. (We were quite a close team with this client as you can see). A half hour later we were readmitted to the meeting. The marketing director said the campaign would not be produced because the CMO didn't like the idea. (Did I mention he had excused himself prior to us returning?)
I think this is why I loathe testing creative concepts so much. The actual motive is so often suspect. Anyone who has sat behind the glass of a focus group knows that this is a flawed method of determining whether an ad concept will be effective. Nevertheless, the archaic system still gets suggested as a viable research method by both agencies and clients.
This is one of the benefits of having small clients with small budgets. They test the work by actually running it. They use results to determine if something will work. If it doesn't we do something else. Even then a client can really believe creative that is not their cup of tea doesn't work. Last year we had a client let us know they really hated the TV campaign we produced. Their employees didn't like it. They had people call in and say they didn't like it either. We asked how there sales were going. They admitted they were up by double digits after several years of decline but it couldn't be the TV work, they said, because "nobody" liked it.