Focus groups should be abolished

Published on .

Malcolm Gladwell, author of "The Tipping Point" and "Blink," keynoted at last week's American Association of Advertising Agencies' Account Planning conference in Chicago. Below is an edited excerpt from his speech.

I want to start with a story about chairs, and I apologize if you read my book. Well, actually, not apologize, but you will be familiar with this if you read my book, and I just urge you, I guess, to go out and buy it again.

Herman Miller about 10 years ago put together a team to come up with a transformative office chair. And so (its designers) went back to their labs and threw out everything they thought about chairs and decided to start from scratch. They realized that one of the problems with chairs is that chairs had a single back and your back, of course, is not one piece. So they made a chair that had a way for you to adjust the lower back.

Another problem with chairs is they're hot. So they designed this special fabric that could breathe. And they did all other kinds of things about the way you could adjust the chair, and finally they had a prototype. Herman Miller did what companies do when you come up with a new idea, it went out and did some market research. And in the chair business, there are two kinds of market research they do.

They test the chair for aesthetics on a scale of zero to 10. Then they also test zero to 10 for comfort. They take this chair out and they gather all their focus groups and the comfort scores come back first and they're pretty good. The aesthetic scores suck. They are some of the lowest scores ever. Now normally, in the real world, when you come up with an idea and the people who are supposed to buy it hate it, you go back to the drawing board, right? If your customer hates it, it's suicide.

But Herman Miller said, "You know what, we're going forward." Over the course of the next two years, this chair that everyone said that they hated, turns out, they didn't really hate it. Sales start to go up. People start to look at the chair and say, "Well, actually it is comfortable and I think it's kind of cool-looking." Suddenly this chair, called the Aeron chair, becomes a cultural and commercial phenomenon and one of the greatest-selling chairs in the history of office chairs.

That's a story that tells us something very sobering about the institution market research. And in particular, the efficacy and usefulness of focus groups. Because the whole point of focus groups is to be able to help us predict what's going to work and what's not. If a focus group cannot do that, then a focus group is actually useless.


What I want to do is walk through why I think focus groups generally fail, and why I think we can do better.

The first notion has to do with what's called the storytelling problem. When you are measuring first impressions or snap judgments, we're not prying into the reasons why we make that snap judgment. It's an absolutely bedrock notion in psychology and cognitive psychology that somehow miraculously escapes the understanding of anyone in advertising.

There is an extraordinary faculty that many art experts have in being able to tell instantly whether a given work of art is real or fake. Boom, it's real. Boom, it's fake. And they are invariably right. But the interesting thing about that is they cannot ever tell you why they think it's fake.

That's something that's fundamentally true of our first impressions and snap judgments. If we cannot have access to the machinery of our decision-making when it comes to first impressions, what happens when you ask people to give those?

This creates engines for seemingly plausible fictions that people will spin in an attempt to spin what is cognitively impossible. Tennis coach Vic Braden spent years hanging out with tennis players asking how they play tennis. He said, "I've been asking pros about the topspin forehand for years, and they all say the same thing. They say, `When I hit a topspin forehand at the moment of impact with the ball, I roll my wrist to generate topspin.'"

He began to tape them and discovered that not a single one of them ever rolled their wrist on impact with the ball. They rolled their wrists after they hit the ball as part of their follow through. And in fact if you did what they said, you will sprain your wrist. What they did was come up with an extremely plausible story: They said, with utter certainty, "We roll our wrists." Of what use is that information? Worse than useless, dangerous.

Point No. 2: the triangle test. You give people the Pepsi challenge. Two glasses, Coke and Pepsi, and tell me which is which. And sure enough, they can. Any group of people randomly selected will probably get that right 85% of the time. Now you say "I'm going to change the test a bit, we're going to use three glasses. The third glass will have either Coke or Pepsi, but this time I'm going to make it easier on you. I only want to know which one is not like the other two."

The success rate when you do a triangle test, three glasses, goes from 85% down to chance, or one-third. Why are the suddenly incapable of telling you which is Coke or Pepsi? By adding that third one, you turn what had been a total first impression into something that required thought, and the minute you ask someone to think about their impressions, you change their preference. You destroy their ability to know immediately the difference between these two things in the real world.

There is a third consequence, and that is that the act of asking someone to explain these things is not only a psychological impossibility, not only impairs their judgment, but it biases them in favor of the conservative, in favor of the known over the unknown.


Let me give you an example. I have a whole bunch of posters and I ask all of you over here to pick any one you want and take it home. You over here, I'm going to focus group you. I want you to write out for me why you like certain posters, then take one home. Six months later I call you, and the first group, you love them. And (the second group) hates them. That makes sense based on what we've thought about because by making you think about your choice, explain in a focus group what you felt, I distanced you from your true preference. But the posters were of two kinds. One set were all impressionist prints and one set of posters were of cats hanging from bars, and it says "Hang in There, Baby." Think about this, why did you (in the second group) pick cats? What was it about what I made you go through that biased you against the beautiful poster in favor of the cheeseball one? It's a language bias.

If I force you to tell me what you think, you are profoundly limited by your vocabulary. It is hard for us to summon the language to explain why we like that impressionism, but it is very easy for us to summon the language to explain why we like the cats.

Now think about the Aeron chair. They say they don't like the chair, of course they don't. The chair is nothing they've ever seen before, but that was the whole plan in designing the chair. But that's what's wonderful about it, that's why this chair will make billions of dollars for Herman Miller, but it's also what dooms that chair in the focus group, because people don't have the language.

Market research, when it is observational or when it is interpretative, is profoundly useful. But those are two critical things. They require the intervention of the person conducting the research. They require the findings that are gathered are considered, and thought about, and processed and interpreted. Back in the 1950s, most of the major advertising agencies on Madison Avenue employed Freudian psychoanalysts for this precise reason, and you don't see that anymore. I think that's a big mistake.

This understanding about what's so terrible about focus groups ought to pave the way that we manage people. First and foremost, it's very important for management to trust the creative talent.

The second thing is patience. The more breakthrough, the more revolutionary and the more innovative an idea is, the longer it will take for people to come to appreciate it.

The third thing is it requires people in management to tolerate uncertainty. The thing that's driving all this focus-group and market-research data is the desire of people with the management power to make every decision as methodical and thought out and certain as possible.

In this article:
Most Popular