Tom Chi  

Lights… Camera… Ack!

October 31st, 2003 by Tom Chi :: see related comic

As many of you probably already know, Hollywood movies these days are often reviewed by focus groups before they reach the theatres. Directors create 2 or 3 possible endings, and focus groups hash out which is “best.” Do they fall in love? Does the villian explode or just get kicked off the island? Some serious changes can happen late in the process.

So what do people think? Do focus groups belong in the realm of creative endeavor? Are they a great example of customer feedback resulting in an improved product, or simply an example of groupthink creating a mediocre one?

I am certain that if Hamlet were being written today, that a focus group would give us a significantly different ending. Ophelia would somehow survive, and vile Claudius would probably be dispelled in a cinematic fireball. Rosencrantz & Guildenstern would be replaced by a pair of bumbling computer generated Gungans, and Hamlet’s “To Be or Not To Be” would become “To wear Nike? or To wear Adidas?” rapped over a P. Diddy sound track — somehow Ben Affleck would be involved.

Anyway, what is going wrong here? Are these just poorly run focus groups, or is there a deeper problem? Movie Studios love this approach because they feel as if they are maximizing profits and minimizing risks - something that we’d all love to do in the software (or any product) industry.

But wait. Is that what we really want to do? I think a major factor in focus group failure is that creative work often thrives on risk. The removal of risk saps the transformative qualities of a work, and leaves it with an unappealing “sameness”. Everywhere we look we see carbon copies of successful bands, movies, or products. And while they might nab a bit of market share, they rarely inspire.

12 Responses to “Lights… Camera… Ack!”
Frank Spada wrote:

I think there are two separate issues here - using focus groups for creative input from internal product development staff and using focus groups with end users or customers as a means to listen to their needs and wants. In the former case you most likely want to foster creativity, following the two heads are better than one approach. In the latter case I would argue that spawning creativity is not the goal. On the contrary, most (software) developers could stand to benefit greatly by first listening to users’ needs (really understanding their environment, their workflow, and the problems they face in using the software/hardware), THEN using some creativity to design a product that fits those needs. When used in these terms, focus groups (or any tool that fosters better communication and understanding with the customer) can be in an invaluable tool.

Elly wrote:

I agree that the removal of the risk factor produces mediocre products. In trying not to offend anyone you may well end up with something that noone really dislikes, but you’re equally likely to end up with something noone really likes either.
For example: If you go out into the street and ask everyone what their favorite and least favorite colours are you’ll not only get a lot of strange looks, but you’ll probably get very similar results for both - So what do you do?
The designer in me says “ignore them all and do whatever you want.” But everyone knows designers never make any money…..

Foolish Jordan wrote:

Hang on here. It’s one thing to say that focus groups often don’t work because of this and that and this other thing; but here all you’ve done is allege that some movies (which ones?) are made worse in some unspecified way (how?) because studios use focus groups.

Where’s the beef? Give us a couple of examples of movies that would’ve been better without focus groups. Better yet, give us some examples of movies that would’ve made more money if not for focus group directed changes.

Do we even know for a fact that movie studios don’t manage their focus groups to avoid the problems in the comic? Maybe they’ve done it so many times that those problems are all dealt with swiftly and easily!

Kevin Cheng wrote:

All very good points Jordan. I can’t speak for Tom’s points but I’d like to address one particular point you raised:

“Better yet, give us some examples of movies that would’ve made more money if not for focus group directed changes.”

I think focus groups are a GREAT way to cater to the lowest common denominator of mediocrity and hence, maximize potential for making money. The music industry of late has been nothing short of stellar at manufacturing formulaic hits. As much as “Jenny From the Block” is completely void of artistic or creative innovation, that tune is catchy and will have my head bopping in spite of myself.

The question is, can we use focus groups to generate creativity rather than just get the widest appeal?

Meri wrote:

I think that the point Tom is trying to make is more that possibly what people THINK they want is not necessarily the most creative, artistic or revolutionary way to do things. I mean have you ever tried showing a prototype to a bunch of potential users? They know EXACTLY what is wrong with it, how much they hate it, even some of WHY they hate it. But if you listen to how they think you should do it, then you should see what you end up with.

By all means use focus groups to fine tune, to sound out, possibly even to maximise profits … but using them for creative input is flawed, because as Elly suggests, design by committee is mediocre at best.

As for specific examples of when focus groups have meant the film turns out worse … do you think studios let that sort of information get out? It’s well known that they use focus groups, that they present alternative endings to different groups and take the one that they think will make them the most money. But, as Tom says, this settling for the safest option, the sapping of risk and creativity, results in safe, generic products that no one will remember past the release date.

Meri wrote:

*waves at KC* In synch..

Tom Chi wrote:

Woah. I took the weekend off, and now all these comments are floating about. Ok, to start:

“Better yet, give us some examples of movies that would’ve made more money if not for focus group directed changes.”

I guess my first point would be that maximizing profits is in the company’s interest, not necessarily in the viewer’s interest. When a critic reviews a film, he/she doesn’t say: ‘this movie is great because it made 200 million in domestic gross!’

Since most of us work for companies, it’s often easy to get the two confused - especially since occasionally products of high quality will also make gobs of money.

So to address your other question, here’s a short list of crappy movies: Men in Black II, Scary Movie III, the Real Cancun, Feardotcom, Freddy got fingered, Dumb and Dumberer, Virus, etc.

I think the existence of most of these movies indicates that the movie industry has not “done it so many times that those problems are all dealt with swiftly and easily!” Or maybe they have done it many times, and it’s just the wrong approach… dunno.

Kevin Cheng wrote:


Naming bad movies is a bit of a futile exercise isn’t it? Naming GOOD movies this past summer would be much more entertaining =)

Foolish Jordan wrote:

Okay, so you’ve listed seven bad movies. How would those seven movies have been better if only the movie studios hadn’t used focus groups?

TimH wrote:

Yesterday i saw an interview with John Cleese about his use of test-audiences for his films (Life of Brian, A fish called Wanda) and his ambition to be innovative and not to yield for mass-culture.
He said you shouldn’t let your audience tell you what to change in a film. You are creative, you are in control, you have responsability.
What you should do is watch them closely and record at what times they:
- laugh
- are bored
- are confused
- are offended.
He gave an example: Often people are bored in a particular scene because the scene that came before it is too long. A test audience will advice to throw out the particular scene but probably it is better to make the scene before it shorter.
Focus groups tend to feel expected to find a solution. But they should just signal the problems from a user’s point of view or from a designer’s. Solving is the designer’s responsability. And I think it is impossible to design in a meeting.

Kevin Cheng wrote:

“And I think it is impossible to design in a meeting.”

Bodker’s “Design Collaboratorium” would beg to differ:

(sorry, requires ACM-SIGCHI membership)

There’s a different version that’s not as in depth here:

Her proposal is a design process involving all parties in a contextual setting. I think the idea is great but incredibly idealistic and only truly applicable for very large scale projects with specialized user sets.

Tom Chi wrote:

Well here’s another way to look at it. You want me to describe how those movies would work better without focus groups. (Un)fortunately, since I didn’t attend those sessions, I can’t say what bad things went down when those bad movies were being baked. But I can point to a huge body of creative work which has somehow managed to be incredible and inspiring without the use of focus groups. So to your point, I at least have evidence to support my weaker statement (i.e. focus groups are not an essential part of a successful work).

My stronger statement is that not only are they not essential, but that they might actually work contrary to the success of a piece. I’ve definitely witnessed design by committee transform projects with great prospect into something less desirable, but these projects are not within our shared domain to discuss. So the best I can offer you is a list of bad movies with the assertion that focus group techniques were applied, but with inconsistent results.

If in the future I get to run a focus group for a movie that we both get to see, then we can discuss at length what went right or wrong, and having run the group I’d be able to tell you what could have been. Until then, I’ll just keep casting screenplays to the wind and reaching for the stars, or whatever.

I do want to say that I appreciate your efforts to ask “where’s the beef?” It’s good for the discussion. But I would also suggest that you also put forward a view. I would maintain, as KC noted in his entry, that focus groups can suffer from team dynamics issues which other forms of usability research do not. I would also agree with Frank that focus groups may play a useful role if applied earlier in the process (as opposed to the way they are used in the movie industry). If you have a different experience where they have proven to be effective then I’m sure that everyone is interested in learning from it.

Leave a Reply

OK/Cancel is a comic strip collaboration co-written and co-illustrated by Kevin Cheng and Tom Chi. Our subject matter focuses on interfaces, good and bad and the people behind the industry of building interfaces - usability specialists, interaction designers, human-computer interaction (HCI) experts, industrial designers, etc. (Who Links Here) ?