Kevin Cheng  

Losing Focus

October 31st, 2003 by Kevin Cheng :: see related comic

Focus groups are fascinating tools. Everybody seems to have a use for them: marketing, usability, visual designers, business analysts, etc. In theory, they bring together a group of potential users and create a dynamic environment from which one could solicit requirements, work context, user preferences and maybe even the answer to religion.

In practice, focus groups tend to be less than ideal. Observe our little focus group in what could be described as figure 1 but is better described as “this week’s comic”:

Panel 1: Ivan asks if everyone likes blue as much as he does. A lawyer would object to a line of questioning posed that way for, “leading the witness”. Not only is he leading them with a stated colour, he has injected his personal opinion into the question. Goth girl then demonstrates another classic problem in focus groups. Strongly opinionated and vocal members of a focus group tend to force a “squeaky wheel” phenomenon where the squeaky wheel inevitably gets the oil whilst the others do not until they squeak.

Panel 2: Our two other members contribute their insights into the colour blue but not in the context of the product. Often, focus groups can stray into unknown territory, travelling along tangents so unexpected you feel like you’re about to discover alien life. Sometimes, the same phenomenon can be beneficial to the discussion, like a brainstorm, but it takes a trained eye to recognize these rare cases. Goth girl again helps us demonstrate a problem with focus groups: group dynamics. If you don’t already have a group that has a rapport and comfort level with each other, there can often be friction that leads to unproductive discussions reminiscent of a dozen men in a jury. Note also her absolute certainty in her preference towards black. Or red.

Panel 3: A handful of other participants are revealed. Sometimes, the dominant speaker is not an individual but a subgroup. Where were these 5 people during the other conversations? Perhaps asleep, daydreaming or just completely indifferent to the entire process. After all, they’re getting $50 (or pounds, or euros) for their “efforts”. Finally, the problem of groupthink can start to rear its ugly head. Participants are no longer contributing as individuals but as one collective group, effectively eliminating the benefits of dynamic interaction in focus groups.

Many of the problems described can be mitigated to an extent by having a trained moderator. Focus groups can be tempting because you seemingly accomplish a lot of data collection from a lot of users in a short period of time but incorrect data can be worse than no data. Detecting incorrect data is difficult to do. If you knew what data was incorrect already, you�re unlikely to need a focus group at all.

Does anyone have any stories to share about focus groups, positive or negative? I’m sure there are plenty more comic strips that could come of such anecdotes.

9 Responses to “Losing Focus”
Paul Reinheimer wrote:

I often find, when lacking a trained moderator, speaking to the focus group members both pre and post group dicussions can help a lot. Introducing the member to the topic on an individual level will often get even the quietest member to voice some opinion, and equally, members who felt silenced due to a strong vocal person/sub group will speak out as to what they wern’t saying before during an individual post group meet.

Using notes taken during pre group meetings can also help stear the group back in the desired direction. In the comic above one could use “Several of you mentioned before the meeting a preference for a full colour line, or a product that could be easily painted or skinned. Do you think that is an important design conisderation, or is colour really not the most important issue here?” (or words to that effect).

Kevin Cheng wrote:

Pre and Post questionnaires are a great idea. If you’re spending the time doing interviews on a one-on-one basis, what value does the focus group add that regular user interviews don’t already yield?

My feeling is that they don’t add much. The dynamic interaction is negated by the risk of gathering inaccurate data.

Meri wrote:

Not much to do with anything, but is Ivan wearing an OK/Cancel shirt with a cartoon on the back there?

:-)

Jon wrote:

Thanks for the cameo :)

Perhaps I can be a spinoff.

Arthur Law wrote:

This is almost the exact situation I had last week in my course on community weblogs. One of the professors is from Wired and he is quite adamant against “Blogger Blue” for the class blog. A political line was drawn for the colour Red from another prof. That colour is too nationalistic towards China and we want to be politically neutral (or left). Black doesn’t convey the concept of the internet propertly, et cetera…
Earth tones here we come.

Paul Reinheimer wrote:

Authur Law: You are far more patient than I.

Somewhere into that argument I would have given up and left them with a black text, white background, no style page, or one with a randomly selected number between 000000 and 999999 and just ignored the higher hex values.

Kevin Cheng wrote:

Once, I worked on a portal site where I had colour coded each site’s major section to reinforce navigational feedback. Basically, a certain part of the header’s background would change colours depending on which tab you had selected.

One of the business sponsors of the project took up hours of my time nearing the deadline of the project just changing the colours because he didn’t like peach. I was trying to make it more interesting than IBM blue and gray but eventually, we ended up closer to those corporate style colours. It’s almost like the converse of the focus group: when a single person who isn’t a user vetoes designs because of his individual bias.

(I feel like there’s a strip in that tale somewhere)

Wundt wrote:

Many of these issues can be averted by screening the participants. If the product is meant for corporate types, it is a simple matter to not include, or to discount feedback from, non-target users (like the goth chick above).

What is worse is when you have discussions like this in the design review, and it is a couple of managers (any of whom could get you fired) going the rounds about which shade of blue to use or whether to use the word “The” or “a” in the label.

Paul Reinheimer wrote:

Its funny you mention that Wundt.
I spent four hours in a meeting with my boss, his boss, and his boss.
We went through every page of a dynamic web application, and argued over the wording. This is of course version 3 of the app (version 1 designed by my boss, version 2 by his boss…)

After all of that, It is changing again. Other people were asked for input, and the application must be changed to reflect it.

I think ultimatly, for a focus group to have any purpose, someone with final authority must be willing to sponser the group. Otherwise its just people sitting around and chatting.


Leave a Reply


OK/Cancel is a comic strip collaboration co-written and co-illustrated by Kevin Cheng and Tom Chi. Our subject matter focuses on interfaces, good and bad and the people behind the industry of building interfaces - usability specialists, interaction designers, human-computer interaction (HCI) experts, industrial designers, etc. (Who Links Here) ?