Kevin Cheng  

Usability Horror Stories

December 3rd, 2004 by Kevin Cheng :: see related comic

Remember that time your user actually broke down and cried in your usability test? Or the time your user got incredibly stressed out, sweating profusely and exclaimed, “I can’t do this!” Well we’re curious about your usability horror stories. What’s the strangest or most memorable user research you’ve done, be it a usability test, a user interview or anything similar? Share with us, because misery loves company.

29 Responses to “Usability Horror Stories”
Gabriel Mihalache wrote:

Links and buttons get clicked ONCE. ONLY icons are clicked twice to activate! I HATE it when people start double-clicking around like it’s supposed to be some sort of magic bullet! Beah!

Carolyn Snyder wrote:

I once had a woman show up with her lovely 2-month old infant in tow. She had somehow (sleep deprivation?) gotten the idea that the session would take only a few minutes. The baby was sleeping angelically. Against my better judgement, I decided to proceed with the usability test. Halfway through, baby awoke and needed the sorts of things that babies need. When a diaper change didn’t soothe her, Mama concluded “She must be hungry.” Trying to handle this with delicacy, I asked, “So… is this an easy thing or a difficult thing?” “Oh, it’s easy,” Mama brightly replied, putting baby to breast. In a conference room full of strangers. Thankfully, thankfully, the client was understanding. In retrospect (why didn’t I listen to myself??!) I should have told her that we couldn’t accomodate infants, paid her, and sent her home.

Carolyn Snyder wrote:

And now I’m going to top my last post… I once had a user get to a porn site by accident in a usability test. We were testing a reputable ecom site. Filling out the order form, the user entered her name and hit tab. Somehow, the cursor ended up in the URL field, where she typed the first 2 digits of the address, “19.” Noticing that the text wasn’t going in the right place, she clicked in the address field and continued filling out the form. A couple fields later, she hit Enter instead of tab, and off we went to I’m no prude, but I can’t begin to explain how disconcerting it is to see those kinds of images when you’re expecting an order confirmation page. It took us about 10 minutes to regain our composure. Fortunately, I was able to reconstruct what had happened, and the user accepted my explanation that this was not her fault, not the site’s fault… (and not my fault either, though strictly speaking she could have sued me). When she left, she said, “I never believed people when they said they got to porn sites by accident. Now I do!” Moral of the story: On the Internet, disaster is always only one click away.

Dustin Diaz wrote:

What I can’t stand is when I get feedback on a project that I’m not entirely finished with. Although I love to hear opinions and idea’s of what I can improve on, but only when I ask for it.

There’s nothing worse than having someone critique your work while you’re in the middle of it. Mainly, when someone tries to tell you how ugly the interface is when you’re in the middle of “programming” it cuz you’re like “It’s not done…and I’m not ‘designing it,’ I’m ‘developing it.’”

Anyway, the worst usability complaints I’ve received is “Why did you change the site?”

In most cases, the site needed a re-design, but I have also found that users don’t like frequent changes either.

Sorry but that’s the best horror story I got.

X wrote:

This is, alas, usability in action rather than design - so there’s your fair warning I may be horrifically off-topic.

Ever try helping someone fix an “easy” internet connectivity issue… over the phone? They are one’s hands, and eyes. There is a “common” task - pulling up the properties of an object achieved through the miracle of right-clicking on an object. Having helped more people through this task than one could feisably shake a stick at in this life, I have yet to come up with any way of describing the task of right-clicking fool-proofedlinessocity.

That’s right. “Pardon me, sir, but would you look at the thing you move around to move the pointer on screen? Yes, the mouse. You know how you always press on the LEFT side of it? This time, sir, I would like you to press on the other; right side.” DOES NOT WORK.

Ron Zeno wrote:

Remember that time your user actually broke down and cried in your usability test

If you have ever had a test participant cry during a test, please seek competent training in usability testing. You have been duped by incompetent trainers!

If you don’t have a protocol for handling test participants when they become upset, then get additional training. If you’ve never been given such protocols, then you trainers were grossly negligent.

donna maurer wrote:

My worst usability testing experiences involve participants who flirt or come on to you during the test. Yuck!

My second worst is where I have cleared a set of scenarios with the client, and the very first participant says “Oh, I don’t use this system to do that task. I always use this other one”

No crying, no babies and no porn sites thank goodness ;)

Ash Donaldson wrote:

Participants I can handle. Sometimes, however, the technology just wants to give in to ‘Murphy’s Law’.

Just a few weeks ago, I was performing a summative evaluation of a new iteration to be released. The testing was being performed in a different state, and we had to fly 2 other company people in to observe - so it wasn’t a cheap venture. Instructions had been given to the developers not to touch the test environment as it would be being used for usability evaluation that day.

You can imagine my surprise when, just as the second participant sat down to perform his first task, he was faced with a “Page Not Found” error. Someone had decided (in all their wisdom) to refresh the database pointing to the test environment. Technically, he said that it was not the test environment - it just supplied all the data for that environment.

Two hours (and 2 paid participants later), we had our ‘refreshed database’ up and running again.

Lesson learned: Be VERY explicit when dealing with developers.

dug wrote:

Firstly, I can’t agree more with Gabriel — it drives me nuts watching folk double-click their way around a web page *grrr*

My worst experience was probably a good thing in that I learnt to not take user literacy for granted.

My client allocated a budget to rewrite a system that stored forms (lists of ingredients, names addresses etc). The new system would replace cutting and pasting in Word and would let my client publish the forms online. Anyway, it was an all singing all dancing wonderful system that was going to make the administrator’s life a whole lot easier.

I designed the system from the ground up sitting alongside the user. Really, the whole thing was built around her. When we launched she completely rejected it — wouldn’t use it.

I tried to train her but she kept asking questions like “can I select the whole thing and drag it over there?” — she essentially didn’t have even most basic browser literacy. How did I not detect that after months of working with her?

I guess I grew up online thinking I could replace all Microsoft productivity apps by clever web-based applications. The experience kinda shook me up and know I’m a lot more understanding of users that need familiar environments and more wary of projects that propose to replace them…

Oh, and in the end the new system was shut down. She still cuts and pastes from Microsoft Word document templates;-)

Sarah Owen wrote:

My horror usability test participant was (surprise surprise) a grumpy older man, who I shall call Mr Grumpy.

I was working on a Document Management System project for a government department. Management had decided they would implement a system which did ‘everything you could possibly want to do with a document’ (through a web browser), and then lock down the file system to ’strongly encourage’ staff to use it.

I had the happy responsibility for designing a simple, easy-to-use UI that, among other things, collected 200 manually-entered fields of metadata for every document anyone created. I wish I was making this up.

I took my paper prototypes to testing, and most of the staff who participated were very nice and tried to be helpful, saving their comments about how much they didn’t want the system to the end.

Mr Grumpy, however, peppered the whole test with comments. He wanted the whole world to know about his righteous indignation, including me.

Firstly, he said he wouldn’t ever use the system, even if it was user-friendly. Secondly, he had some great strategies in place for making sure he never had to - he was stock-piling floppy disks to store all his work once the file system was blocked, and if that failed, he’d just bring in his laptop and use that instead. And lastly, he intended to share his plans with everyone in the department - management couldn’t just force new technology on him!

Staff opposition was one of the reasons the project was canned a few weeks later, so I guess Mr. Grumpy got his way.

Joel Ness wrote:

In regards to the “double-clicking” phenomena, I’ve seen that for 20 years and accept it as a natural response to learning a graphic interface. New Mac or Windows users soon learn that you can’t do much of anything in the Finder or on the Desktop by single-clicking. And, since most of their interactions relate to opening a file or an application, they can’t do what they need to do _without_ double-clicking. No one explains to them that double-clicking is a short-cut for a more complicated way of doing things that now one uses.

There’s too much mental overhead to try to remember when to double-click and when to single-click, and double-clicking everything has few negative consequences other than the occasional “Hey, why did this appear?” when the first click dismisses a dialog box and the second click happens to trigger a button on the window beneath.

If the first thing they need to do to be successful on a computer is learn to double-click, then they’re going to keep doing that unless the cost of doing so (unexpected results) outweighs the the cost of figuring out a more complex set of rules.

Moi wrote:

We used to take the pi.. mickey out of one of the testers who used the “random click” idea as one of his tests. Until he managed to crash the app. as a result. Never underestimate the power of random clicking!

Jonathan Simon wrote:

From my website…

I spent my first few weeks at my first job developing a prototype for a new trading system. This involved several discussions with business analysts and traders. Around my second week there, I met my first trader on a trading desk. At just four months out of music conservatory and working for a major Wall Street investment bank, I was more than a little overwhelmed and nervous.

A business analyst and I went over to the trading floor. There were more computer monitors and computers in a single room than I had ever seen. This was definitely a far cry from orchestra rehearsal! There was a flurry of activity: people yelling, phones ringing, phones slamming, and more yelling. I sat down with a bond trader in the midst of this cacophony to conduct my first design interview.

I asked the trader a few basic questions about his job and how the system might be designed to help him. This lasted about 1-2 minutes before he stopped me and saidÖ

ďLook, donít waste my time gathering input. I hate the systems you guys build. I donít want them, I donít like them, and certainly as hell donít want to pay for them. So the last thing Iím going to do is waste any more of my time talking to you now. Just build something and give it to me when youíre ready.Ē

Not exactly a usability test, but you get the idea.

Rebecca Turner wrote:

I was conducting a task-analysis session, a focus group type of thing where we had brought in two customers and 4 developers with the thought that “gosh, these developers never hear directly from customers what they want to do and how they want to do it. It would be so valuable for them to get to sit face to face and listen to the discussion.”

One of the developers, was not, shall we say, ready for public consumption. He needed to be there as the tech lead for the GUI development, but this was a man for whom bathing was optional, as were table manners and lining up the buttons correctly on his shirt. I just prayed that he would be presentable, and sat him on the other side of the room from the customers.

All went well during the two-day session, he asked excellent, insightful questions, and I had actually relaxed and thought to myself “This guy’s not all that bad. We should do more of these things with him around.” But then, in the interest of time and finishing up before the customers had to catch their flights, we decided to have a working lunch.

The developer decided to get take-out Chinese (the rest of us brought in sandwiches). After listening (and trying to ignore) the slurping sounds coming from his corner of the room as he consumed his noodles and mongolian beef, I then looked over and saw him tilting the entire styrofoam container to his lips as he DRANK the remaining juices and bits from the bottom of the container. We all just sat there in stunned silence as he wiped his chin on the back of his sleeve, and we carried on the best that we could for the remaining 2 hours of the session.

Incidentally, the developer thanked me for inviting him afterward, saying how enlightening the whole thing had been and that “I never get invited to these things!” Gee. Wonder why.

Jay Zipursky wrote:

This is not my horror story but I did have a usability tester relate what must have been someone else’s horror story (or at least a bad dream).

I asked the participant if he had been involved with any other usability tests. He said he’d done one before and said it was a bad experience. What I assumed were the tester and a developer couldn’t keep their mouths shut. They continuously made noises or comments whenever the tester did something “wrong” or unexpected. The tester said it was uncomfortable and I ensured he didn’t have a repeat experience.

Kevin D wrote:

I once went out to meet a participant who was having a smoke outside. When I approached him he was holding a “Self wrapped cigarette”. I should have cancelled when he showed me his portfolio of hidden images he has found in a number of software splash screens. I was young naive and desperate to get participants (you should notice the excuses liberally applied here). I was testing an 3rd or 4th iteration on a document management system UI, and part of the test was doing a card sort task. This guy (BTW he must have been around 70 or so) sorted the cards in the most idiosyncratic fashion. Such as “this pile is for all the items that have an “e” for the last letter. This happened in spite of several redirects to sort according to content and not spelling. Another pile was of items that reminded him of water. Afterwards, I realized that it mightn’t have been tobacco in that hand rolled ciggy.

It was late, so none of the team was watching, but I had no idea whether to keep the data or not. He appeared lucid and did not “seem” high, outside of his outlandish sorting methods. I finally kept the data since I had no additional “proof” that he was under the influence of the “Wacky Tobacky”. I labeled him an outlyer.

Dayan Golden wrote:

I was conducting one of my first full product reviews of Turbotax. The participant, an outspoken gentleman who bore a striking resemblance to Walter Matthau, had a very complicated tax return and his numerous complaints in this one problem area (the Sch E) seemed to be generating good feedback, so I let him struggle for FAR too long (big mistake!). He finally got so frustrated with the software and so irritated at being watched that he offered some choice words and stomped out of the usability session.

Dayan Golden wrote:

Prompted by the baby story…
I once had a woman bring her two young children to a usability test. The toddler spent the session pulling books off the bookshelf, and knocking over those little trinkets I had placed in the room to give it a “homey” feeling. The older child was tossing paper airplanes around the small room. And somehow, mom kept right on working.

I can’t really call this a horror story because we were laughing hysterically behind the one-way mirror. We figured this situation was probably as true to life as it gets.

dug wrote:

…I then looked over and saw him tilting the entire styrofoam container to his lips as he DRANK the remaining juices and bits from the bottom of the container…

This, of course, being the correct way to eat noodles…

Andy Lloyd wrote:

Having comissioned and conducted usability tests in a museum environment, none of the test-subject related “horrors” seem that bad (try testing a fingerprint scanner ID system with 7-year olds in the middle of a science centre…). The nightmares were more likely to come after the testing when we reported back to developers. I always found it uncomfortable explaning to someone that the fact NO-ONE understood what they were supposed to do is not because they were all stupid or “doing it wrong”, but…

Drew wrote:

I’m a little embarrassed to admit it, especially in front of Ron Zeno who might insult my admittedly sketchy education, but we went through a brief phase of experimenting with more “relaxed” usability testing styles to see what difference it made. The experiment was a success, technically, because we sure did see a difference.

See, in order to help recruit participants we started to spin the recruitment announcement as more of a “sneak peek” at new features for our existing customers, leading them to feel that they were to be catered to and not instructed to perform tasks. And we also made the mistake of believing people who claimed they worked side by side in their home office. Well then, we thought, why not bring them in and test them together?

So my colleague and I found ourselves in a room with a husband and wife who came down from the mountains (literally) to chat with us. They were obviously sharp people but they had their own way of doing things, only part of which looked anything like the way you and I use computers. They didn’t fight but they did bicker in that emotion-neutral, habitual way that long-married couples have and there was all sorts of backseat driving going on. They stayed for over three hours, digressing to complain about some other aspect of our software whenever the mood struck them and trying to get free tech support out of us designers while we were trying to keep the test going. The part we still talk about is where — and surely it wasn’t a total non sequitur, though I can’t remember the context — the husband pulled out an enormous hunting knife and mock-threatened to “fix” the computer with it.

On their way out they joked that they now knew where we worked and would come down next time they had a problem. We smiled, cold sweat beading on our foreheads, and promised ourselves to do more remote usability testing from then on.

Juan Casares wrote:

Not horror, but funny:
I was testing a kiosc in a cinema. With, well, actual customers. I would apporoach them and ask them to participate.
So, I went to a lady and explained, we were designing a kiosc and needed her help to improve it. She started asking all this questions about the kiosc. Finally, she seemed satisfied. She opened her purse and gave me $5, “to help”.

PS: Eventually she was a great user.

Moi wrote:

> I then looked over and saw him tilting the entire styrofoam container to his lips as he DRANK the remaining juices

Oh come on Rebecca, have you ever tried drinking with chop sticks?

Ron Zeno wrote:

I’m a little embarrassed to admit it, especially in front of Ron Zeno who might insult my admittedly sketchy education

Sorry to have offended you, Drew, or anyone else that’s been a victim of poor training. It’s hard to point out that people are being trained by incompetents, without offending a few who didn’t realize it at the time. I’d hoped you would be angry at the trainer rather than the messenger. Nevertheless, my post was made out of shock and outrage, rather than from the perspective of cooly trying to find the best way to point out that people have been duped.

Anusha wrote:

This one is more like great experience with a participant!
One of my participants introduced herself as the avon lady and said, “here is a piece gratuity for you - for a change” and handed me the skin - so - soft moisturizing lotion!

Drew wrote:

We’re drifting off-topic, Ron, but I’ll clarify my feelings about what you wrote. I learned how to perform usability tests on the job; at the time I was working as a designer, and I worked with usability test specialists whose job it was to administer usability tests. The skills I use now to test my designs in work situations where the money is not available to hire a professional test specialist (but where the designs still need to be tested!) were all learned in that way. So I wasn’t really “trained” in usability testing but have learned what I know through observation and reading and experience, and while I realize you haven’t met them I’m not inclined to agree with you that the test specialists I used to work with were “incompetents.” That’s all.

Ron Zeno wrote:

Drew - My concern and outrage is with the competency of trainers, not testers. If a trainer, no matter the circumstances, cannot give you some basic guidelines for interacting with test participants that will allow you to prevent test participants from becoming so upset that they cry, then those trainers are not competent to be training anyone.

Sorry that you, and anyone else, is upset at where I draw a line on competency in training usability testing.

Caroline Jarrett wrote:

I was mentoring someone who was running her first usabiity test as facilitator. The participant was a nice old lady who hadn’t much experience with computers or the net.

The product wasn’t much good and the participant wasn’t having much success, but she’d got a good rapport with the facilitator and was chatting away happily despite her frustration with the product. Then it got worse and she said “At this point I’d run screaming from the room”.

Fortunately the learner facilitator kept her cool, and responded to the participant’s body language and generally relaxed demeanour. She just wrote down the comment and waited.

Turned out the participant was perfectly OK with that and continued to work away despite the problems.

Facilitator needed to be revived afterwards with a nice cup of tea and a sit down.

Anonymous wrote:

Okay, I just *have* to comment on the ’stupid double clicking’ on links in web browsers. Now I -know- it is only supposed to take a single click to cause a hyperlink to be correctly followed and most of the time it does work that way. However, there are times when I have noticed that clicking a link once results in waiting *ages* for the page to load, or actually results in a timeout error. BUT if the same link was double clicked, everything was nice and snappy. Now this bugged me, cause it only happened once in a while, but I eventually just gave up trying to figure it out and began double clicking all links in the browser. It never hurt (and don’t worry, I know the difference bewteen a hyperlink and a javascript rendered ’submit’ button ;) and sometimes it avoided the external wait. Now I must admit that this was when I was still oblivious to the amazing developments called “Opera” and the almost as good “FireFox” so I was using Moron$ofts ‘Internet Exploder’ @ the time. One day, someone saw me double click a link and asked why I did so, explaining that they thought only a single click was necessary. I answered that I knew it only required a single click ‘but seems to sometimes work faster/better’ so ‘thats just what I do’. But that got to bugging me as I thought more on it, WHY should it be faster??!! So I began searching for an answer, and I *think* I finally found one. [] We can see that in that article Moron$oft has used a nonstandard request convention in some versions of its product, so its my belief that the reason the double click process is sometimes ‘faster’ is that it somehow aborts that foolish improper request sequence. I also think that other users have like me just noticed that it ‘wroked better’ that way and so without questioning why just began to double click all the time. Unfortunately most of them lack the sense to realize that what works with hyperlinks it totally different than what should be done in applications with buttons, etc. and therefore cause endless trouble with their needless double clicking. So, it is a learned habit, that used in the context it was learned in actually does work better. Lets all blame Moron$oft :P Actually, my pet annoyance with users is those that turn on the ’sinlge click to open’ option in windows explorer. IF I want to open, I’ll double click. IF I want to select, I’ll single click. This whole ’select on hover’ thing Moron$oft put in MAY save time in the long run (as fewer click are needed) but I have only experienced frustration when using it because of it interrupting other mouse activity in ways that have had unexpected and often negative results.

Leave a Reply

OK/Cancel is a comic strip collaboration co-written and co-illustrated by Kevin Cheng and Tom Chi. Our subject matter focuses on interfaces, good and bad and the people behind the industry of building interfaces - usability specialists, interaction designers, human-computer interaction (HCI) experts, industrial designers, etc. (Who Links Here) ?