Remember that time your user actually broke down and cried in your usability test? Or the time your user got incredibly stressed out, sweating profusely and exclaimed, “I can’t do this!” Well we’re curious about your usability horror stories. What’s the strangest or most memorable user research you’ve done, be it a usability test, a user interview or anything similar? Share with us, because misery loves company.
While usability tests project an air of science and procedure, in practice they typically boil down to the interaction of *two* people and a machine. While the usability specialist does his/her best to minimize their effect on the tested, there is always something of a “Heisenberg usability effect.” Simply put, an observed user behaves differently than an unobserved one.
I don’t know of studies that have quantified this effect, but from testing experience I know that it can shift user behavior quite radically. What’s worse is that it doesn’t shift in a predictable direction — some users become anxious and fail where they normally wouldn’t, while others stay calm longer to secure their ‘usability gift pack’.
Despite our best efforts, social relationships and power structures always exist. A 12 year-old user being tested by an adult researcher will behave differently than they would visiting the same site with their friends. A particularly attractive user might be treated differently than a crumudgeon. As professionals we try to reduce these variances, but we can’t control for a participant who behaves differently around a male usability specialist vs. a female one. Our usual answer to this has been to leave the room entirely. So the user ends up sitting alone in a small room with a wizard-of-Oz-like voice telling them to turn to the next task. This does remove some of the visual/social distractions, it is still a strange, strange environment.