Kevin Cheng  

Making Senses of Design

June 25th, 2004 by Kevin Cheng :: see related comic

Multitasking. The word is applied in many different contexts: How Windows does or does not multitask; How drivers multitask by talking on the phone or changing the radio station; How a person talks on the phone while doodling on a piece of paper. Perhaps it’s the word task that throws us off but we don’t commonly think about some basic multitasking done every day, at a more micro level. For example, holding a conversation.When you listen to somebody talking in person, you are processing multiple streams of information. You listen to their message, of course, but you also look at their body language - hand gestures, facial expressions, etc. You may even be touching the person. All of these are inputs, through various modes, and they’re being processed effortlessly. Essentially, you are multitasking when you have a conversation.

Conversely, consider your modern day work environment. You may be surfing a website (as you are now) and somebody messages you on your favourite instant messenger. You open the message, respond quickly, and return to your reading. That’s not multitasking, that’s task switching. Maybe you have some music playing in the background - are you multitasking then?

The ability to multitask comes down to the ability to receive, process and subsequently respond to multiple input streams. To illustrate this, I’ve modified a two dimensional version of Wickens (1984) model of cognitive dimensions.

Modified Wickens Model

What the diagram illustrates is that we can see words, see pictures, hear sounds and hear words all at the same time. As we move to the cognition level, we have difficulty processing from two modalities but we can process both verbal and spatial information. Thus, I can read text and hear music, or read text and pictures but I can’t hear someone talking and read a book at the same time. Finally, we can respond with verbal and spatial information (talking with gestures) but we cannot do so across modalities (writing and saying something at the same time). The exception, of course, is when the streams of data are matching - that is, I can write and say what I’m writing out loud.

As with most things human related, training can permit us to go beyond this model and some professions, such as radio operators, do just that. The diagram is not sufficient to illustrate all aspects of human information processing. Other factors such as attention and human error are prevalent and are also covered in Wickens’s book.

Understanding the nuances of this model could greatly improve some of our designs, whether they are in the computer domain or not. While we’re not really capable of holding simultaneous conversations, training a dog in yoga and playing the guitar simultaneously, we do manage a great deal.

Designs can utilize this model to create meaningful interfaces that attract the user’s attention only when truly necessary. The Sonification of Web Servers, presented in WWW9, is a geiger counter like system which indicates the status of a web server to a server administrator. This system is a great example of utilising our ability to process sounds while we work (thanks to Matt May for that pointer).

Equally, designs can recognize what types of information enforce a message rather than distract from it by using a complementary set of spatial and verbal information.

We can use the model to discover potential problems. A common multitasking topic is driving and cell phone usage. A lot of research has been conducted in this area using various methodologies. The Wickens model can also be applied to help understand the problem.

The task of driving is primarily a visual task. Drivers visually perceive the information around them and process the spatial (roads, signals, traffic) and verbal (signage) information. Observation of traffic signals, of road conditions, of other cars around you and of the status of the driver’s vehicle all support this premise. They respond to these through gestures (adjustments to the vehicle). Occasionally, additional, non-visual, feedback is processed such as a car horn or siren. However, these are discrete, symbolic acoustics which are meant to deliberately draw attention from the driver and demands attention.

The task of making or answering a phone call is also primarily visual. Looking up phone numbers, dialling numbers, checking called identification, etc., are all potentially visual tasks (alternatives to visual methods of placing or answering calls exist). In order to perform these operations, the driver must momentarily stop observing the driving environment because they are unable to process two different streams of visual information simultaneously. Thus, the driver runs an increased risk of causing an accident.

Additional factors exist in driving. My analysis and application of the model is quite simplistic. Reaction times to emergency situations are not considered in the model. Also, driving is a trained task so we do have some ability to go beyond the basic Wickens model (such as having a conversation whilst driving). When observing the effect of secondary tasks on the primary task ofr driving, it has been found that novice drivers allow the secondary task to affect the primary task more than experienced drivers (Lansdown, T. C., 2002).

Our current environment is filled with distractions, both on the virtual desktop and in the office environment itself. We need to combine the knowledge of what our limitations (and our strengths) are with an understanding of the context within which our target users work. With this combination, we can take more care, and hopefully be more innovative, in our use of aural and visual feedback.

[27 Jun 04: Added paragraphs 9-12 on applying the model to driving]

13 Responses to “Making Senses of Design”
Chris wrote:

Interesting…Goes to prove that you can do two things at once.

Ben wrote:

Very interesting! I’m no HCI expert, but I can see definite applications of this model to designing better vehicle cockpits / driving seats / dashboards. We seem to have seen such attempts as beeps (seat belts, low fuel), voice (navigation, door ajar), as well as visual for those same things (and spedometer and such).

But — at least personally — I find that certain information is more easily conveyed (or interpreted by my peanut brain) in one form over the others (speed) while others can be presented in either way (route navigation). And yet, driving is a predominantly visual task already; then we turn on the music to flood out the auditory perception?

What’s left for us to be able to concentrate on new incoming information? I wonder how designers would prioritize the methods by which certain data is presented?

Kevin Cheng wrote:

Ben, I’ve added a few paragraphs on applying the model to the most (in)famous example of driving.

You’ve pretty much nailed it though. I’ve read numerous papers on the subject and no matter the method, the result always seems to point to the fact that our cognitive load is close to capacity when driving in emergency situations. The problem is that we are so rarely in such a situation that we risk doing other activities whilst driving (changing the radio station, talking, even shaving or reading the newspaper).

Makes you wonder what the iDrive people were thinking though.

Hania Kutcher wrote:

What about the sense of touch? I scuba dive; underwater there’s little aural, verbal, and sometimes even visual (night diving, golf ball retrieval) input. But you definitely rely on touch, e.g. your skin for water temperature (water absorbs hear 25 times faster than air), water current/speed; your ears for pressure/depth; your body for movement, direction. And remember that water acts like a lense distorting objects to make them appear 25% closer and 33% bigger. You can’t trust your eyes. If this example seems too far out, remember that NASA uses the underwater environment to simulate space (a place that definitely has a lot of computers). And believe it or not, there are underwater computers in use, e.g. WetPC — :-)

Hania Kutcher wrote:

Forgot to mention that scuba diving is definitely multi-tasking: 3 axis of movement, lots of visual input (in the day) — fish and stuff, monitoring your guages — depth, time, direction/compass, air (don’t want to run out of air!). Not only do you have to control your body, but also your equipment, your environment (don’t piss off the big critters), and your emotions (excitement, fear, sometimes panic — e.g. someone turned off your air and you’re dropping like a stone). Talk about multitasking. Throw in underwater camera equipment (video, lighting, delicate expensive widgets) and the knowledge that you should monitor your group (should surface at the same time, stay together so you don’t get lost, watch if someone needs help) and you’re really multi-tasking.

Kevin Cheng wrote:

I am actually advanced PADI certified so I can relate. Obviously, we have the senses of touch and smell which Wickens doesn’t cover. In addition, there’s been talk of a general awareness that people have which you may even dub a sixth sense.

I haven’t seen what research has been done in relation to any of these so I can only hypothesize. First off, scuba diving is a trained activity. At least, it’s supposed to be one. Such that regulating your buoyancy and ear pressure are second nature and monitoring your air and depth beoome as natural as checking your speed and gas whilst driving. Given the reliance on hand signals underwater, diving is almost entirely visual and all correlate to the same task. It’s like reading the speedometer and checking what lane you’re in and shoulder checking.

Touch is used to become more aware of your surrounding but much like the use of audio alarms, you only really notice input when it varies significantly. e.g., when you pass under the thermocline or your leg gets tangled in some weed.

Talking about this is making me miss diving now …

Bob Salmon wrote:

you only really notice input when it varies significantly.

I don’t know how true it is, but I’d heard that if you sit completely still while listening to a constant pure sine wave, eventually you stop hearing it as it’s conveying no new information. And I remember that when the more extreme dance music started getting mainstream success, one ‘musician’ said that in the future dance music would be just a police siren. Could the police go after the musicians with the DMCA then?

Multi-tasking? Just ask a parent. I think it’s a skill injected by evolution, for the survival of the species, and as a bloke I’ve found it a learned behaviour. Certainly the discussion of emergencies is relevant here - multi-tasking as a parent is periods of flitting about keeping many plates spinning, interspersed with dropping everything (sometimes literally) to concentrate on one. The ability to know the status and relative importance of things is crucial, so you know how to divide your attention.

MeriBlog : Meri Williams' Weblog wrote:

Multitasking Mice Fertilitiy

Interesting article over at OK/Cancel all about multitasking PHP, XML, and Character Encodings: a tale of sadness, rage, and (data-)loss — worth reading just for the title ;-) Very comprehensive guidebook to developing with web standards I love the…

Arthur Law wrote:

Just for interest’s sake, here’s the quote about 10% of the brain being used.

“Thought processes are disturbed not only by these engramic commands but also by the fact that the reactive mind reduces, by regenerating unconsciousness, the actual ability to think. Few people possess, because of this, more than 10% of their potential awareness.”

L. Ron Hubbard
Dianetics: Modern Science of Mental Health
Church of Scientology

Bob Salmon wrote:

Things like Microsoft Research’s Sideshow fit here - the ability to monitor many things at once and then concentrate on just what’s important.

Rachel B wrote:

I think it’s worth nothing that the ability to multitask varies from person to person — more likely personality-type to personality-type — and decreases with age. Hence, the need to, say, turn off the car radio while parallel parking, once you’ve reach middle age.

A while back, I also read about multitasking ability varying according to gender, though I can’t find the source right now. The example used was a tour bus operator, where the women could drive and talk, whereas the men had trouble storytelling while driving. Watch young moms as they hold conversations while making coffee while keeping their toddlers in their peripheral vision. I wonder if there’s research to support the anecdotal evidence.

Bob Salmon wrote:

Watch young moms as they hold conversations while making coffee while keeping their toddlers in their peripheral vision.

I’d like to strike a blow for equality here - some of us dads try to do this too! (I know we’re probably not as good at it as the mums, but at least some of us try!) This morning before work I was changing a nappy (diaper), while fending off another child wielding a hardback book, while having a conversation with my wife in another room. (Yes, I come to work for a rest.)

Carla Ciurlizza - Comentarios wrote:


Gracias a CesarS en Backdraft caí en OK/Cancel, un lugar muy interesante, donde me encontré con el artículo Interruptive Technology and the Death of Deep Thought. Parece que no soy la única que piensa que la tecnología mientras más nos…

Leave a Reply

OK/Cancel is a comic strip collaboration co-written and co-illustrated by Kevin Cheng and Tom Chi. Our subject matter focuses on interfaces, good and bad and the people behind the industry of building interfaces - usability specialists, interaction designers, human-computer interaction (HCI) experts, industrial designers, etc. (Who Links Here) ?