As computers have moved into every corner of modern life, our discipline has grown in turn. Yet long before the computer, designers engineers and craftsmen have been coming up with and honing the tools that have shaped our world. A curious and unfortunate change has happened with the arrival of the computer, however. We have largely scaled back our efforts to change our tools and simply accepted a vast array of unnecessary limitations.
Most tools throughout history have had too important avenues for innovation. One was the form of the object itself, and the other was the method by which the object was used. These roughly equate to hardware (form) and software (method of use). In the last 20 years our focus has narrowed to be primarily about software. Rarely is interface hardware brought into question in the design of computers as tools. Thus we are trapped in a world where we specify how mice move around windows that we manipulate on a screen. We’ve attained a great amount of depth in this subject while a whole host of other input and interaction modes remain in their infancy.
To give you a taste of my line off thinking, take the following example. What if a pair of glasses was created that had a small camera embedded in the frame. In addition, a small motion sensor was placed in a position to detect whether the wearer has blinked. The entire rig is powered by embedded lithium batteries, and a port is provided for data transfer and recharging purposes.
Given this set-up (the components of which are pretty inexpensive), one could devise the following method of use — when the wearer sees something they would like to capture on camera, they simply double-blink. That’s it. Now why is this better than cell-phone cameras? Well the main difference is that you don’t need to dig a device out of your pocket, set it to camera mode, bring it to your eye, place the subject in frame, and press the button. The latter sequence is guaranteed to take your attention away from what you are trying to capture, while the former interaction could soon become instinct.
Such a project is comparatively radical to what mobile device designers are talking about (right now the industry is focused on how things look on that small screen), but truly, design ideas like this are more in line with how tools were developed all throughout the ages. The ability to alter both the form and the method of use were basic components of innovation.
Nowadays because of the advanced training it requires to design develop consumer electronic hardware, we’ve closed off amazing possibilities. Still, I have hope that as powerful embeddable processors, improved pluggable sensors and actuators, and simplified hardware development platforms come online, that this will start to change. As things are currently progressing, this decade will still belong to software, but perhaps the next will herald a significant return to challenging the form of our digital tools.
Yes, interface design has almost entirely devolved to “how can I move the mouse thru menus” design.
But how do you end up thinking this leads us to insane ideas like blink interfaces? How about starting simple and solving needless problems? I used to be able to press a button to get something done. Now, its all contextual menus. On cameras, and printers and other things that don’t have that many items. My favorite, a stopwatch program on a mobile phone: Right softkey is start/stop. Left softkey brings up a MENU WITH TWO ITEMS. Stopwatches have worked for many, many decades with two buttons, but not anymore.
It seems a lot of the physical interface innovation is in simplifying interfaces. “Many buttons” seems to mean “complex interface”, and people then assume the inverse is true also: “few buttons” means “simple interface”. Even worse, if “few buttons” is good, then “fewer buttons” is better.
For a while, I had a pager that was billed as “easy to use” because of its one button interface. It was a terrible interface because three functions were selectable based on how long you held down the button. So it went from having to know what each button is for (and with a physical button, it can be labeled for visual and tactile feedback) to knowing how long you need to hold the button down and waiting for the display to cycle through all the options. The “problem of complexity of multiple buttons” was a solution in search of a problem.
Too many buttons or not enough buttons to push. We need (radical) input device that adapts itself based on the context of the application being used. I’m thinking something along the lines of the fingerworkds keyboard (no longer being sold) that merges mouse and keyboard together. I believe this device failed not because of the idea, but because nobody likes to type on a flat surface (need feedback from the keys). If there’s a way to get keyboard feedback from somehting like the fingerworks keyboard, then that would be a good starting place (next step) for input innovation.
For background- the fingerworks keyboard allow custom hand gestures and the like for specific applications. So based on the context of the application, you could perform different tasks. I admit, they appeared to be a lot of them, I’m not sure how hard they would be to learn/execute. Maybe other people who own them could chime in.
I think when the nano industry combines with the omputer industry plus the new internet that will be built by then it will make anything possible online and allow for a a more vurtual world.
There’s a lot to be said for having multiple physical buttons in order to avoid modes. In general, UIs are better when a simple clearly-defined user action results in a specific system reaction, a property likely lacking from a blink-triggered camera, unless the user is Ford Prefect.
Reminds me of the amusing story of the “nouse.”
Well, I don’t know if I’d want to take out a pair of glasses, turn them on, and double-blink to take a picture of something. (Not all of us wear glasses…?) But, I see your point.
Perhaps hardware innovation has moved into the industrial design realm, which tends to seem at odds with UI people - that is usability and industrial design/hardware innovation are mutually exclusive.
Personally, I’d like to move into device/hardware design, while still keeping my usability focus. I just don’t want to spend another x-teen years in school. Do you (or anyone) have any suggestions on schools/programs focusing on hardware/device design for people who did NOT come from an electronic or mechanical engineering background?
It seems even without considering usability, a electronic product is often first divided into hardware and software teams. Or worse, the hardware and software are done by completely different companies. I think this is part of the problem.
I’ve seen several situations where the software interface designers never even thought to inquire about changing the hardware’s user-interface components (even when they worked for the same company). Have hardware UI folks had similar experiences?
I was encouraged to see Smart Design’s demos of how they prototype UIs that have tightly knit hardware and software components (as the DIS 2005 studio tours).
Perhaps one solution might be to form teams around the product, rather than around expertise?
Tom asks: “Where did hardware innovation go?”
Robby asks: “When will software innovation start again?”
The amount of money required to develop, say, the proposed double-blinking-glasses into a commerical product is probably six to seven figures, but a couple of kids can make a really cool software product over summer break for the cost of soda.
But with software so much easier to make than hardware, how come we’re not advancing? Right now I’m sitting in front of machine that can compute a billion operations a second—and I’m typing into a constrained little text box in a fixed-pitch font with no spell check. If I happen to click just one button on the mouse I will lose my work! That’s just pathetic.
Actually the look and blink interface exists, at least it did some years ago. I remember seeing it on TV (no, not a Sci-Fi show as being a research effort at some British University. I guess it failed to get to market though. Admittedly, the objective was a way of interacting with a desktop computer, but the basic idea is the same as your suggestion.
What about virtual keyboards, don’t they count? Projecting a keyboard onto a surface that you can then type on, and have the projector/sensor devices translate your movements to key presses, sounds like an advance to me.
Now if you’re talking paradigm shifts, like from card punch to keyboard and from keyboard to WIMP, then that kind of thing comes along much more rarely (perhaps once every 20 or 30 years), IMO. I guess one is due.
This hardware innnovation conversation reminds me of the comparison of the computing industry to the automobile industry and Robby makes some good points about costs.
I like hardware innovation Apple style — which arguably is incremental, and not a sea change that you’re asking for. Notice how Apple didn’t release a multi-button mouse until they could do it right (not a cumbersome scroll wheel, but a highly responsive knob). Also, the recent magsafe feature which protects your laptop from being knocked off a desk by a careless trip on the cord. (But, I’ll wait to be wowed when power is wireless.)
As for software innovation, Robby, if you were using a Mac you could accidentally load another page, and if you came back your field info would be intact. Oh, and if you were on a Mac you’d have spell check right in that field too. Some people are doing software innovation. They’re just not all in Redmond. =)
Remember the old IBM model 42H1292 keyboards? nice click, good feedback of the keys…. compared to todays crap (which, of course, is cheaper), all gone . Is hardware stepping back?
O.K. - maybe not always. An example are the powerful graphics chips in todays computers. Hardware innovation itself is nothing without going hand in hand with software innovation. Only the use of these graphics capability by software makes the whole innovation on todays (ok, windows vista is not today ggg) desktops.
For my opinion a good start in interaction revolution is the ARCHY project from Jef Raskin. A new way of thinking about interaction, combined with a suitable device. For me this project showed how hard it is for all of us to “think innovation” as we are every day biased by the crap we use day by day.
In my opinion, part of the problem with innovate hardware designs is the restriction of the human body. Not so much the physical limitations of the body but the physical limitations of someone’s body.
To use your camera-in-the-glasses example, not everyone wears glasses (as Allison pointed out), but even less obvious are people who have trouble controlling when they blink. My cousin blinks whenever anything gets anywhere near her eyes or even threatens to. By pointing at her face I could force her to take roughly four pictures a second.
This is, of course, a problem limited to one particular person. But something like a camera that can be held, even with its horrible hardware interface, can be even be used by someone with extreme physical limitations. I just recently watched someone with Cerebral Palsy use a Palm Pilot. Granted, he needed some help, and he could not do all that much with it, but the interface of the Palm Pilot allowed him to do it with a simple, easy hardware implementation that turned nearly everything into a software issue.
Hardware (or poorly designed hardware?) tends to impose on the physical body while software tends to impose on the mind. Someone with Cerebral Palsy will likely never be able to drive a car due to the hardware interface, not because they are unable to understand how to drive.
I do not know anything about design, so I would not be surprised if these sorts of things are addressed when designing hardware, but it seems that there are inherent limitations that come with hardware and reducing them gives more people the ability to learn how to use the product.
Bam! How’s this for hardware innovation by guess who? That’s right Apple.
Ahem, and this one you’ve probably seen… Combination hardware and software innovation. http://mrl.nyu.edu/~jhan/ftirtouch/
The future is Perceptual User Interface and MultiModal Interaction (non-command based)… in other words: Eye-Tracking plus Voice and Gesture recognition.
The products are ready. The probelm are Market and Market Leaders (like IBM - Acer - FUjitso - etc. etc.)
That’s an interesting topic. There are several simple reasons why hardware innovation is not as rapid as software and Robby Slaughter and MrHen answered the question. It’s the price and limitations of what you can do with and for human body.
Innovations in a physical world can go only as far as it makes sense for physics of human body. There is no need to make higher resolution images in print industry. Human eyes can’t see far beyond 1200 dpi. No need to make more frames per second in movies and so on.
As for interaction with physical world the best tools that people have are their hands. People used them for millions of years and they are most precise manipulators in human body. So it probably makes sense to leave hands as a main tool for interaction with software products. And any complex products where precision is a key factor.
The question is whether the mouse is the perfect tool for that. I’m sure it’s not. I believe that the future is touch screens. There is nothing more intuitive than touching something and seeing direct response.
I touched this theme before on my blog http://andreysmagin.com/blog/touchscreensrevolution And believe that the world will see multiple variations of touch screens.
You already can have great tools like Wacom’s Cintiq http://www.wacom.com/lcdtablets/index.cfm, which give you much more precision than any mouse and are very easy to use and very responsive.
Those tools are not cheap but with time prices will go down and we will see soft touch screens that give not optical but mechanical feedback we will see 3d touch screens and so on.
Personally I don’t use mouse. I use Wacom’s tablet not Cintiq though
Argh! Why does everyone pick holes in this one example? - it’s supposed to be an EXAMPLE!
To satisfy everyone’s problems with it; change the action required to be a small key fob wirelessly linked that you just click; or a ring that you tap twice, or a touch sensitive pad on the side of the glasses; and if you don’t want to wear glasses have it in a headband; or combine the camera with your bluetooth headset.
For crying out loud people! Address the issues raised - don’t nitpick a blue sky idea that is used solely to illustrate the problem!
I love the idea with the glasses… I’d pay a lot of money for those right now.
A little bit of proofreading:
>Most tools throughout history have had too important avenues for innovation.
“too” should be “two”
To give you a taste of my line off thinking, take the following example.
“off” should be “of”
James, I see your point, but….
I don’t think that anyone here was suggesting that an easier way to take a picture without a true camera, (which is what a camera-phone is), isn’t a good idea. Thinking of an innovative way to take a picture with some common object, like a phone or even a watch or pencil, is the gist of the argument, right?
Being nit-picky is how hardware innovation (or any innovation) works in order for designers to understand the problem. We are dealing with physical human limitations and with current software/hardware limitations, both of which strongly depend on development time and money, and market requirements. If I were to take an educated guess, for the real world, money, market and time are the biggest contributors to innovation of any kind.
FYI: I think DARPA still does hardware innovation…?
Seriously, that glasses idea is terrible. Double blinking - sheesh!
I think hardware innovation is happening all around us, it’s just not big jumps. There’s small incrememental improvements.
Look at the Sony EyeToy, and the new Nintendo Wii. Then look at all the nifty ideas that are invented in Media Lab, Sony Labs, and places like the Ivrea Institute. The ideas are there, but for general purpose computer usage, the mouse and keyboard is actually a pretty good interface design, so you won’t see much in tht area.
The iPod is actually quite revolutionary as an interface design, and it’s allowed more options for everday products.
Hardware innovation is happening all around us, however for the most part people are pretty happy with what they have, but it’s incrementing slowly but surely.
All we need are new kind of interface and interaction technique developped for next generation of input devices (eye-tracking, gesture & voice recongition, etc.).
New Hardware = New interface.
Projects like eye-mouse (move the mouse indicater with your gaze) are dongerous…
Kevin, you said: “…the mouse and keyboard is actually a pretty good interface design, ” For most of the past 15 years I have worked as a technical translator, spending hours a day on a keyboard. It’s not bad for ordinary typing, especially with some improvements such as the Natural keyboard, but I would not say it’s “pretty good”. Dvorak is a little better, but not worth the effort to switch full-time, I’m afraid. Dvorak didn’t predict the extra keys that we have added to the right side of the keyboard: Enter, Backspace, Ctrl and all those punctuation marks. A mouse is great for a lot of things but it’s definitely not optimal to swing your arm 8-12 inches to the side to shake the mouse, find the cursor, etc. To reduce this motion, I would like to see a cursor control device near the thumbs or between the halves of a split keyboard.
On a related subject, cell phones have gotten so small that the buttons are almost unusable, IMO. I still have my Nokia 5185 that’s about 6 years old. It’s big and heavy and the battery doesn’t last 36 hours any more, but it still has the best button layout of any phone I have ever used. I can dial it without looking at the keys, and be confident that I have done so correctly. Every month I stop in cell phone stores to see if there is a suitable replacement, but no. Almost every newer phone requires me to look at the keys, press my thumb much more carefully, often having to use the end or tip which is much more awkward. Voice dialing works great for my family who are mostly calling either other or friends, but I need to call new numbers all the time for business.
How about a small mouse ball built into a keyboard between or above the “7″ and “8″ keys, below the f6 key?
OK/Cancel is a comic strip collaboration co-written and co-illustrated by Kevin Cheng and Tom Chi. Our subject matter focuses on interfaces, good and bad and the people behind the industry of building interfaces - usability specialists, interaction designers, human-computer interaction (HCI) experts, industrial designers, etc. (Who Links Here) ?