by Erin Bradner
At the annual ACM Human-Computer Interaction Conference (CHI 2011) this year, I approached an unsuspecting human-factors engineer from IBM and asked him, point blank: “Are you an innovation giant?”. I wanted to push on the idea I believed Bill Buxton was getting at when he shipped his personal collection of technology to the conference. This idea was that innovation isn’t fundamentally about giant personalities and unparalleled genius. It’s about wicked smart people with healthy curiosity and a lot of hard work. Innovation is not a singular flash of genius in an open ocean of possibility, it’s the outcome of cross-fertilization and mash ups in a technology Galapagos.
Jim Lewis, from IBM, was the unsuspecting, wicked smart designer I cornered. In 1993 he helped to design the Simon, which has come to be known as the world’s first smart phone. He is an author on several of the user interface patents used in the iPhone today. Here’s the surprise: Jim tells me that there may have been genius behind the world’s first smartphone -- a genius who had the vision for the PDA-phone mash up and the emotional intelligence to motivate a brilliant team to execute on that vision. Let’s hear it from Jim.
Are you an innovation giant? If not, what do you do?
Jim Lewis: Nope. Not a giant. I'm pretty close to the national average height :) I’m a human-factors engineer. I take published findings and methods of investigation and apply them to designs to make systems easier to use. I've done that at IBM since I walked through the door 30 years ago, right up until just before this conversation … applying what we know to try to make systems that are easier and more pleasant to use.
Why is the Simon included in the Buxton Collocation at CHI?
Jim Lewis: When I heard about the Buxton Collection, I knew it was something I had to see. I haven't seen a working Simon in over 10 years. At CHI, Bill Buxton asked if there was a genius designer on the team. That is a difficult question. For the most part I would say 'no' but if I had to name one person, I would credit the guy who started the whole thing off, Frank Canova. If anyone on the team was a genius, it was him. The Simon was his vision.
What was out there and known in the personal organizer market at the time? The HP 85, the Apple Newton and the Sharp Wizard were all fairly well known. It was his idea to take a cellular phone form-factor and work within that form-factor to provide the organizer functions. That’s what is now known as a smart phone. It was also Frank’s idea to make it a touch screen. He got a small team together at IBM to make a prototype in 1992 and he took it to COMDEX. The response at COMDEX was positive enough that they wanted to make a product out of it. In 1993 they came to my department with a question about the minimum size of usable touchscreen buttons. That’s when I joined the team.
Who was on the team?
Jim Lewis: Frank came to our group of graphic designers, industrial designers and human factors engineers. There was the graphic designer, Peter Eastwood, who had both technical and artistic skills. He designed icons and also designed a screen refresh process that was ultimately patented. There were industrial designers who designed the appearance of the device, they were all trained as artists. They made sure the device was consistent with the branding principals of the organization. And then me, the human-factors engineer who worked out the details of the interaction design. We truly worked as a team.
What does that mean, to work as a team?
Jim Lewis: We would paste the graphic designs all over the wall ... “If I touch this, it goes to that.” We spent a lot of time and paper getting to where we could visualize the interactions. We asked ourselves “How are we going to design a keyboard for email, address book entries?”. This connected us to the initial question that brought the Simon developers to my department in the first place - “How small can I make a key on a touchpad and make it usable?”. Since I had been following touch screen research, I was familiar with the University of Maryland research by Ben Shneiderman and Andrew Sears. They had published findings showing success with designs using lift-off on buttons as small as 2mm square. A key characteristic of touchscreen buttons is when they trigger: do they act on land-on or lift-off? Land-on takes effect as soon as you touch the screen… that's fast. But the consequence of that is that there is no room to adjust if you make a mistake, so you need to use large buttons. Land-on is good for ATM kiosks.
For a small display with small buttons, you'll want to use lift-off. For example, if you have an onscreen keyboard with lift-off and you touched the letter F when you meant G, you could slide over to G and then lift your finger off -- not the fastest possible input but more forgiving, and you can make the buttons much smaller. With well-designed lift off, you can acquire very small targets. You can slide your finger in a text string, get the insertion point where you want it, listen or watch for feedback such as a beep, then drag to select text. This is an example of how I do my work. I keep track of what is being published and understand how that may apply to a specific design problem. In applying research to design, you can misapply research when there are variables you're not considering. It makes for a fascinating and satisfying intellectual exercise.
To quote the Buxton Collection notes from CHI: “First shown in 1993, this was the world’s first so-called “smart phone”. It only had two buttons: on/off and volume. Access to all other capabilities was via a resistive film touch sensor … Not bad for 14 years before the iPhone!” Did the CHI blurb get it right? When you were working on it, did it feel like the Simon was a decade ahead of its time?
Jim Lewis: From the perspective of the touchscreen yes, it felt ahead. At the same time, we knew that we were working on something that was not completely dissimilar from what was out there. There was no other device that would allow you to have someone on the phone, look up a contact and give them that information while you're on the phone. The device itself was unwieldy… it was a brick. It was paying the price for being early 1990s technology. It wasn’t particularly durable either. If you stepped on it, you could break it. Its price point was pretty high too - $899 in 1993. It also didn't have small multi-function hard buttons with the touchscreen soft buttons. We called it a personal communicator to distinguish it from PDAs (Personal Digital Assistants). PDAs had the organizer tools, but none were able to act as a regular telephone. You’d see people carrying phones and PDAs and it seemed logical to say “let's do a mash up!” (although we certainly didn’t use that term in 1993).
What did it feel like while you were working on that team? Was there any feeling of uncertainty?
Jim Lewis: I don't remember a feeling of uncertainty. We, the full team, built the entire operating system from the ground up so all of the aspects of touch were completely under our control from a very low level. My main memory of it was that it was the most delightful design experience I've ever had. The team was outstanding, in terms of skills and the relationships we had with each other. Those relationships have carried on. The Simon didn't survive in the market due to price and fragility. But those of us who worked on it still get together and have lunch. Julie Goodwin did the coding for the phone. She still sends out occasional emails to get us all together for lunch.
Are there any Simon patents that you’re especially proud of?
Jim Lewis: When you look at the Simon patents it looks like cast of thousands. But as I recall, there were about forty people who were full-time on the project (I’m not sure about the exact number, so please don’t hold me to that). Marking a phone number then popping up a contextual menu which includes the option to dial is in one patent. We were proud about that. It was clever. We also did some things around panning and zooming images received by fax, but no multi-touch – that technology wasn’t generally available yet. The graphic designer received a patent for his screen refresh technique.
Tell me about the design process. Was it innovative?
Jim Lewis: We followed a fairly straightforward user-centered design process. Once we had the key concepts and the primary tasks, we looked at how people did those tasks using current products. I had no formal design training. The graphic and industrial designers had formal design training. When the graphic designer made icons, I would run usability tests to be sure they were usable.
We wanted to avoid the mistakes of current products. We wanted to move ourselves forward in the design space we'd chosen. Other than trying to create something and using iterative feedback from usability tests to sharpen our designs, nothing fundamentally different about our process comes to mind.
Does it take genius mind and giant personality to innovate? If not, how is it done?
Jim Lewis: My feeling is there are many paths toward innovation. I wouldn't exclude the possibility that someone has a brilliant idea and all the skills needed to make it a successful product. Only a relatively small proportion of innovation comes from that source. If you ran a company, would you bank on that small proportion? Not if you wanted to stay in business. The probability isn't there.
The more common path is assembling a group. It doesn't hurt if you have a little bit of charismatic leadership there, which is what we had with Frank Canova. The hardware engineers were following the best engineering practices; the software engineers were following the best software practices. Then Frank brought industrial design, graphic design and human factors onto the team. Does a team always come together in a magical way? Probably not, but it sure did for the Simon. We contributed our best practices and, in this particular case, it turned out well for everyone involved.
That said, Frank was clearly the leader and visionary. He made everyone feel great about the project, no matter what stage things were in. He had a knack about making you feel really good about being part of it. When you had to put in those extra hours, you felt you were part of something larger. You were willing to go the extra mile. In some ways it was unusual. Everyone was at the top of their game. I've worked on a lot of different projects at IBM with good colleagues, but with that team, we were all friends. We spent a lot of our time laughing… that’s what happens when you make a team out of people with a lot of sharp intellect and sense of humor.
~
So the story behind one of the most wildly popular innovations of the past 20 years, the world’s first smartphone, includes genius and magic after all. The genius came from spotting a mash up, assembling a cross-functional team and motivating that team with vision and compassion. And the magic? Jim tells us that the magic came from the human factors – not just human-factors engineering, but the full spectrum of human factors that operate when the right team comes together at the right time.
Subscribe