100 User Experience (UX) Design and Evaluation Methods for Your Toolkit
This is the 17th in a series of 100 short articles about UX design and evaluation methods. Since I am about to go on a 6-week sabbatical, I thought that it might be fun to look at a method Ben Shneiderman called “the user interface race”. This is an entertaining method for examining how experts solve problems under substantial time pressure and public scrutiny.
Method 17 of 100:
In the old, but classic book, The Art of Human-Computer Interface Design (edited by Brenda Laurel, 1990), Ben Shneiderman wrote a short chapter on user interface races. In the user interface race, participants (generally experts whose egos are secure) compete against the clock and other criteria to accomplish a specific goal. The results of the race are then used to improve the quality of products and promote usability and good design.
When to Use:
You can use user interface races to:
- Reveal how users work under pressure.
- Evaluate competitive products.
- Promote good usability in a public and entertaining way.
- Provide insights into shortcuts and tasks strategy employed by power users (a group often neglected in traditional usability evaluations).
Procedure:
Planning for the Race
- Choose participants who are open to public competition.
- Choose a task for your race that is challenging for participants and engaging for viewers.
- Develop the criteria for judging task performance. For most tasks, you might just use time, but you might also include a panel of judges to rate the quality of the solution (like the Olympic panels that judge divers and skaters).
- Choose the judges for the competition and train them in the race method and ground rules.
- Develop the tasks for the race. Keep the tasks challenging, but make sure they fit in a reasonable amount of time.
- Develop the ground rules for local and remote viewers of the race.
- If you are testing products that are new to the participants, give them equal amounts of time to “train for the race”.
Conducting the Race
- Welcome viewers and participants to the race. Review the procedures and explain the task and scoring systems.
- Have the participants read and sign consent and non-disclosure documents if they are from outside your company.
- Provide time for the participants to warm up with the system if they are not using their own hardware and software.
- Give the race participants the task and ask them to read and ask any questions. Avoid giving away any key tips.
- When the participants are ready, give the signal to begin (Ready! Set! GO!)
- When time is up or the participants have finished the task, Signal the end of the competition based on your rule for stopping.
- Declare a winner and (optionally) award prizes (which could be gift certificates, plaques, or branded merchandise).
Strengths and Weaknesses:
+ Seeing people work under time pressure without much structure (e.g., the series of tasks in a usability test) can yield useful insights.
+ User interface races can be fun and be used to promote usability.
- There aren’t many cases in the literature about the user interface race method so some practice might be required before you stage a real event. Consider doing a trial run with friendly colleagues.
- I’ve run across people in workgroups who didn’t like the competitive aspect of the race method. You are likely to have a few colleagues who don’t like the concept of a race. You do have to choose participants who have relatively strong egos and who can accept winning or losing.
References:
Berkun, S. (2003). INTERACTIONARY - Sports for design training and team building. Retrieved on April 28 from http://www.uiweb.com/dsports/default.htm.
Keith S. Karn, Thomas J. Perry, Marc J. Krolczyk (1997). Testing for Power Usability: A CHI 97 Workshop. SIGCHI Bulletin Vol.29 No.4, October 1997. Retrieved on April 28, 2005 from http://bulletin2.sigchi.org/archive/1997.4/karn.html
Shneiderman, B. (1990). User interface races: Sporting competition for power users. In B. Laurel (Ed.). The Art of Human-Computer Interface Design. Reading, MA: Addison-Wesley. Pp. 221-224.
What’s Next in the Series?
I will be on sabbatical for the next 6 weeks. I may pen a post while I am away, but if not, I’ll continue with number 18 in October.
Comments