Designing the User Experience at Autodesk

Method 20 of 100: The Magic Thing

100 User Experience (UX) Design and Evaluation Methods for Your Toolkit

by Chauncey Wilson

This is the 20th in a series of 100 short articles about UX design and evaluation methods. This entry focuses on the use of a prop called the “magic thing” that can serve as an ideation tool in the design of mobile, ambient, and ubiquitous systems.  

Method 20 of 100: The Magic Thing
In a 2000 article, Guilio and colleagues describe various role-playing techniques that support the ideation and concept development of mobile services and devices. In their research, they asked participants to carry a “magic thing” as they walked around different locales and use it “to imagine how a portable device could support [them] in a particular situation” (p. 199). The rationale was that the use of a prop might provoke some novel ideas that would not emerge sitting around a table doing traditional group brainstorming.  One common approach for this method would be to give people scenarios and then let them walk around a particular area like a city or building to imagine what kind of “magic” a portable device could provide.  One scenario might be to find a restaurant in a city that you have never visited before that serves a certain kind of food and is quiet.  Another simple scenario might be to envision software that would help you, a brand new employee, locate people with specific kinds of knowledge in a large office.  In the graphic below a magic thing is used to evoke ideas about how context-aware devices might help someone in a strange city.

According to different sources, Jeff Hawkins, one of the inventors of the PalmPilot PDA, carried a small block of wood about the size of the Palm device with him to help envision how the device might benefit him at work (e.g. to generate a to-do list or use a calendar) and at home (e.g. to create a shopping list). The block of wood was his “magic thing” and provided inspiration for a generation of mobile devices. 

Howard and his colleagues (2002)  described “endowed props” that are rough physical simulations of devices that can used in role-playing  The props can be used to provide some constraints on the ideas participants generate so the results are “plausible science” rather than science fiction. Howard et al. suggest endowing props with particular capabilities like speech recognition, one at a time, so as not to provide too much magic to the participants.

When to Use: 
You can use a magic thing to:

• Serve as a rough example of the form factor of proposed devices (e.g. a pen, tablet, phone, or clip-on device)
• Complement scenario creation exercises
• Add some fun and minimal constraints to role-playing studies
• Provide a triggering function for ideas

Procedure:
The two articles referenced below provide several variations on role-playing or scenario enactment that involve magic things or endowed props. Here is a simplified procedure adapted from Howard, et al.

1. The design/research team introduces the participants (the actors) to a scenario that they are to act out.  For example, the scenario might have you lost in a city like Rome after missing your tour bus.
2. The participants review the scenario with a member of the design team and personalize it to their own situation.
3. The participants act out a scenario without the magic thing.
4. The participants are given or choose a prop (their magic things).
5. They are told what magical powers it has (to start). For example, the magic thing knows where it is (e.g., has GPS) and has information about commercial establishments and ratings of service for many companies.
6. The participants and design team then engage in participatory role-playing where the magic thing can acquire new powers and lose old ones.  For example, you might start with a magic thing that is location aware with information about services and then add a magic power like augmented reality where you visualize information as an overlay over camera images.
7. The sessions are recorded and the ideas that emerge are captured and applied to a new round of role-playing that expands on promising ideas.  Here you take good ideas about how to build magic into a portable device and do a deeper level of role playing to get more specific ideas.

References:

Giulio Iacucci, Kari Kuutti, and Mervi Ranta. 2000. On the move with a magic thing: role playing in concept design of mobile services and devices. In Proceedings of the 3rd conference on Designing interactive systems: processes, practices, methods, and techniques (DIS '00), Daniel Boyarski and Wendy A. Kellogg (Eds.). ACM, New York, NY, USA, 193-202. DOI=10.1145/347642.347715.

Steve Howard, Jennie Carroll, John Murphy, and Jane Peck. 2002. Using 'endowed props' in scenario-based design. In Proceedings of the second Nordic conference on Human-computer interaction (NordiCHI '02). ACM, New York, NY, USA, 1-10. DOI=10.1145/572020.572022.

Method 19 of 100: Laddering Questions

100 User Experience (UX) Design and Evaluation Methods for Your Toolkit

by Chauncey Wilson

This is the 19th in a series of 100 short articles about UX design and evaluation methods. Today, the focus will be on the laddering question method, a semi-structured interview technique where you ask “why, how, what, and ‘tell me more’” questions multiple times to drill down, up and sideways to go from general to specific, specific to general, or from one item in a category to another item.

Method 19 of 100

Laddering questions can provide insight into attitudes, beliefs, and values that affect requirements, brand image, consumer preferences, and buying decisions. For example, you might be interviewing someone as she works with software in her office. During the interview, the person says, “This feature is great!” Then you would follow up with a “ladder down” question – “Why is that feature great?” The person might say “It saves me a lot of time.” Then you continue with another ladder-down question – “Why does it save you a lot of time?” The person replies that “I didn’t have to find the dialog box – I could just use the pop-up menu.” You could go further, but in this series of “why” questions, you have gone from a general emotional statement to one that gets at the underlying issue – a particular feature reduces the amount of movement and search time required to perform a task. Laddering is a way to move from surface statements to root causes, details that are actionable, or core values that might affect adoption or satisfaction or loyalty.

In addition to drilling down, you can also drill sideways and ask a set of questions that reveal multiple examples or categories related to a particular topic. For example, if you were interviewing a usability expert about what methods he uses to discover usability problems, he might say “usability testing.” You could then ask “Are there any other methods that you use to get at usability problems?” He would pause and say “pluralistic reviews.” You could drill down on “pluralistic reviews” and then drill sideways and ask for other methods that involve users, getting the answer “diary studies.” A combination of drill-down and drill-sideways questions can provide useful and persuasive data.

There are a number of methods that are based on laddering questions including the “5-Whys” method that is often associated with root cause analysis.   

When to Use:

You can use laddering questions to:

• Understand organizational culture
• Complement other methods
• Elicit and understand knowledge from experts
• Elicit knowledge and details about a new domain
• Determine details about how new features and product attributes affect the user experience
• Produce data that are actionable
• Link features and product attributes with user/customer values

Strengths and Weaknesses:

+ Laddering is a technique that gets at core values and the underlying reasons for particular behaviors or choices.

+ Laddering is a way to elicit semi-tacit information that might not be revealed by other UX methods.

- Laddering can be tiring for participants who are continually asked “why” or other laddering questions.

- Conducting interviews with many laddering questions requires a facilitator who can keep the participant engaged. People who use laddering questions need to be comfortable with the repetitive questioning style and be prepared to vary their prompts as needed.

- On some occasions, laddering can get into sensitive issues (e.g., how corporate politics affects productivity).

- Laddering questions assume that there is hierarchical information. In some domains, the information may not be represented hierarchically.

Procedure:

You can conduct interviews that are focused on laddering or you can use laddering questions as part of a more general interview protocol. Here is a high-level set of steps for using laddering questions.

1. Develop a small set of laddering probes that you can use in your study. Consider problems for moving down, up, and sideways. Here are some ideas for laddering questions:
   
   “Why do you think that this product is “great”?  (laddering down when a person makes a general statement and you want to get an underlying value, attitude, principle or rationale)
   
   “Why is this important to you?” (laddering up when a person mentions a specific feature and you want to understand the higher level value of the feature)
   
   “Can you tell me about similar/different types of….?” (laddering sideways to get examples or variations)
   
   
2. During an interview, you can elicit features, attributes, concepts, or constructs of interest. You could, for example, ask a person to list the features that she disliked the most or that are most important to productivity.
3. Ask the person to prioritize the list and then begin doing a “laddering interview” starting with the items at the top of the list. The person might tell you that she most dislikes the collaboration aspects of the product. You could then ask “Why do you dislike the collaboration aspects?”
4. Ask laddering questions until you get answers that have reached “the end of the ladder” or that are actionable.
5. Repeat the laddering exercise with the next item in the list or the next item in the interview where laddering is useful.

References:

Corbridge, C., Rugg, G., Major, N.P., Shadbolt, N.R. & Burton, A.M. (1994)
Laddering: Technique and Tool Use in Knowledge Acquisition.
Knowledge Acquisition, 6, pp. 315-341

Hawley, M. (2009). Laddering: A research interview technique for uncovering core values. UXmatters. Retrieved on January 20, 2012 from http://www.uxmatters.com/mt/archives/2009/07/laddering-a-research-interview-technique-for-uncovering-core-values.php

Rugg, G., Eva, M., Mahmood, A., Rehman, N., Andrews, S. & Davies, S. (2002)
Eliciting information about organisational culture via laddering. Information Systems Journal, 12, pp. 215-229

Sherlock, G. & Rugg, G. Using laddering and on-line self-report to elicit design rationale for software Proceedings of EASE & PPIG joint conference, Keele University, 8-10 April 2003 Addendum, pp. 453-472

The ‘5-Whys’ Method. http://www.mapwright.com.au/newsletter/fivewhys.pdf

---

What’s Next in the Series?

In the next post, I’ll describe the method of “Magic Things.”

Method 18 of 100: The User/Task Matrix

100 User Experience (UX) Design and Evaluation Methods for Your Toolkit

by Chauncey Wilson

This is the 18th in a series of 100 short articles about UX design and evaluation methods. I’m back from my sabbatical and ready to keep building the UX methods toolbox. Today, we’ll look at a simple task analysis method called the User/Task Matrix.  

Method 18 of 100

Understanding the frequency and importance of tasks that are performed by different classes of users (or personas) is critical for many design decisions. For example, if an architect will be using a function hundreds of times a day for weeks on end, then you would want to design a highly efficient user interface that included shortcuts and ways to select many things at a time. In contrast, you might have some features, like creating templates, that you do occasionally and designing the UI for templates might require support for fast learning and “re-learning” after gaps of use. If you are designing something that you only do once a year and that task is critical, you might design a step-by-step procedure. A simple approach for exposing frequency and importance by user class is the user/task matrix. This approach can also indicate which tasks are primarily the responsibility of a single user group and which ones cross groups.

Here is a generic example of a user/task matrix. The letters represent task frequency: H = high; M= medium, and L = Low. Task 4 in the matrix is a critical task and is marked with a red background. Only one user group performs that critical task with low frequency so the interface related to that task should focus on preventing errors and be easy to learn.

User Groups	User Group 1	User Group 2	User Group 3	User Group 4	User Group 5	User Group 6
Task 1	H	H	M	M	M	M
Task 2	L		L	M	H	M
Task 3	L	L	L	L	M
Task 4		L
Task 5	L	L				L

When to Use: 

You can use a user/task matrix to:

• Summarize data related to frequency and importance of tasks.
• Determine whether some tasks are used by a few types of users or many users. In the example matrix, tasks 1, 2, and 3 are used by almost all groups with varying frequencies. This information can be helpful for making trade-offs about what features should get priority during the design and development cycle. For example, something that is important, done by many user groups, with moderate to high frequency, might be an area to focus usability sessions.
• Consider whether the primary focus for design should be on ease of learning or ease of use (efficiency).
• Determine if different classes of users can be combined into a single class.
• Provide some clues about where to focus user assistance development.  

Procedure:

1. Ask questions about task frequency and importance when conducting user research. 
2. Develop a set of user groups or personas that represent the major classes of users. Keep in mind that there are some low frequency tasks (like initial deployment of software across an organization) that are critical to success and often performed by a small group of users.
3. Develop a visual coding scheme for highlighting frequency and importance in your matrix. My example is very simple.You can extend the basic concept in my example to create something visually compelling. 
4. Use the matrix as input to feature and UI design trade-offs and other design activities. A related method called the UXI matrix was presented at the 2011 UPA conference by Jon Innes and James McElroy and discussed by John Schrag at http://dux.typepad.com/dux/2011/06/the-forgotten-user.html

 

Strengths and Weaknesses:

+ The user/task matrix is simple and conveys critical information for design teams who are considering what aspect of usability they should focus on during conceptual design

+ You can easily substitute the language of your organization into the matrix. For example, you might substitute “stories” into the task column and “personas” into the user group row.

-       If you have hundreds of tasks, the matrix can become unwieldy and you might have to create a hierarchical version.

References:

Hackos, J. T., & Redish, J. C. (1998). User and task analysis for interface design. New York, NY: Wiley.

Wixon, D. & Wilson, C. E.  The Usability Engineering Framework for Product Design and Evaluation. Handbook of Human-Computer Interaction (Second Edition). Elsevier: Amsterdam, The Netherlands, 1998, pp. 653-688.

 

---

What’s Next in the Series?

In the next post, I’ll describe an interview technique called “laddering” which is used to get at root causes, understand how knowledge is organized, and get insight into organizational cultures.

Method 17 of 100: The User Interface Race

100 User Experience (UX) Design and Evaluation Methods for Your Toolkit

by Chauncey Wilson

This is the 17th in a series of 100 short articles about UX design and evaluation methods. Since I am about to go on a 6-week sabbatical, I thought that it might be fun to look at a method Ben Shneiderman called “the user interface race”. This is an entertaining method for examining how experts solve problems under substantial time pressure and public scrutiny.

Method 17 of 100:

In the old, but classic book, The Art of Human-Computer Interface Design (edited by Brenda Laurel, 1990), Ben Shneiderman wrote a short chapter on user interface races. In the user interface race, participants (generally experts whose egos are secure) compete against the clock and other criteria to accomplish a specific goal. The results of the race are then used to improve the quality of products and promote usability and good design.

When to Use:

You can use user interface races to:

• Reveal how users work under pressure.
• Evaluate competitive products.
• Promote good usability in a public and entertaining way.
• Provide insights into shortcuts and tasks strategy employed by power users (a group often neglected in traditional usability evaluations).

Procedure:

Planning for the Race

1. Choose participants who are open to public competition.
2. Choose a task for your race that is challenging for participants and engaging for viewers.
3. Develop the criteria for judging task performance. For most tasks, you might just use time, but you might also include a panel of judges to rate the quality of the solution (like the Olympic panels that judge divers and skaters).
4. Choose the judges for the competition and train them in the race method and ground rules.
5. Develop the tasks for the race. Keep the tasks challenging, but make sure they fit in a reasonable amount of time.
6. Develop the ground rules for local and remote viewers of the race.
7. If you are testing products that are new to the participants, give them equal amounts of time to “train for the race”.

Conducting the Race

1. Welcome viewers and participants to the race. Review the procedures and explain the task and scoring systems.
2. Have the participants read and sign consent and non-disclosure documents if they are from outside your company.
3. Provide time for the participants to warm up with the system if they are not using their own hardware and software.
4. Give the race participants the task and ask them to read and ask any questions. Avoid giving away any key tips.
5. When the participants are ready, give the signal to begin (Ready! Set! GO!)
6. When time is up or the participants have finished the task, Signal the end of the competition based on your rule for stopping.
7. Declare a winner and (optionally) award prizes (which could be gift certificates, plaques, or branded merchandise).

Strengths and Weaknesses:

+ Seeing people work under time pressure without much structure (e.g., the series of tasks in a usability test) can yield useful insights.

+ User interface races can be fun and be used to promote usability.

- There aren’t many cases in the literature about the user interface race method so some practice might be required before you stage a real event. Consider doing a trial run with friendly colleagues.

- I’ve run across people in workgroups who didn’t like the competitive aspect of the race method. You are likely to have a few colleagues who don’t like the concept of a race. You do have to choose participants who have relatively strong egos and who can accept winning or losing.

References:

Berkun, S. (2003). INTERACTIONARY - Sports for design training and team building. Retrieved on April 28 from http://www.uiweb.com/dsports/default.htm.

Keith S. Karn, Thomas J. Perry, Marc J. Krolczyk (1997). Testing for Power Usability: A CHI 97 Workshop. SIGCHI Bulletin Vol.29 No.4, October 1997. Retrieved on April 28, 2005 from http://bulletin2.sigchi.org/archive/1997.4/karn.html

Shneiderman, B. (1990). User interface races: Sporting competition for power users. In B. Laurel (Ed.). The Art of Human-Computer Interface Design. Reading, MA: Addison-Wesley. Pp. 221-224.

---

What’s Next in the Series?

I will be on sabbatical for the next 6 weeks. I may pen a post while I am away, but if not, I’ll continue with number 18 in October.

Method 16 of 100: Backcasting

100 User Experience (UX) Design and Evaluation Methods for Your Toolkit

by Chauncey Wilson

This is the 16th in a series of 100 short articles about UX design and evaluation methods. Today’s method is “backcasting”. This is a method for futures planning that can be done in a workshop format.  

Method 16 of 100: Backcasting

Backcasting is a strategic planning method that asks a group of stakeholders to create ideal future scenarios and then work backwards as a group to figure out what is needed to get to the ideal states from the current state.

Frequently applied to sustainability, energy, and urban planning futures, backcasting has also been employed by UX design groups to model the user experience of a product 3, 5, 10, or even 20 years into the future.

When to Use:

You can use backcasting to:

• Develop visions of the future.
• Support strategic design projects that involve multiple products and users.
• Test assumptions for a complex, multi-year project that will certainly be the “next great thing”.
• Diverge from predictions that are conveniently extrapolated from what you already know (the forecasting approach) to freely envision a reality that is an extreme success.

 Procedure:

1. Find a location where you can stick things on a large wall or long table. You will need many small and large sticky notes in multiple colors.
2. Recruit a group of stakeholders and thought leaders. Backcasting is an activity that is easier if all participants are in the same room (though you could envision a future where this is done remotely and use backcasting to work out how to have a successful remote backcasting session).
3. Determine how far in the future you will start from; e.g., companies often develop 5-year plans that would be candidates for a backcasting workshop.
4. Establish a set of questions about the time frame, current state, future ideal states, actions, indicators, risks and opportunities that the facilitator will use during the backcasting exercise. Matthew Milan has a good list of questions on these topics starting on page 52 of his public slide show on backcasting (http://www.slideshare.net/mmilan/backcasting-101-final-public).
5.  Brief your stakeholders on the purpose of the backcasting session and the ground rules.
6. Conduct ideation workshops to identify the current state and future ideal states or scenarios.
   1. Describe the current state of the project, problem, or product. For example, you might explain the current user experience of a suite of products made by your company.
   2. Define one or more possible (and successful) future states. For example, you might envision futures with augmented reality, tablets, nano-technologies, ambient feedback, and brain-wave user interfaces.
7. Consider each future state and work backwards to identify actions, assumptions, risks, benefits, and other indicators that could lead to these future states.
8. Consult additional stakeholders who were not involved in the backcasting exercise and get their feedback.
9. Publish the results using charts, maps, stories, list of actions required, risk/benefit trade-offs, and photographs of the actual items produced during the backcasting exercise.

Strengths and Weaknesses:

+ Simplified backcasting can be done on the cheap in one or two-day workshops using sticky notes, pens, paper, and string.

+ Backcasting is a way to reach a common understanding of successful futures and the steps  required to achieve them.

+ Backcasting provides actionable information.

- Backcasting requires a strong facilitator and solid preparation.

-Valuable backcasting results can be easily lost, unless there is an ongoing campaign to keep the results in the eyes of the stakeholders.

References:

Backcasting. Retrieved on July 25, 2011 from http://wearearising.org/2009/01/13/backcasting/

Milan, M. Backcasting 101: Retrieved on July 20, 2011 from: http://www.slideshare.net/mmilan/backcasting-101-final-public

Quist, J., & Vergragt, P. (2006). Past and future of backcasting: The shift to stakeholder participation and a proposal for a methodological framework. Futures
Volume 38, Issue 9 , November 2006, 1027-1045

Wikipedia. Backcasting. Retrieved on July 20, 2011 from:http://en.wikipedia.org/wiki/Backcasting 

---

What’s Next in the Series?

The next UX method posting will be a summer surprise J.

Method 15 of 100: Misuse Scenarios

100 User Experience (UX) Design and Evaluation Methods for Your Toolkit

by Chauncey Wilson

This is the 15th in a series of 100 short articles about UX design and evaluation methods. Today’s method is the “misuse scenario”. This is a method that focuses on possible misuses, both unintentional and malicious, of a system.

 Method 15 of 100: Misuse Scenarios

Scenarios are used to focus product teams on user needs in specific contexts. There are a variety of scenario types that you might consider in design. Here are a few general categories:

• Typical scenarios - These scenarios focus on common tasks and contexts encountered by the major personas.
• Atypical scenarios - These are infrequent, but sometimes important, scenarios. An example here would be the scenario for the recovery of information after a major and extended power failure. Recovery after a disaster is rare and an atypical scenario might describe this rare event (sometimes called a “black swan”) and how it would be handled.
• Extreme (but not misuse) scenarios - Extreme scenarios often focus on scalability issues like what happens to a Web site when a particular event results in quadruple the number of people expected by the requirements document pounding your server.

I would add misuse scenarios to this list. Misuse scenarios might deal with events like vandalism or inadvertent damage.   An example where misuse scenarios might aid design would be public kiosks (e.g., mailing kiosks) that you find in museums or malls. What kinds of misuse scenarios might be relevant? Here are a few:

1. Vandals put superglue in the slots for credit cards, receipts, and stamps. (malicious misuse).
2. Parents put their infants on the scale designed to weigh packages of no more than 15 pounds (unintentional misuse).
3. A slightly intoxicated couple spills a drink during the science museum’s New Year’s gala (unintentional misuse).
4. People with the flu and other diseases handle the mouse and touch screen in a health clinic (inadvertent contamination of public computer systems).
5. An evil person gathers information from a web browser at a library that wasn’t cleared after someone left quickly to deal with a home emergency (malicious misuse).
6. The software is being used for something completely unintended (could be malicious or not).

When to Use

Misuse scenarios are probably something to consider on any project where there is potential for someone to perform either malicious actions or actions that result in an unintentional, but unfortunate damage to a person, organization or the product itself.

This method can be used early in the product design and development process as part of requirements gathering and conceptual design. Including misuse scenarios in the UX design process could lead to a system design that would minimize the damage caused to the system itself.

 Procedure

1. Consider typical or atypical scenarios or common use cases and convert them to misuse cases.
2. Tap field support or technical support colleagues and ask them to recount stories related to misuse of a similar product or system.
3. Consider aspects of the environment that might interact with the system and users to create a misuse scenario.   For example, consider how a new handheld device might be affected by a torrential rain storm or extreme heat or the actions of a person who found the device on the ground and was lacking in ethics.
4. Brainstorm with a range of colleagues about “how could things go wrong with our system?”
5. Create a matrix of misuse cases and brainstorm ways to eliminate or mitigate any threats.

Strengths and Weaknesses

+ There are often stories and bug reports that can be used to develop misuse scenarios. Using misuse scenarios strengthens the likelihood that the entire user experience is considered during design.

- It is sometimes hard to envision misuse scenarios for new products. Sometimes the focus is so much on typical scenarios, use cases or user stories, that there is little effort spent on formulating misuse scenarios and possible solutions.

- Misuse scenarios can be quite vivid and may prompt design teams to go overboard in designing against misuse.

References

Alexander, Ian. Misuse Cases: Use Cases with Hostile Intent. Retrieved on June 29, 2011 from http://perceval.gannon.edu/xu001/teaching/2010spring/GCIS504/termpaper/readingList/MisuseCasesHostileIntent.pdf

Alexander, Ian, Use/Misuse Case Analysis Elicits Non-Functional Requirements, Computing & Control Engineering Journal, Vol. 14, 1, pp. 40-45, February 2003

Wikipedia. Scenario. Retrieved on June 29, 2011 from http://en.wikipedia.org/wiki/Scenario_(computing)

---

What’s Next in the Series?

The next UX method posting will discuss backcasting, a technique for generating a vision of a desirable future and then working backwards to see what is needed to get to that desirable future.

Method 14 of 100: “Yes, and …”

100 User Experience (UX) Design and Evaluation Methods for Your Toolkit

by Chauncey Wilson

This is the 14th in a series of 100 short articles about UX design and evaluation methods. Today’s method is “Yes, and …”. This is a technique borrowed from improvisational (improv) theater that can be used to generate ideas and requirements. The technique requires little training and can be used with small or larger groups.

Method 14 of 100: “Yes, and …”

In the world of improv theater, performers start with an idea or scene and then build on that by adding a bit of information. The process of adding new information is referred to as “Yes, and …” and this method allows performers to continually redefine the scene and make leaps into new dramatic (or comedic) territory.

The approach can be used in the UX domain as a way to explore possible user scenarios, generate requirements, brainstorm, or envision future concepts. To start the process, you provide an initial verbal scenario and then ask colleagues to say “Yes, and …” and then fill in some new information or add new requirements.

Here is a short hypothetical “Yes, and …” session undertaken by a group of design colleagues or foodies who are defining the concept of an electronic menu for a new restaurant:

• I am planning a new restaurant and want to use electronic menus that offer the diner all the information that a waiter or waitress could normally provide.
• (a colleagues says) “Yes, and the electronic menu must be water proof because kids might spill things on it…
• (another colleagues says) “Yes, and we need to provide pictures of our meals…
• Yes, and we need to a way to allow people to do substitutions…
• Yes, and we need to include a way for people to tell us about allergies...
• Yes, and the calorie count must be included – that’s the law now…
• Yes, and we need to know if this is a romantic dinner or that someone is going to propose…
• Yes, and when someone is going to propose, we can bring out our special cake and make sure this is something they remember. 
• ... and on and on and on with “Yes, and …”

When to Use:

This method can be used early in the product design and development process as a way to elicit requirements, explore concepts, or even help develop personas. This approach sometimes reveals gaps or missing requirements that might elude more structured approaches. It is also a fun and simple approach that can enliven a tired team.

 Procedure:

1. If you read the improv literature, you will find different approaches for the “Yes, and …” method, but the basic process is to assemble a group and provide a starting statement or scenario like the electronic menu example.
2. Describe the "Yes, and ..." method and provide a brief example.
3. Consider if there are props that you could use to trigger ideas, for example, in the electronic menu example, you might bring an iPad to the “Yes, and …” session.
4. Ask someone to write the ideas where they are visible to the group or tape the session so you capture all the details.
5. Have the leader of the group start the “Yes, and …” session with the introductory statement.
6. Encourage someone to say “Yes, and …” and add something new. If you need to, you can say “Yes, and …” to prompt the next reply.
7. Continue until the group slows down.
8. Thank everyone for their contributions and willingness to engage in the “Yes, and …” activity.

Strengths and Weaknesses:

+  The basic “Yes, and …” method as applied to UX does not require a lot of training (though if you are planning on doing improve for a living and will depend on humor, you will need a lot of practice to be perceived as spontaneous). As a side note, some training on improv and theater techniques could be quite useful for a UX group. There are many papers on the application of theatrical methods to product design.

-  I’ve seen this method get a bit out of control and you might need to intervene at some point and direct the group away from the ridiculous without seeming like a dictator. In improv theater, you would not generally engage in this type of blocking of the scene development, but in the product development world, sometimes you need to change a focus that has gone outside the bounds of usefulness or appropriateness.

References:

Danzico, L. (2010). From Davis to David: lessons from improvisation. interactions 17, 2 (March 2010), 20-23.

Gerber, E. (2009). Using improvisation to enhance the effectiveness of brainstorming. Proceedings of the 27th international conference on Human factors in computing systems (CHI '09), 97-104. 

Wikipedia. Improvisational theatre. Retrieved from http://en.wikipedia.org/wiki/Improvisational_theatre on June 1, 2011

 

---

What’s Next in the Series?

The next UX method posting will discuss misuse scenarios. These are scenarios that focus on threats, both intentional and unintentional, to a system.

 

 

Method 13 of 100: Consistency Inspection

100 User Experience (UX) Design and Evaluation Methods for Your Toolkit

by Chauncey Wilson

This is the 13th in a series of 100 short articles about UX design and evaluation methods. Today’s method is “consistency inspection”.  This is a general method for examining one or more products for internal or external consistency. Every company I’ve worked for has had a goal to “make our products consistent,” but as I described in an earlier DUX blog, http://dux.typepad.com/dux/2009/03/the-consistency-conundrum.html, the concept of consistency is complex. The goal of this posting is to suggest a general method for finding and cataloging inconsistencies within a single product or between multiple products. Comments on this general method would be much appreciated.

Method 13 of 100: Consistency Inspection

When to Use:

Consistency inspections are useful when you get the “make things consistent” mandate from senior management, acquire new products that you will be integrating with existing products, or when your users are complaining about how consistency hampers learning and increases errors.

 Procedure:

1. Create a small team of “inspectors” with different backgrounds – design, usability, training, QA, user assistance, development, and product management. The rationale for a small team is that different perspectives will yield more consistency issues.
2. Define the target of the inspection. The target can be a particular area of a product, a product, or a set of products. Ask the inspectors to review specific areas of the target product against any consistency exemplar and references. In a major product with many features, you might assign sub-teams that will each look at a particular type of consistency.
3. Define an explicit list of what consistency attributes are the focus of the inspection. You may want to focus on a single attribute like icon/menu consistency or multiple attributes (e.g., terminology, layout, use of UI controls, workflow, work-context).
4. Provide the “exemplar” or reference against which you will compare within a product or between products. For example, you might have a style guide like the OSX or Windows 7 user experience guidelines with standards and guidelines that you use to determine where the product or products are inconsistent. You might designate a particular application as the exemplar – the product that you want all others to be consistent with – and note where related applications, like those in a suite, differ from the exemplar. You might also have a pattern library that you can use as a “consistency reference”.
5. Train your consistency inspectors on the following topics:
   1. The definition of consistency and the different levels (e.g., within a dialog, between dialogs, across one or more applications, with applications that are used with a particular application, with the Mac or PC operating systems) and attributes of consistency.
   2. Examples of “consistency bugs”. Here, you want to provide clear examples of items that you want to track as consistency bugs and have discussions to help the team grasp the range of consistency problems that will emerge.
   3. How to report consistency issues? A template for reporting might include: The product or products under scrutiny, the particular component (e.g., file open dialog or pop-up menu), the exemplar or reference that you using to determine if something is inconsistent, and the type of inconsistency.
6. Provide the inspectors with the target of the inspection, which can be a spec, a wireframe, a prototype, or working product. Early consistency inspections are highly recommended to avoid embarrassment late in development. For example, you might find that there are two different search engines in the same product that worked very differently – an embarrassing and costly consistency gaffe!
7. Categorize and prioritize the issues and work with the inspection team on a plan to improve consistency.

Strengths and Weaknesses:

+  This method provides some explicit guidance about what to look for and compare against when you are asked to review a product or set of products for the multifaceted concept of consistency.

+  Creating a team and training them on consistency can promote consistency across groups. For example, once developers or product managers do a consistency review, they will be sensitive to issues of consistency in the future.

-  Consistency is a complex characteristic since there can be many reference sources against which to make consistency judgments. In addition to this inspection method, you need to promote consistency in many ways (see the oldie, but good book, “Coordinating User Interfaces for Consistency” for many ideas on how to promote and improve consistency)

References:

De Oliveira, R. & Da Rocha, H. V. (2007). Consistency priorities for multi-device design. In Proceedings of the 11th IFIP TC 13 international conference on Human-computer interaction (INTERACT'07), Cécilia Baranauskas, Philippe Palanque, Julio Abascal, and Simone Diniz Junqueira Barbosa (Eds.). Springer-Verlag, Berlin, Heidelberg, 426-429.

Grudin, J. (1989). The case against user interface consistency. Commun. ACM 32, 10 (October 1989), 1164-1173.

Nielsen, J. (1989). Coordinating user interfaces for consistency. San Francisco: Morgan Kaufmann. [Note about the book at http://www.useit.com/jakob/constbook_preface_2nded.html]

Wilson, C. (2009). The Consistency Conundrum. http://dux.typepad.com/dux/2009/03/the-consistency-conundrum.html

---

What’s Next in the Series?

The next UX method posting will describe a method borrowed from improv called “Yes, and…” that you can use as an ideation technique for stories, scenarios, and requirements.

 

 

Method 12 of 100:Teachback

100 User Experience (UX) Design and Evaluation Methods for Your Toolkit

By Chauncey Wilson

This is the 12th in a series of 100 short articles about UX design and evaluation methods. Today’s method is called “teachback”. In the teachback method an expert explains concepts, workflow, procedures, or instructions, associated with a particular domain to a non-expert (who could be a user or an interaction designer). The non-expert then tries to teach back what the expert had explained. During the teachback session, the expert corrects any misunderstandings. The misunderstandings and the actual expert discussion are both data that interaction designers can use as input for product design.

As an example of how teachback can be used for design, consider an interaction designer with some basic knowledge of a product or domain. That designer is tasked to work on some new features that are complex, so she finds several experts and asks them, one at a time to explain work related to that new feature. She then teaches back what she learned from each expert and asks the expert to correct any errors or misconceptions. The interactions are recorded. After several teachback sessions, the interaction designer should have a much deeper understanding of the issues and concepts associated with the new features and probably discovered some use cases that had been missed.

Method 12 of 100: Teachback

When to Use:

Teachback is a technique for extracting information from experts that would be useful for gathering requirements, learning about workflow, and understanding mental models. The method is also used in healthcare to determine if patients understand instructions about medication or medical procedures. In the healthcare arena a medical provider, home health aid, nurse, or doctor, would describe a procedure to a patient and then ask them to “tell me in your own words how you would…”

Procedure:

The teachback requires a domain or subject matter expert, a non-expert (the “learner”), and, optionally, a facilitator who takes notes and handles the recording. The learner and the facilitator can be the same person.

1. A subject matter expert is asked to describe a concept, task, or something else about a particular domain or product. A facilitator records the sessions with audio or video (or just takes notes if electronic recording is not possible.
2. The non-expert (e.g., the interaction designer listens carefully to the expert.
3. The non-expert is then asked to “teach back to the expert” what the expert had previously described or explained.
4. When the non-expert is teaching back what she learned, the expert corrects any misinterpretations, errors, or simplifications.
5. The session is repeated with another expert who again teaches the non-expert.
6. The transcripts from all the sessions are examined for common misinterpretation, errors, or simplifications.

Strengths and Weaknesses:

+   The basic approach is simple. An expert discusses or explains something to a non-expert related to a particular domain or product. The non-expert then describes back what he/she learned.

+ The method can highlight concepts that are hard to understand, instructions that are difficult to grasp, or gaps in mental models.

- The teachback method is not highly structured. The expert can present information in different ways using different styles, which might create problems for the non-expert.

- The non-expert needs to have some basic awareness of the concepts for a product or process so the expert doesn’t feel that his or her time is being wasted.

- The non-expert might feel bad if the expert’s corrections are too harsh.

- If you choose an expert who is arrogant and obnoxious, you might have an unhappy learner and a wasted session.

References:

Johnson, L. & Johnson, N.E., (1987). Knowledge Elicitation Involving 

Teachback Interviewing in Kidd, A.L., (Ed.), 1987, Knowledge Acquisition for Expert Systems: A Practical Handbook.

---

What’s Next in the Series?

The next UX method posting will describe some ideas on how to do consistency reviews. This is a long-delayed continuation of an earlier column about consistency in software design. The next posting will be somewhat more general than earlier ones.

 

UXD Method 11 of 100: Bodystorming

by Chauncey Wilson

This is the 11th in a series of 100 short articles about UX design and evaluation methods. Today’s method is called bodystorming. Bodystorming is an immersive ideation method for exploring ideas through role-playing and physical interaction with props, prototypes, actual products, and physical spaces. The overall goal of bodystorming is to understand the relationships between people, their physical location, and the things (e.g., tools, devices, materials) they use in that environment.

If you were designing a postal kiosk for a mall, for example, you might go to a mall with a prototype system and role-play a scenario about trying to use the kiosk when you are holding a baby and trying to weigh a package. Another bodystorming scenario suggested by Oulasvirta, Kurvinen, & Kankainen (2003) involves role playing elderly users (with memory problems) as they try to remember product information in a supermarket or drug store.

Method 11 of 100: Bodystorming

When to Use:

Bodystorming is useful when you are designing devices or interior or exterior spaces. For example, you might use bodystorming to understand how users of different heights and ages would experience different versions of aircraft cabins (for example, what are the problems with lifting luggage in crowded planes from the floor to the overhead bins), or the layout of modern train cars. Bodystorming can be quite useful in understanding the experience of teams who work in close quarters like doctors and nurses in an operating room or the cooking staff in a restaurant. Bodystorming is a way to envision how people will interact with ubiquitous computing systems like smart homes and virtual meeting spaces.

Procedure:

There are different approaches to bodystorming. Here is a basic approach.

1. Get a small group together for the bodystorming.
2. Define the locations were a product or service will be used. Go to those locations and observe how people interact. Watch how people interact with each other and the artifacts in their environment.
3. Develop the prototypes and props that you need to explore an idea. You can do this on the cheap with cardboard, sketches, existing furniture, and whatever else you can find nearby.
4. Identify the personas or roles that are important for understanding your product, service, or environment. Keep in mind that people may play the role of hardware or software as well as the roles of customer, user, or troubleshooter.
5. Role play different scenarios. Feel free to improvise and role play new situations and scenarios that emerge from your initial round of bodystorming.
6. Reflect on the bodystorming experience. What did you learn? What new questions emerged? You are likely to find new possibilities as well as gnarly problems.

Strengths and Weaknesses:

+ The method is useful for generating empathy for users.

+ The method provides clues about the impact of the environment on the user experience.

-  Bodystorming involves role playing and improvisation which may be difficult for some members of a product team.

 

References:

Dennis Schleicher, D., Jones, P. & Kachur, O. (2010). Bodystorming as embodied designing. interactions 17, 6 , 47-51.

Fairbrother, A. (2008). Bodystorming. Retrieved on April 4, 2011 from http://annefairbrother.co.uk/2008/03/26/bodystorming/

Gray, D., Brown, S., & Macanufo, J. (2010). Gamestorming: A playbook for innovators, rulebreakers, and changemakers. O’Reilly.

Oulasvirta, A., Kurvinen, E., & Kanjaunen, T. (2003). Understanding contexts by being there: Case studies in bodystorming. Personal Ubiquitous Computing 7, 2, 125–134.

---

What’s Next in the Series?

The next UX method posting will describe a method called teachback that is used to elicit and understand information from subject matter experts.