Designing the User Experience at Autodesk

Health, Wealth, and Usability

by John Schrag

Usability lies in the details. If you’ve worked in the UX field for a while, you’ve probably heard that spoken. While it is important to build the right features in your software, it is equally important to build the features right – to take care of the details that make the difference between a user flowing effortlessly through a task, and that same user getting completely stuck at step one.

Unfortunately, this property of usability work can make it very difficult for the work to get done at all. From the high-level view, we all know that reaching a certain level of usability is critical for the success of a product. Many usability teams fight hard to get usability bugs treated just the same as functionality bugs. But there are drawbacks to this approach: in the trenches of development, individual detail-changes that leads to that usability can seem inconsequential when compared to the “real” bugs competing for developer time. “Yes, we know that people have trouble figuring out how to use the feature, but at least it’s possible.” UX fixes can get pushed further and further down the priority list, until they fall off the end of the development cycle.  

Usability practitioners aren’t the only ones with this problem. Have you ever wondered why so many health care practitioners are in such bad shape physically? They of all people know how critical it is to eat well and get regular exercise, but many of them never do. It’s not because they’re lazy (have you any idea how hard nurses work?) but rather because they are faced with the same problem. They know that exercise and good eating in general is critical, but every day they (and we) are faced with prioritization choices. It’s easy to justify skipping a good meal this one time (I have to do the grocery shopping!), or not exercising this one-day (I have to pick up my sister at the airport!). Tomorrow we’ll get back on track, right? But tomorrow something else always seems to come up. A series of completely logical small decisions turn into a health disaster over time. What’s to be done?

There is a third area where this problem is prevalent, and also where we find clues to some solutions. That third area is money management. We all know that we need to save regularly for our retirement, but we know people who never do – they live paycheck-to-paycheck, never able to save, even though they may earn good money. Again, this comes from what seems to them like a series of completely logical decisions: “No money in the retirement fund this month, because we need to pay for cousin Sue’s wedding present. Next month we’ll get back on track.” But next month, there’s always something else. In personal finance, however, there are well known and widely used solutions to this problem. I’ll talk about two in particular: “Pay Yourself First” and “Budgeting”, and see how they might apply to UX.

“Pay Yourself First” is where you set up automatic deductions from your pay or your bank account. So the money is removed from your awareness before you get a chance to spend it on something else. This is probably the best technique for ensuring that you have long-term savings – it prevents you from ever having to make those kinds of prioritization choices. “Budgeting” is where you decide in advance how you are going to divide up your resources (money) into each category of spending. It’s a way to make sure that line-by-line prioritization decisions don’t lead to ridiculous outcomes. (Imagine if a government had to compare the relative priority of every line item in their spending --- all money would be spent in critical health care. Is fixing the potholes on that road really more important than saving Grandma’s life?)

Can we apply these principles back to our UX problems? We sure can.

For Usability Practitioners, “Pay Yourself First” can happen a few ways:

1.  If you have outstanding usability fixes from a previous release, try scheduling them early in the next development cycle – before new bugs are coming in.
   
   
2. Whenever possible, try to do your usability testing and improvements on paper prototypes of your designs, so that by the time development starts work, the usability problems have been fixed.

There are also a few ways to use the idea of “Budgeting” – these will require the cooperation of the development team:

1. Budget by time: set aside one day a week (or every two weeks) where the development team does nothing but fix little usability bugs. A concentrated effort can plow through a large number of bugs in a short time, leading to real improvement. Towards the end of the development cycle, it is likely that development may decide that “regular” bug fixing is more important, but by that time a lot of the required usability work will already be done.
   
   
2. Budget by person: Have one developer (this is a great job for an intern or summer student) who is dedicated to fixing small problems, including usability problems. It works even better if this developer reports to the usability team, and can’t be pulled onto other duties.

In the comments section, let me know how your team gets its usability fixes in. Let's get a conversation going.

Method 17 of 100: The User Interface Race

100 User Experience (UX) Design and Evaluation Methods for Your Toolkit

by Chauncey Wilson

This is the 17th in a series of 100 short articles about UX design and evaluation methods. Since I am about to go on a 6-week sabbatical, I thought that it might be fun to look at a method Ben Shneiderman called “the user interface race”. This is an entertaining method for examining how experts solve problems under substantial time pressure and public scrutiny.

Method 17 of 100:

In the old, but classic book, The Art of Human-Computer Interface Design (edited by Brenda Laurel, 1990), Ben Shneiderman wrote a short chapter on user interface races. In the user interface race, participants (generally experts whose egos are secure) compete against the clock and other criteria to accomplish a specific goal. The results of the race are then used to improve the quality of products and promote usability and good design.

When to Use:

You can use user interface races to:

• Reveal how users work under pressure.
• Evaluate competitive products.
• Promote good usability in a public and entertaining way.
• Provide insights into shortcuts and tasks strategy employed by power users (a group often neglected in traditional usability evaluations).

Procedure:

Planning for the Race

1. Choose participants who are open to public competition.
2. Choose a task for your race that is challenging for participants and engaging for viewers.
3. Develop the criteria for judging task performance. For most tasks, you might just use time, but you might also include a panel of judges to rate the quality of the solution (like the Olympic panels that judge divers and skaters).
4. Choose the judges for the competition and train them in the race method and ground rules.
5. Develop the tasks for the race. Keep the tasks challenging, but make sure they fit in a reasonable amount of time.
6. Develop the ground rules for local and remote viewers of the race.
7. If you are testing products that are new to the participants, give them equal amounts of time to “train for the race”.

Conducting the Race

1. Welcome viewers and participants to the race. Review the procedures and explain the task and scoring systems.
2. Have the participants read and sign consent and non-disclosure documents if they are from outside your company.
3. Provide time for the participants to warm up with the system if they are not using their own hardware and software.
4. Give the race participants the task and ask them to read and ask any questions. Avoid giving away any key tips.
5. When the participants are ready, give the signal to begin (Ready! Set! GO!)
6. When time is up or the participants have finished the task, Signal the end of the competition based on your rule for stopping.
7. Declare a winner and (optionally) award prizes (which could be gift certificates, plaques, or branded merchandise).

Strengths and Weaknesses:

+ Seeing people work under time pressure without much structure (e.g., the series of tasks in a usability test) can yield useful insights.

+ User interface races can be fun and be used to promote usability.

- There aren’t many cases in the literature about the user interface race method so some practice might be required before you stage a real event. Consider doing a trial run with friendly colleagues.

- I’ve run across people in workgroups who didn’t like the competitive aspect of the race method. You are likely to have a few colleagues who don’t like the concept of a race. You do have to choose participants who have relatively strong egos and who can accept winning or losing.

References:

Berkun, S. (2003). INTERACTIONARY - Sports for design training and team building. Retrieved on April 28 from http://www.uiweb.com/dsports/default.htm.

Keith S. Karn, Thomas J. Perry, Marc J. Krolczyk (1997). Testing for Power Usability: A CHI 97 Workshop. SIGCHI Bulletin Vol.29 No.4, October 1997. Retrieved on April 28, 2005 from http://bulletin2.sigchi.org/archive/1997.4/karn.html

Shneiderman, B. (1990). User interface races: Sporting competition for power users. In B. Laurel (Ed.). The Art of Human-Computer Interface Design. Reading, MA: Addison-Wesley. Pp. 221-224.

---

What’s Next in the Series?

I will be on sabbatical for the next 6 weeks. I may pen a post while I am away, but if not, I’ll continue with number 18 in October.

Transforming Qualitative Data into Actionable Information

by Joshua Ledwell

­­Interviews and open-text survey questions are two common user research methods that often result in large amounts of qualitative data. Some UX practitioners don't know where to start analyzing all the notes and open-text answers their research generates. Others may accurately analyze the data, but the results may take too long to generate or they may result in no business value for their stakeholders. 

To speed up the analysis process, consider coding your survey responses, interview notes and transcripts. A coding scheme establishes a framework for analysis by organizing the information into categories most useful for the researchers. After exhaustively reviewing a set of interview notes about our engineering software, we induced this list of nine workflow-oriented codes. We successfully used the same coding scheme on later survey results and interviews.

Code	Description	Useful to
Followup	A customer question or issue that requires an answer.	Mark specific issues for a customer service response.
Key quote	The customer's words.	Use a quote to emphasize a point to stakeholders.
Other tool	Participant uses another product to get the job done.	Learn how others products fit into a customer’s overall workflow.
Pain point	Something the participant doesn't like about the software.	Examine these issues to find improvement ideas.
Positive	Something the participant likes.	These help balance a report that might otherwise contain all negatives.
Process	How customers use the software.	Validate assumptions about how your users work.
User	Information specific to a firm or participant.	Effectively, this category is a catch-all for data that’s interesting but probably not actionable.
Wishlist	Customer feature requests.	Don’t turn these directly into requirements. Analyze wishlist requests to ensure you understand the root customer need.
Workaround	To get the job done, participant is using the software in unintended fashion.	These point to missing, difficult to discover, or inadequate features.

Coding of qualitative data may take place in the moment, as a facilitator takes notes during an interview. The coding may also take place retrospectively when analyzing survey responses or transcripts. Either way, the strength of this simple coding scheme is speed. The facilitator can use written shorthand symbols or macro keyboard shortcuts to quickly mark each sentence or statement. A method we found successful was to break up passages of text into bullet points or paragraphs, then paste the text into Microsoft Excel. Each item becomes a single row that can be tagged in another column with one of the codes, then sorted and manipulated at will.

Sometimes a single customer statement may apply to two or more codes. In these cases it’s often best to break the statement into its coded pieces. For example, a customer may say “We use the color-coding filter to mark which user owns each part, but it really slows down the software.” In this case you have both a Process note and a Pain point to consider.

This coding framework is simple enough that a whole team can collaborate on the analysis. By distributing pieces of a large survey or several interview transcripts across team members, a large amount of data can be processed in a shorter period of time. However, note-taking shorthand, word choice and handwriting may be difficult for a separate coder to understand, so it is usually best to have the original facilitator code their own notes, or at least ensure they are available to clear up any potential ambiguity.

Fictional example of coded interview notes:

Code	Note
Other tool	Enterprise Customer A uses our widget along with paper documents.
User	They are currently using our older product and weighing the purchase of our new product.
Key quote	"Widgets are more useful than paper, but why are they slower to use?"
Pain point	Their widgets take too long to boot up in the morning.
Followup	How can they set up their widgets to start up faster?
Process	For their planned upgrade, they'll need to roll out the new widgets one office at a time. Some offices will have the old widgets for a long time.
Wishlist	Need to be able to take control of the widgets in an entire office over the network.
Workaround	They implemented user interface based on old paper forms, but on the widget they just skip the fields that they don't need.
Positive	They like the idea of the mobile widget project.

Once your qualitative data is coded, some of it may be actionable immediately, especially “Followups” and “Pain points”. Other feedback, such as “Wishlist” and “Workaround” items, may lead to valuable discussion and creative ideas about how to enhance the software. The remaining items may go into a report for stakeholders, or be used in a holistic affinity diagramming exercise.

A final note: Our team has found the coding scheme above useful, but practitioners should consider what framework will best meet the goals of their customer research. If your goal is to create user personas, for example, your coding should relate to the components of your personas. Now, code away and enjoy transforming all of that text into actionable next steps for your organization!

References

Patton, M. Q. (2002). Qualitative Research and Evaluation Methods (3rd ed.). Thousand Oaks, CA: Sage.

Strauss, A. & Corbin, J. (1990). Basics of Qualitative Research. Thousand Oaks, CA: Sage.

Thomas, D. R. (2006). A general inductive approach for qualitative data analysis. American Journal of Evaluation 27 (2), 237-246

Method 16 of 100: Backcasting

100 User Experience (UX) Design and Evaluation Methods for Your Toolkit

by Chauncey Wilson

This is the 16th in a series of 100 short articles about UX design and evaluation methods. Today’s method is “backcasting”. This is a method for futures planning that can be done in a workshop format.  

Method 16 of 100: Backcasting

Backcasting is a strategic planning method that asks a group of stakeholders to create ideal future scenarios and then work backwards as a group to figure out what is needed to get to the ideal states from the current state.

Frequently applied to sustainability, energy, and urban planning futures, backcasting has also been employed by UX design groups to model the user experience of a product 3, 5, 10, or even 20 years into the future.

When to Use:

You can use backcasting to:

• Develop visions of the future.
• Support strategic design projects that involve multiple products and users.
• Test assumptions for a complex, multi-year project that will certainly be the “next great thing”.
• Diverge from predictions that are conveniently extrapolated from what you already know (the forecasting approach) to freely envision a reality that is an extreme success.

 Procedure:

1. Find a location where you can stick things on a large wall or long table. You will need many small and large sticky notes in multiple colors.
2. Recruit a group of stakeholders and thought leaders. Backcasting is an activity that is easier if all participants are in the same room (though you could envision a future where this is done remotely and use backcasting to work out how to have a successful remote backcasting session).
3. Determine how far in the future you will start from; e.g., companies often develop 5-year plans that would be candidates for a backcasting workshop.
4. Establish a set of questions about the time frame, current state, future ideal states, actions, indicators, risks and opportunities that the facilitator will use during the backcasting exercise. Matthew Milan has a good list of questions on these topics starting on page 52 of his public slide show on backcasting (http://www.slideshare.net/mmilan/backcasting-101-final-public).
5.  Brief your stakeholders on the purpose of the backcasting session and the ground rules.
6. Conduct ideation workshops to identify the current state and future ideal states or scenarios.
   1. Describe the current state of the project, problem, or product. For example, you might explain the current user experience of a suite of products made by your company.
   2. Define one or more possible (and successful) future states. For example, you might envision futures with augmented reality, tablets, nano-technologies, ambient feedback, and brain-wave user interfaces.
7. Consider each future state and work backwards to identify actions, assumptions, risks, benefits, and other indicators that could lead to these future states.
8. Consult additional stakeholders who were not involved in the backcasting exercise and get their feedback.
9. Publish the results using charts, maps, stories, list of actions required, risk/benefit trade-offs, and photographs of the actual items produced during the backcasting exercise.

Strengths and Weaknesses:

+ Simplified backcasting can be done on the cheap in one or two-day workshops using sticky notes, pens, paper, and string.

+ Backcasting is a way to reach a common understanding of successful futures and the steps  required to achieve them.

+ Backcasting provides actionable information.

- Backcasting requires a strong facilitator and solid preparation.

-Valuable backcasting results can be easily lost, unless there is an ongoing campaign to keep the results in the eyes of the stakeholders.

References:

Backcasting. Retrieved on July 25, 2011 from http://wearearising.org/2009/01/13/backcasting/

Milan, M. Backcasting 101: Retrieved on July 20, 2011 from: http://www.slideshare.net/mmilan/backcasting-101-final-public

Quist, J., & Vergragt, P. (2006). Past and future of backcasting: The shift to stakeholder participation and a proposal for a methodological framework. Futures
Volume 38, Issue 9 , November 2006, 1027-1045

Wikipedia. Backcasting. Retrieved on July 20, 2011 from:http://en.wikipedia.org/wiki/Backcasting 

---

What’s Next in the Series?

The next UX method posting will be a summer surprise J.