Designing the User Experience at Autodesk

Collaborative Design

by John Schrag

A while ago, I posted an article (Values in Software Design Practice) describing what values my co-workers and I believe contribute to a healthy software design practice. At that time, I promised to go into more details on the individual items of that list. This posting describes why we value Collaborative Design over Design by Referendum or Design by Fiat.

Just to be clear, I want to first define what I mean by these approaches.

Design by Fiat* means that one person makes all the decisions. This is how many people imagine design works – at the core of this idea are the mythic Great Designers, those big names that dominate Fashion, or Automotive, or Architectural Design. Great ideas just pour out of them – or so people think. Sometimes these designers think so themselves, which can be a real problem.

I’ve worked at companies where design proceeds as if everyone were a Great Designer. A problem is identified, a feature is decided on, a designer writes a design document which gets tossed to development to build. There is no review (or only cursory review), and little or no usability testing. That’s a lot of faith to put into one person’s best guess.

Even worse, in real life the person making the design decisions is sometimes not a designer at all. It can be a product manager who identifies solutions rather than problems to his design staff; or a developer who has her own vision and (as a developer) the means to get what she wants implemented over the protests of the designer. But no matter who is giving the orders, the result tends to be a design that works really well – for that one person. Such designs are often neat and clever, but tend to not be resilient to changes in use or context. They can miss business concerns (which the product manager would know about), quality and cross-product issues (which QA might be aware of), technical constraints (known only to the engineers) or important but infrequent use cases (that expert users would never miss).

On the opposite end of the spectrum is the dreaded Design by Referendum (also called Design by Committee). In this system, a team of people must all agree to every design decision. While this solves the problem of insufficient input, Design by Referendum does not recognize the unique skills for elicitation, organization and synthesis that a good HCI designer can bring to the table.

Any group activity involves politics, and group-design is no exception. Decisions start getting made on the basis of fairness, of who-suggested-what, and did-everyone-get-something-they-want. You can often recognize group designs, because they tend to include a hodge-podge of features that lack unifying principles. And simple design is out of the question.

I have seen these two design approaches pitted against each other in arguments -- my boss doesn’t listen to me (the great designer) so instead we end up with some awful thing that was designed by committee!

But I think the best design is found in the middle ground, in what we call Collaborative Design. But how is that different?

I like to say that design is like breathing. You have to inhale before you can exhale. And you don’t stop at one breath – you have to keep breathing in and out to sustain yourself. Designers ‘inhale’ when they connect with the world; when they observe, listen, research, absorb information, and hear what others have to say. Designers ‘exhale’ when they communicate their thoughts and designs to others.

It’s not an accident that the word “inspiration” literally means “inhaling”.

Collaborative Design respects the role of the designer to gather, organize, and synthesize information; then to create or modify things based on this information. But it also respects the expertise of others; the reactions and knowledge of all the stakeholders are crucial to get a design from good to great. But that doesn’t mean we ask stakeholders other than the designer to design.

A few years ago my colleague Ian Hooper and I taught a one-day workshop on design methodologies at UPA. As part of our own preparation, we looked at how design was conducted in many different fields, and we kept finding the same idea – a loop of first gathering information, then evaluating and synthesizing (and dreaming!), followed by creating and communicating – then repeating this loop until done. Many disciplines have the idea of the “design review”, which goes by many names (“pin-up”, “charrette”, etc.) Some of these have are loosely defined; others have long-established methodologies.

A good design review starts with everyone agreeing to the common goal of helping improve the design. Then the designer presents the problem background and walks all the stakeholders through the design details. After that, the other stakeholders do their part; they ask questions, present what-if scenarios, and make suggestions for improvement. It is now the designer’s job to listen, to dig deeper into the scenarios and suggestions to make sure that all the reasons behind them are understood. Importantly, the designer doesn’t have to react to any of the feedback; he or she just needs to listen and absorb it. Multiple conflicting suggestions for improvement can be entertained at the same time, because no decisions are being made yet. When the exploration is done, the design review is over, and the designer can stop ‘inhaling’.

It’s time for the designer to reflect and synthesize what he or she has learned into an updated design. This doesn’t necessarily mean following particular suggestions (even when they come from the boss), but rather making changes that demonstrate you understood the reasoning behind the suggestion. Collaborative design is about respecting and utilizing everyone’s expertise, without losing the heart of design.

(Of course, you still need to validate your design by prototyping and usability testing… but that’s another article.)

Collaborative design, under whatever process name, is a very old idea, but still a good one. Like breathing. So make sure you always inhale --- you have to take a deep breath before you can sing.

*“Fiat” is from the Latin for “let it be done”, and has nothing to do with the Italian car company known for its beautiful designs.

Design Values: Ease of Use over Ease of Coding

by Ian Hooper

This is a follow-up article to John Schrag’s post on Design Values. The original post described the values we believe contribute to a healthy software design practice. This post looks deeper into the importance of supporting Ease of Use over Ease of Coding.

In my experience at other companies, I have seen that it is common practice to measure productivity in terms of the speed with which a new product or service can be brought to market. In practical terms this means that the faster a developer can complete a functional requirement, the better their efficiency. This provides a strong incentive for all involved in software development to streamline the process as much as possible. Under the constant pressure of tight deadlines, usability is seen as a threat to this "productivity," and as such in situations where the choice between ease of use and ease of coding is uncertain, ease of coding wins out.

However, actual studies have shown again and again how failure to properly consider the user experience can impact the bottom line. For example:

• An insurance company invested $3 million in an application to be used by independent agents to support them in selling this company’s products. However, agents simply refused to use the application, because the interface was "unlearnable and unusable."
• A financial services company had to scrap an application it had developed, when, shortly before implementation, developers doing a User Acceptance test found a fatal flaw in their assumptions about how data would be entered. By this time, it was too late to change the underlying structure, and the application never implemented.

The Importance of Designing Usable Systems, Dray & Associates © 1995 Association for Computing Machinery, Inc

Finding the right balance between efficient programming solutions and a positive user experience is where the challenge lies. There will be many times where these two goals will not come into conflict. When they do, embracing the design value of ease of use over ease of coding can help to resolve the tension between these two worthy principles.

Sometimes it is not immediately apparent which approach will result in the optimal solution for the users. To put it another way, it is not always clear where your efforts as a designer and user advocate will be most valuable. That is where early and iterative design can help identify the areas that will need extra attention or a different programming approach. Try doing some contextual task analysis or running a charrette as a quick way to rapidly spot the areas where the standard coding approach may not be sufficient. After that you can explore solutions by using any of the tools familiar to interaction designers such as card sorting, heuristic evaluation, personas, usability testing, use case scenarios and more.

An example of where we chose ease of use over ease of coding is in a recent product I was working on where we wanted to adopt a particular mode of navigation. The existing navigation scheme was unfamiliar to our target users and initial usability tests showed that this created a barrier to learning. We decided that we could match the navigation style used in Autodesk Revit, a program that our target users would be using. The application was modified to use the middle mouse button to pan the camera when pressed, or orbit the camera when pressed along with the Shift key. We made the new version of our application available to a select few in our target group. Having just implemented Revit-style navigation I was surprised to hear our users say “It’s too bad they don’t just use the same navigation as Revit”!??!

It turns out that our application only registered keystrokes sequentially – it could not detect simultaneous keys. The programmer set the system to orbit when the middle mouse button was pressed following the Shift key. This generally worked, provided you pressed the Shift key first, but if you just hold the mouse button down and toggle between Shift down and Shift up, our system would not recognize the change. This was a common behavior for many Revit users, and so for them, the navigation controls did not appear to be working.

Changing the system to allow for simultaneous keystrokes meant major changes to the application messaging system. The effort was significant, but failure to do this would have meant that our application would not have benefitted from the prior learning they had done on Revit. It may seem funny then that we were pleased with our next test release when nobody commented on the navigation at all. It just worked exactly as they expected.

In the end, we put the effort on the software and not on our users. By choosing ease of use over ease of coding we achieved our goals and created an experience that was natural and pleasing for our customers.

Deconstructing 'done': Achieving results over writing reports

by Desirée Sy

John Schrag wrote earlier about Design Values, and others have expanded on some of these ideas. This follow-up post looks at why we designers value Achieving Results over Writing Reports.

This isn’t so much a duality as it is two points on the continuum of design doneness. Writing reports is a part of what we do as designers, but it’s not an end to itself—just as an architect’s job isn’t complete when the blueprints are finalized. The exciting part is realizing that just as an architect’s job doesn’t start when the blueprints are being drafted, there is far more to an interaction designer’s job than writing specifications.

(http://farm1.static.flickr.com/170/371670597_f0d01ddf53.jpg)

What can we do as designers to achieve results that are better—faster, more vivid, more malleable, more effective—than pounding out a 279-page specification?

Is there a better way?

Let’s conduct some thought experiments with a few typical design activities and their deliverables.

Usability test

You’re conducting a usability test on a low-fidelity prototype, and the design idea fails to solve the primary design criteria set for it. What do you do to get this problem fixed?

1. During a design review session, you present the problem as part of the data in a PowerPoint scatter plot.
2. Pull some ideas out of an email thread with the whole team, after sending a link to your report of the whole test series.
3. Make changes to the paper prototype, in consultation with the developer who observed the session, and see if the next user can complete the task.

Answer (c) allows the team to move through design iterations more rapidly than writing comprehensive usability test reports and then reacting to those results allows. This also illustrates one of the concepts underlying the rapid iterative testing and evaluation (RITE) usability test methodology. Through personal experience, I can say that developers are more engaged and have a more complete understanding of underlying problems (rather than your proposed solution) when they participate in rapid design iterations.

Site visit

You're at a customer site, watching users pushing real data through the latest release of the software. You identify a critical issue that has an impact on the version currently in development. How do you convey this problem back to the team?

1. Three weeks later, as part of the “Problems” section of a field visit report that also incorporates data from the next two weeks of visits.
2. A month later, as part of a PowerPoint presentation to the team about all the field visits.
3. The next day, as an upload via FTP of a recording of a re-creation of the workflow with the current build version on your laptop with a voice-over describing the issue, supplemented by an example of the customer data.

The pros of producing a stream of “re-created highlights” for your team as the site visits progress are that you are highlighting the most important issues for the team (and therefore, for yourself) in a timely way, and you are keeping everyone engaged in the narrative of data collection for the full 2 weeks. Also, adding customer data provides context and data-driven requirements, helping the whole team to frame the problem more accurately.

The cons of such informal reporting are that you are paraphrasing the problem, and there is some loss in translation when you are using (for example) a different version of the software to demonstrate the problem. Also, there are going to be some issues that are only revealed as part of a pattern of observations over time.

And yet…haven’t we all been at a site where we saw one instance of a critical problem that doesn’t require another example for us to know that it needs addressing? Shouldn’t we try to find more timely (and less boring) ways to engage team members to fix these than by writing a report and insisting everyone read it and then do the right thing?

Convey UI design

You’ve finished part of a larger design and the low-fidelity prototype met all the design criteria in usability testing. The next step is to explain the design and the user experience acceptance criteria so that development can build it, and QA can test it. Which of these actions will most effectively help the team build the product?

1. Get a head start on section XXVI.34.ii (b) of the specification, which won’t be distributed until the whole design is complete.
2. Conduct a detailed design walkthrough of the entire UI design for developers, QA, and technical writers. Answer questions about the design solution, and capture the minutes of the meeting.
3. Write a short design brief for the part of the design that’s done, and link a series of these together later in a higher-level document.
4. Demo the smaller design piece using the final version of the low-fidelity prototype to complete a few common and critical tasks for developers, QA, and technical writers. Answer questions about the design problem. Capture the meeting, including the questions and answers, in brief design history notes. Keep a running series of these notes for all the design pieces.

I’ve found that a combination of (c) and (d) work much more effectively than (a) and (b). What makes the continuous (vs. complete) demonstration of workflow (vs. UI walkthrough) approach more effective is:

• It breaks the implementation down into more manageable, understandable pieces of work.
• It emphasizes the desired workflow as the outcome, rather than the minutia of the design, except where the visual or interaction design details have an important effect on the workflow.
Therefore, it puts the UI elements in the context of the user’s work and tasks for the whole team.
• It helps you understand where you failed to communicate design details to other team members, so that you can fix the design brief. Inevitably, when the team has a better understanding of the problem that you’re solving, someone improves your design.
• Most importantly, you’ve distributed the responsibility for achieving results to the entire team.

Making things happen

Designers are often frustrated because even after doing all the “right” design activities, and then distributing research reports, usability test reports, wireframes, and specifications, somehow the “right” solutions don’t end up in the product.

We’ve sometimes reacted by trying to fix the reports or enforce their authority — by forcing sign-offs; by presenting reports instead of getting team members to read them; by holding up coding until documents are complete; and in the case of specifications, by pushing more and more detail into more and more pages.

These fixes are symptoms of mistrust. Forcing compliance to read or act on reports is a breakdown of trust that the findings aren’t being taken seriously. Adding more content to reports is a breakdown of trust in the validity or completeness of the findings. I think most of us can think of times when doing either of these things hasn’t resulted in the “right” result, even though collectively, the team knew better.

Both approaches are based on the myth that a designer uncovers the user’s problems and creates the One True Solution, and then developers implement that vision (either successfully, or incompletely).

I think that this is the wrong way to infuse design thinking into products. Instead of forcing “their” solution on a team, designers should get as many team members as possible to understand users’ problems, and then get everyone engaged in solving those issues. Ultimately, the likelihood of getting results into a product increases exponentially for each team member who is committed to solving problems and owning the solution. Sometimes, writing reports isn’t the best way to do this.

Design Values: Complete Workflows over Long Feature Lists

by Lynn Miller on January 21, 2010

This is a follow-up article to John Schrag’s post on Design Values. The original post described the values we believe contribute to a healthy software design practice.  This post looks deeper into why we value Complete Workflows over Long Feature Lists.

As with the proceeding posts, stating that valuing complete workflows over long feature lists does not mean that we are against long feature lists. If you can get a long feature list added to a product and still maintain complete workflows – go for it!

However, implementing a long list of disjoint and disparate features does not serve the end-users well. Even implementing a list of coherent features is unacceptable if the final result doesn’t support a complete workflow. As in the image below, constructing walls and a door doesn’t make a livable house if there is no roof. I would think potential end-users of these houses would have traded off the windows-feature to get the roof-feature implemented.

Photographer: David Goldblatt
Incomplete houses, part of a stalled municipal development of 1000 houses.
Lady Grey, Eastern Cape, 5 August 2006

Long feature lists often result from gathering customers' proposed solutions. The customer requests, “I want a print menu” (a solution), instead of “I need to share my work” (a problem). But as Theodore Levitt of the Harvard Business School said, "People don't want to buy a quarter-inch drill. They want a quarter-inch hole."  If you implement features without understanding what the customer really wants, then you can end up with multiple solutions for the same problem, with none of them quite succeeding.

To get a complete workflow you need to understand what your customers end goal is. From this you can either design a complete solution that gets the user to where they want to go or you can identify the gaps that need to be addressed in your current solution.

The Kodak EasyShare digital cameras are a good example of understanding what users are trying to accomplish and providing a complete solution. Kodak recognized that their user’s end goal wasn’t to take pictures – but to share pictures. Their solution was to create an inexpensive digital camera that people could slip into a cradle, click a button, and share photos effortlessly through email. Kodak became the market share leader because of this.

Understanding your customers end goal isn’t very difficult. Talk to your customers about why they are asking for a certain feature, instead of blindly accepting the solution they propose. (Note: Their solution may turn out to the dead-on, but you are still better off if you know the reasoning behind adding the feature.) Try the 5 Whys technique – ask "Why?" until you get to the true cause of a particular situation or problem. Then you know that you are addressing the right issue – the one that will make your customers happy and won’t waste your development resources.

Design Values: Quality of Data over Ease of Data Collection

by Lynn Miller on November 21, 2009

This is a follow-up article to John Schrag’s post on Design Values. The original post described the values we believe contribute to a healthy software design practice.  This post looks deeper into why we value Quality of Data over Ease of Data Collection.

The first point that needs stating is that valuing quality of data over ease of data collection does not mean that we do not value Ease of Data Collection. It is important to make data collection as easy as possible in order to reduce the time commitment and financial costs, but cost reduction must not overshadow the need to use the right method for the data needed.

Let me give you an example.  A while ago, one of our product teams was trying to decide if they should continue to provide printed documentation to our customers.  The team created a survey which was sent out to the customer base. Surveys are low-cost and fast to create. At the same time, members of the product design team were on a string of customer site visits.  Customer visits are slow and fairly expensive. While at the sites, we looked around for the printed manuals or asked to see them. The results from the survey showed that 80% of our customers said they were using the printed documentation. The results from the customer visits showed that the majority of printed manuals were still in their original shrink wrap, locked up where end-users couldn’t access them, or were sitting in pristine condition on a shelf away from the users’ work area. The survey might have been low-cost and quick – but it got the wrong answer.  If we had continued to print the manuals based on the survey results, we would have spent far more money than the cost of the trips to the customer sites.

Getting quality data means matching the type of data collection to the type of data needed, and some types of data are harder to collect than others.

Gathering behavioral data seems to be most affected by the desire for easier data collection. Good quality behavioral data needs direct observation. Going for anything less than direct observation is sacrificing quality for ease of collection because of the factors that affect people when they are asked to self report their behavior.^1,2  When people are relying on memory, their answers can be distorted, incomplete, or biased.³

Surveys are the easiest form of data collection and thus the most abused. How often have you been asked to fill in a survey and found that you couldn’t answer a question and there was no way to indicate this? Sometimes it is just poorly worded questions, but often it is because the information should be gathered using a different technique.

For example, a well-known store sent me a survey a few weeks after I had made a purchase there. It had this question on it:  How many minutes did you stand in the check-out line? How could I possibly remember that with any degree of accuracy? If the store was considering making changes based on the data they were gathering, they would have been better off sending people into the stores to time the check-out line. Otherwise they might make changes based on my saying I stood in line for an hour, when really it only felt that way.

Quality data is data that allows you to make the right decisions. To get that data, UX professionals should be willing to put in the amount of work necessary and not be tempted to “get what they can” due to time or resource constrictions. Going for fast or cheap now can lead to expensive changes later on.

1 Blair, Edward and Scot Burton (1987). Cognitive processes used by survey respondents to answer behavioral frequency questions. Journal of Consumer Research 14: 280-288.
2 Bradburn, Norman M., Lance J. Rips and Steven K. Shevell (1987). Answering autobiographical questions: The impact of memory and inference on surveys. Science 236: 157(5).
3 Schacter, D. L. (1999). The seven sins of memory: Insights from psychology and cognitive neuroscience. American Psychology, 54, 182-203

Why User Centered

by Douglas Look on June 26, 2009

Here’s a compilation of insights that I gathered from colleagues and friends put together as a Pecha Kucha style presentation.  I originally put this together as part of a lecture for a design planning workshop at the Institute of Design—tried to share some real-world insights around why user centered design is important, how to gain support, and various tips and tricks offered up by professionals in the field. 

Click here for the presentation: http://www.youtube.com/watch?v=tITPvAkfk2g