Designing the User Experience at Autodesk

The Blog: User Centered Design Tool?

By Erik Egbertson on December 3, 2009

Many professionals use blogs as a two-way communication tool, reaching out to users and asking for their feedback, thereby becoming more user centered.  Keeping content updated is paramount in attracting and retaining a regular blog audience, but how can designers leverage a blog to support their projects rather than letting the blog steal time away from design activities?  Can a blog be used similarly to a traditional focus group in the design process?  Can it answer new questions or evaluate design proposals?

As a designer, I often need to validate some functional requirements or get customer feedback on a set of high-level design approaches. In some instances, I may be looking for something as simple as a name for an element or function that defies classification. I see these situations as opportunities to post to the blog and solicit feedback.

In order for the blog to be an effective UCD tool, I’ll first introduce the design problem with a scenario, key features, and related tasks. The second part of the post will explain the options and/or pose questions to the readers, encouraging them to use blog comments for further discussion. Next, I’ll moderate the responses and counter-responses that follow any individual’s blog comment, just as I would in a focus group. This exchange can provide useful information such as new use cases, unforeseen problems, or requirements essential to the emerging design.

In my experience, there are pros and cons to using a blog as a design tool:

Benefits

• Wide audience – A blog has the potential to reach hundreds of users knowledgeable in the topics of interest. For example, readers of the Inside the Factory blog span over 90 countries and account for hundreds of views per post.
• Speed – You can post an entry and begin gathering information from it in a short amount of time. Responses often start coming in the same day, allowing the design process to proceed quickly. Within 24 hours, you’ll have a clearer picture of the user’s mental model.
• Frankness – Users are generally much more comfortable being critical on-line than in larger groups or in face-to-face meetings. You may have to watch for cases where one user seems to influence others, but it’s more common to see users taking opposing positions and leveraging different arguments related to the problem. Again, this mirrors the interactions in a good focus group.

Risks

• Qualifying individuals – Unlike the selection process for forming a focus group, blog interaction is uncontrolled. You can’t screen individuals to confirm that they reflect specific characteristics of your personas or that they have the requisite experience on this design problem. If your inquiry applies to a broad range of potential users, you probably don’t need to worry about qualifying users. If you do need to qualify individuals, though, it may be possible to match them with demographic data gathered from previous surveys or test sessions. Another option is to direct the readers to a more formal survey at the end of your post, using the survey to screen for particular characteristics.
• Disclosure – Because blog readers are not under a non-disclosure agreement, use careful language when discussing anything that is a work in progress. In special cases, you may add some “fine print” disclaimers at the end of the post that specify the discussion in the blog as separate from a promise of future delivery. Users typically understand this, and they value the exchange of ideas and the possibility of informing future designs.
• Blog jacking – Sometimes a customer talks about something other than the topic of your post (perhaps an existing feature or some unanswered wish), and may post links from other forums supporting their case. Although this seems disruptive, you can view it as another form of feedback: you can reflect back what you’ve heard from the individual who goes off topic, record the issue internally, and then gently nudge the conversation back to the topic at hand. These are the times when you fulfill the role of moderator.

In summary, users appreciate the directness and two-way exchange that a blog can offer. For the designer, a blog can be a useful tool to solicit feedback to inform your designs, but should not replace more thorough upfront research or user testing.

Are there any other designers out there who solicit design feedback through their blogs? What are your experiences?

Design Values: Quality of Data over Ease of Data Collection

by Lynn Miller on November 21, 2009

This is a follow-up article to John Schrag’s post on Design Values. The original post described the values we believe contribute to a healthy software design practice.  This post looks deeper into why we value Quality of Data over Ease of Data Collection.

The first point that needs stating is that valuing quality of data over ease of data collection does not mean that we do not value Ease of Data Collection. It is important to make data collection as easy as possible in order to reduce the time commitment and financial costs, but cost reduction must not overshadow the need to use the right method for the data needed.

Let me give you an example.  A while ago, one of our product teams was trying to decide if they should continue to provide printed documentation to our customers.  The team created a survey which was sent out to the customer base. Surveys are low-cost and fast to create. At the same time, members of the product design team were on a string of customer site visits.  Customer visits are slow and fairly expensive. While at the sites, we looked around for the printed manuals or asked to see them. The results from the survey showed that 80% of our customers said they were using the printed documentation. The results from the customer visits showed that the majority of printed manuals were still in their original shrink wrap, locked up where end-users couldn’t access them, or were sitting in pristine condition on a shelf away from the users’ work area. The survey might have been low-cost and quick – but it got the wrong answer.  If we had continued to print the manuals based on the survey results, we would have spent far more money than the cost of the trips to the customer sites.

Getting quality data means matching the type of data collection to the type of data needed, and some types of data are harder to collect than others.

Gathering behavioral data seems to be most affected by the desire for easier data collection. Good quality behavioral data needs direct observation. Going for anything less than direct observation is sacrificing quality for ease of collection because of the factors that affect people when they are asked to self report their behavior.^1,2  When people are relying on memory, their answers can be distorted, incomplete, or biased.³

Surveys are the easiest form of data collection and thus the most abused. How often have you been asked to fill in a survey and found that you couldn’t answer a question and there was no way to indicate this? Sometimes it is just poorly worded questions, but often it is because the information should be gathered using a different technique.

For example, a well-known store sent me a survey a few weeks after I had made a purchase there. It had this question on it:  How many minutes did you stand in the check-out line? How could I possibly remember that with any degree of accuracy? If the store was considering making changes based on the data they were gathering, they would have been better off sending people into the stores to time the check-out line. Otherwise they might make changes based on my saying I stood in line for an hour, when really it only felt that way.

Quality data is data that allows you to make the right decisions. To get that data, UX professionals should be willing to put in the amount of work necessary and not be tempted to “get what they can” due to time or resource constrictions. Going for fast or cheap now can lead to expensive changes later on.

1 Blair, Edward and Scot Burton (1987). Cognitive processes used by survey respondents to answer behavioral frequency questions. Journal of Consumer Research 14: 280-288.
2 Bradburn, Norman M., Lance J. Rips and Steven K. Shevell (1987). Answering autobiographical questions: The impact of memory and inference on surveys. Science 236: 157(5).
3 Schacter, D. L. (1999). The seven sins of memory: Insights from psychology and cognitive neuroscience. American Psychology, 54, 182-203

Design Values: Validated Data over Expert Opinion

by John Schrag on October 16, 2009

Design Values:  Validated Data over Expert Opinion

A while ago, I posted an article (Values in Software Design Practice) describing what values my co-workers and I believe contribute to a healthy software design practice.  At that time, I promised to go into more details on the individual items of that list.  This posting describes why we value Validated Data over Expert Opinion.

Some people mistook my earlier post to be a criticism of experts, which was not intended at all.  I value expert opinion highly, and my employer values the software design expertise of my team.  The question we were looking at is this:  when you are making design choices that require information, is design and domain expertise enough to come to the right answers?

A few years ago, some of my colleagues decided to run a first-experience study on one of our software packages.  The purpose of such a study is to gain an understanding of what our users go through in their first hour of use.  What do they experience?  Where do they get stuck?  How far can they get in the software?  What are their learning strategies?  As a side experiment, my colleagues asked several experts in the company for their expert opinion as to what problems users would run into, and compiled them into a list.  (These experts included the software designers, domain experts, and the people who trained users on the software.)  Then my colleagues ran their study, observing sixteen people using this software for the first time, and made a list of the problems that users actually ran into.  The result?

There was not one common item on the two lists.

We repeated this experiment a year or two later on a different product, and got exactly the same result.  While our experts knew a great deal, they were not good at predicting the behaviour of novice users.   Perhaps this should surprise no one; experts cannot ‘unlearn’ what they know to get into a beginner’s mindset.  (But ask yourself this:  when you design training materials for your software, how do you decide what goes in them?)

Here’s another story:  working at a previous company, I had the opportunity to visit with one of our engineering customers and learn about their workflows.  We arrived early, and spent the first hour in a meeting room with a lead engineer who described her workflow in great detail, and explained the problem areas as she saw them.  After that, we had the opportunity to shadow her for a day as she worked.  When we actually observed her process, we saw that it was markedly different than what she had described.  She was spending about half her time doing long tasks that had never been mentioned in her workflow description, because to her they were not part of her job – just things she had to do in order to be able to do her job.  But those tasks (which were repetitive make-work) represented a huge opportunity for us to improve her workflow.  This is a common problem – people are very bad at self-report.  If you need to collect behavioral information, you cannot rely on people to be experts even on themselves.

These stories provide two examples where validated data – observation of users at work – was provably clearly better than expert opinion.  But what about design expertise?  Does being a ‘design expert’ give you what you need to make the right choices?  What about things that you just can’t get validated data for?

While good design practices are valuable, the fact remains that every design decision we make is a guess.

In most cases, it is an informed guess.  If we’ve done our research and taken time to think through the problems we are trying to solve, then we can probably make pretty good guesses, and get our interface designs right much of the time. (Certainly an expert will do a better job than a non-expert!)   But we’re still guessing.  Will our users be able to find the functions they need?  Will they be able to use what they find?  Will it satisfy their needs?   We can’t know for sure if our designs work until they’ve been tested by real users doing real tasks.   Once your design has passed the goals you set for it in a usability test, you are no longer guessing.  You know you have a good design – or, if not, you know what you need to change.

And that is the short answer.  We value validated data over expert opinion because we value knowledge over guesswork – even good guesswork.  Sometimes the data surprises us – and that’s usually where opportunity lies.

A Case Study in Large-Application Usability Benchmarking

by Melissa Dawe on October 5, 2009

As a user researcher with a primarily qualitative background, I have to confess that when I was asked to conduct a usability benchmark study on AutoCAD, I was not exactly jumping out of my chair. Frankly, I was wary of the quantitative emphasis of the method and the proposal to reduce the whole user experience down to a single number. I was also more than slightly nervous about designing a benchmark study for a product as complex as AutoCAD.
 
Despite my trepidation, I ultimately developed respect and appreciation for usability benchmarking. I would even call myself a convert. I observed the power in a simple usability scorecard (that even execs can understand!) to raise awareness of usability in an organization. I also found that the data collected in a benchmark study can be more than just numbers, and qualitative observations can bring focus to long-existing usability issues that might otherwise get lost.
 
What is a Usability Benchmark?
 
Whereas a typical usability study focuses on specific features or aspects of an application, a usability benchmark measures the overall usability of the app. It provides a single score that can then be used to compare usability of the entire application release after release.
 
To design the AutoCAD usability benchmark we relied heavily on Sauro and Kindlund's excellent work in quantifying usability with the SUM metric (see www.measuringusability.com). The metric combines three aspects of usability: effectiveness, efficiency and satisfaction.  These measures are standardized into z-scores and then combined into a single score.

Sounds simple enough, right? Nothing is simple when you’re talking about an application as complex as AutoCAD. We encountered three major hurdles:

1. Deciding on what type of users to run through the benchmark
2. Coming up with a good set of benchmark tasks
3. Communicating results using the SUM metric

Hurdle One: Determining the Participant Profile
 
Selecting target users for a benchmark study is important because it greatly affects the tasks you choose and how you measure success (e.g. how you determine your task target times). Are you focusing on users who have never used the product (i.e. measuring first experience)? Are you targeting expert users in order to focus on changes in efficiency?
 
In addition to skill level, your users may also vary in other important ways. Like other widely-used, large feature set applications such as Microsoft Word or Excel, AutoCAD has a diverse range of users. It has been used for 25 years by millions of people to design very different things, from skyscrapers to boats to skateboards.
 
I assembled an advisory team and we decided on two target user profiles: experienced AEC AutoCAD users, and students or novice users who'd received some training in AutoCAD. We had a different “First Experience” study for new users without training.
 
Hurdle Two: Developing a Set of Benchmark Tasks
 
Deciding on the right set of tasks to benchmark was a daunting challenge. AutoCAD is a complex, professional application with 25 years under its belt - it has thousands of commands and tools and semester-length college courses dedicated to learning it. How can you benchmark the entire product in a reasonable-length user session? I toyed with the idea of serving free unlimited coffee and challenging users to stay awake for 24 hours, but then remembered the ethics section of my behavioral psychology class and went with a more humane approach. 
 
Ultimately we developed a dozen high-level tasks that covered core areas of the product. The tasks were piloted and refined so that they could be reasonably completed in a 3-hour session. The tasks did not prescribe which features or tools to use since the tasks needed to remain relevant in future releases and accommodate UI changes and new features.  
 
Once we'd perfected our tasks, we weren't quite done - we needed to determine success criteria for each task. Should we consider a typo a task failure? At the end of the day you are going to say "n people succeeded in this task," so it is crucial you have team buy-in on how you define success. For our tasks, success was output-oriented -- users could take any path through the interface, as long as they sufficiently reproduced the goal drawing.  
 
Hurdle Three: Communicating results using the SUM Metric
 
The biggest challenge with using SUM came when we were getting ready to present our data. The SUM score is represented as a z-score, which ranges from -4 to +4. We had a table of scores displaying 0.4, 0.8, 1.2, etc. We realized that if we presented to our management, "AutoCAD plotting received a usability score of 0.4!" we were sure to get a room full of blank stares. 
 
To make the scores easier to understand, we converted the z-scores back to their percentage score equivalent (MS Excel's normsdist will do this for you), and then we had a new problem. When the mean is equal to the target (i.e. the target is met), the z-score is 0. A z-score of 0 translates to 50%, which is not exactly a compelling score to indicate we met our usability goal. To address this, we re-translated and adjusted our target numbers to equal minimum acceptable usability, which is more meaningful as the 50% value. Thus, what we considered good usability was in the 80% to 90% range – a more meaningful translation for our broad audience.
 
Benefits of Usability Benchmarking, or "Why I'm a convert"
 
Despite my early wariness, I believe that developing a usability benchmark is absolutely worth the effort. For one, it has the potential to raise awareness and impact of usability in an organization – by providing a single score, usability can be tracked by high level management along with performance, sales, and other product metrics.
 
But the benchmark study had a hidden benefit that is perhaps even more important than the single score. For three straight hours (resulting in a total of over 50 hours of video), we had the opportunity to observe users working in AutoCAD in an uninterrupted, fairly natural way. The high-level tasks gave users a lot of room to choose how to work, giving us insight into the paths users take and the issues they experience. We were able to uncover patterns of stumbling blocks that revealed small bugs or design problems that had not been reported or whose importance wasn’t previously understood. We compiled a "Top 10 List" of usability issues encountered by each user profile (experts and novices), and these have been added to the roadmap for future releases.
 
I can't say that I was disappointed when the benchmark study was over and I could go back to doing site visits and qualitative research. But I've become convinced that usability benchmarking is an important method in a user research program, as it measures and communicates the usability of a product in a way the entire organization can understand. It's not as sexy as video clips and user quotes, but it gives you some lovely bar charts and sometimes bar charts are just what your audience needs to invest more in usability. And after all, isn’t communicating in your audience’s language what usability is all about?

Design Partners: Passing on the Knowledge of UX

by Rebecca Richkus on September 24, 2009

Last year, I was fortunate to be part of a design collaboration for a major feature in AutoCAD MEP.  I was the Interaction Designer, and Gregg Stanley, a Quality Assurance Analyst and a Subject Matter Expert (SME) on mechanical piping design (the feature theme), was to work with me to provide feedback and to help document the design. Our collaboration resulted in a great feature for the product, and provided a concrete example of how involving others in the design process equates to a better overall design. I decided to submit a proposal to the Usability Professionals’ Association (UPA) 2009 conference, with the idea that our collaboration experience would educate and benefit others.

Gregg was an SME on piping and on our software, but knew little about user-centered design processes. Certainly he’d seen our design documents, and was aware of the value of usability testing. But as a QA Analyst, he hadn’t been involved in early design work. Meanwhile, I had typically done requirements analysis and conceptual design without much input from anyone outside the User Experience group.  We both found the experience of working together valuable and rewarding. But as we brainstormed ways to present about our collaboration, I became aware of the impact of our work together, particularly on Gregg’s view of the design process.

As we drafted our UPA proposal, we each wrote our version of “What happened during the project and what did you learn from it,” and then passed it to the other. When we compared notes, I realized that Gregg didn’t understand what the heck I was doing at the beginning of our project. He mentioned being frustrated, not knowing why I was so often asking “Why? How? How often? When?” about user scenarios he had identified. He also couldn’t understand why I wanted to keep generating solution ideas. Wasn’t one enough?  While we worked very well through those process phases, the why of it all was mystifying to him. As the presentation progressed, I became fascinated with how he went from frustration to being able to speak authoritatively about our process to others. By the end, he understood that generating many solutions helps identify the best one. He knew that early focus on the abstract requirements helps to ensure that the solution is more holistic, rather than a patchwork of fixes. He also appreciated how the use of personas helps to focus questions. Our persona, Paula, began to more concretely represent a real, typical user, with specific design needs different than those Gregg had envisioned.

I often hear from other Design and Usability practitioners that their counterparts in other functional groups, particularly development, think that designers simply write specifications. We in the design field understand that the documents we produce are a result of the analysis and design work we’ve done (or possibly a tool to help us progress in a design). But it’s not always clear to others how or why we got to a documented solution.  So I wondered, “At what point did Gregg grasp the design process? And how can we, in the UX group, help that happen for others in our organization?”

To answer that, Gregg and I worked to identify the process that unfolded from our collaboration, which included not only the steps in the design process but also the forming of a working relationship. Both aspects helped him understand where I was coming from.

The two main drivers for a successful relationship were to respect each other’s opinion and to use active listening to understand what the other was saying. Gregg had many detailed solution ideas at the beginning of our project. At that point, I didn’t want to decide on a solution, but wanted to understand what problem he saw and why he felt those solutions were important. By actively listening, I often learned of a new requirement or a flaw in the current product that had eluded me, and Gregg felt his opinion was heard and valued (and it was!). We took those good ideas and wrote them down so that they weren’t lost, and both Gregg and I could then move on to thinking of other possible solutions.  Many times we came back and used the initial ideas, but for the times we didn’t use Gregg’s idea, this process allowed him to understand why a different solution was needed.

The second area of success was having Gregg involved from the beginning. In prior projects, other team members came onboard only after the requirements analysis work was mostly complete or when conceptual design work was far along. My teams would often help in finalizing designs without a full appreciation of the users’ problems we were trying to solve. In this case, the fact that Gregg helped write the Requirements Analysis document—and I helped him understand the separation of user requirements (what users need to be able to do with the software) from more specific design requirements (how our software will do something)—meant he and I were focused on the correct problems. Our vision of what the overall solution would be came from a shared understanding of the requirements, and we returned to this vision repeatedly as we generated conceptual design ideas and created the detailed design documents.

This experience highlighted for me that relationship-building and involvement in all parts of the design process are key factors when educating team members as to what user experience and design are about. In turn, these feature teams can create even better designs, and the team members involved will pass on this new knowledge in their next project, hopefully adding to better designs in the future. So grab your neighbor developer, QA analyst, or writer and sign them up to start working on a design with you!

Have you already had a similar experience? What tips can you share for those attempting to gain more buy-in and involvement from other functional groups?

Usability Testing with User Proxies: When is "Close" Close Enough?

by Desirée Sy on August 14, 2009

How can we designers get valid feedback from more design iterations in less time? One bottleneck in the design flow is finding a steady stream of usability testers. Between the extremes of the perfect (an actual user, on site) and the unacceptable (the developer who's coding the feature), lies the grey zone of user proxies. Can you use internal employees with relevant domain knowledge to usability test your products, and still get valid data?

The answer, as with all design questions: It depends.

We have successfully used internal employees as usability test participants on products where we have an agile user-centered design methodology (see “Adapting Usability Investigations for Agile User-Centered Design” in the Journal of Usability Studies for a lot more detail on our agile methods). Since in agile design the prototypes are incrementally and progressively getting closer to the end product, we took the approach of using tasks and testers that also incrementally and progressively got closer to real work done by actual users.

With agile, our designs are being iterated in an incremental series of “mini designs,” or design chunks, so the earliest prototypes do not allow people to complete full workflows, but only separate operation-level functionality.  It is extremely hard to usability test these early “mini prototypes” externally because they do too little, have very little task-level context, and require extensive intervention from a facilitator to operate. 

An example of one such early prototype might test the dragging algorithm for a zoom operation (where users zoom towards the click-point as opposed to the centre of the screen). While this interaction requires a high-fidelity prototype to test it, it's also possible to create low-fidelity prototypes to test granular interactions.

We usability test these types of “mini prototypes” internally, with extremely artificial forced tasks. For example, to test a brush resizing algorithm for a 2D paint application, I asked internal testers to match the brush size of different brushstrokes I'd prepared on a canvas using different prototypes.  I switched between different prototypes for the testers since I was not testing discoverability of the function, but instead the smoothness and efficacy of the different algorithms. 

Recruiting internal testers requires just as much thought as recruiting externally. You need to understand what you are testing and select testers accordingly. 

We recruit internal testers from Quality Assurance, the Help Desk (Support), and our pool of subject matter experts.  We do not use developers or other UX designers.  All internal testers must work on a different product than the one being tested to minimize bias. Also, all internal testers must have particular qualities in common with our target users: in the brush resizing example, they needed to be people who sketched (so, for example, a QA person who was an expert in video rotoscoping but did not sketch was not recruited – just as an external person would not be recruited without hand sketching experience).

Our sessions with internal users are very fast, not conducted in a lab, and not recorded. Internal staff don’t want videos of them failing a task to be seen by people who might mistakenly see this as a sign of domain incompetence instead of a design failure.
 
Once the foundational interactions are locked down, we combine prototypes to enable more complete interactions that relate to a user activity. These more complete “workflow prototypes” are tested with external users. Initially, these users may be other user proxies (for example, students studying animation, industrial design, or other sketch-intensive fields who may not necessarily be users of the product). Eventually, we'll test refined workflow prototypes with actual users.

With some thought, I believe you can use internal employees and other user proxies successfully as part of an overall design validation strategy, whether or not you are using an agile development method.  The main things to consider are:

• your users' specialized characteristics or environmental contexts
• how different these are from those of your user proxies
• how relevant particular characteristics or contexts are to using a given prototype.

Of course, if what you are testing is heavily dependent on a particular characteristic or context, then you need to usability test with real folks doing actual work—sometimes in the field. But more often than not, there are design decisions that don't depend on all user characteristics. By using internal employees and other user proxies to usability test interactions in early design work, you can increase your design velocity without sacrificing the validity of the feedback.

Sketching Design

by Yan Schober on August 7, 2009

In his book, Sketching User Experience, Bill Buxton advocates for sketching as a technique and process that can put experience front and center in design. I am a big fan of sketching and use the techniques I first learned in architecture school for interaction design. In this post, I’m going to give you a quick peek at the types of sketches I typically create in my design process with the hope that it will inspire you to try sketching for you next project.

First Sketch
After the initial user research, the first sketch gets me thinking about the big ideas.

Blank pages are intimidating, so I start each project by doodling a few things that are known entities, really anything to get me going. I prefer to do this on loose sheets of paper rather than in my sketchbook.  This way, I stay loose and uninhibited. I can indulge in doing lots of fast, ugly sketches without worrying that they’ll clutter up my sketchbook. I keep a stack of papers next to my desk which I burn through quickly.  It’s like warming up before a long workout.

A trick that usually works for me is to start this when I don’t have much time, which could be on the train or at the end of a long day. It sounds counterintuitive, but knowing that I only have a short amount of time actually releases my creativity. I have no expectation that anything will get done so I don’t censor my ideas. Most important, it seeds my thinking about the project and a brilliant idea may come to me on the drive home or in the shower the next day.

Here is an example of a first sketch:

  

A bit ugly, perhaps, but it served its purpose: it got me thinking!

Test Sketch
A test sketch helps me explore my initial ideas.

Once a few promising ideas have surfaced, I redraw the original idea, but larger and with a finer pen. This sketch is all about taking my initial ideas on a test drive. It entails adding more detail and running the ideas through various primary workflows to see if they hold up. Also, an important part of this stage is sharing the ideas with other designers. I incorporate feedback into the next round of sketches, enriching the design with different points of views and accelerating its evolution. Here is an example:

 

The goal here is to begin to understand the user’s main workflows.

Design Development Sketch
Once an idea has passed the initial stress test, it is ready for further design development.

I use design development sketches to flush out ideas that continue to show promise after the initial phases of sketching. I start working on a more detailed level to see if all the pieces are coming together. At this point, I often jump into Illustrator or Photoshop to test out some basic proportional relationships or to create simple wireframes. Here is an example of a hand-drawn design development sketch:

At this point, it’s starting to look like a real application.

Presentation Sketch
The presentation sketch quickly communicates ideas to non-designers.

If an idea makes it past the design development phase, I often draw up some quick, more polished sketches that I can show to the rest of the team to get more in-depth feedback. The main goal here is to communicate my ideas to a wider audience such as product management and software development to get feedback on feasibility and cost. Most important, I can get buy-in for the project’s general design direction. Sketches look unfinished so people are not afraid to speak their mind, encouraging a productive dialog.  Here is an example:

Note the high level of detail, and how much closer this is to a specification.

Conclusion
As is often the case in design, sketching is rarely a linear process. I often jump from test sketch to presentation sketch just to get some feedback and then start over because I realize that the idea was not good. I always try to come up with at least 2 or 3 early concepts so I can see which one develops best. Sketching is quick so it allows you to spend less time creating functioning prototypes and more time developing ideas.

The most important thing to keep in mind is that none of these sketches have to be pretty. I often hear people say, ‘I wish I could sketch’. In my mind, this misses the point of sketching. Design sketching is not about the physical artifact, but primarily about the mental exercise that it forces you to go through and the communication it facilitates with others. A quote from my RISD professor sums it up nicely, ”Beautiful drawings can get in the way of good design because people fall in love with the drawing, not the idea it represents.“ I encourage everyone who has never sketched before to give it a try.

The Problem with Problems

by Chauncey Wilson on July 20, 2009

User Experience and usability practitioners are on a continuous hunt for problems that plague our users.  This seems straightforward – find problems from testing, user forums, observation, and other methods, prioritize the problems, and generate solutions that eliminate the complaint.  However, some events that we call problems in one context may not be problems in another. Take a simple example of a drop-down list that allows you to choose a single item from a longer list.  This control (assuming that it is well labeled and you can press a letter to jump to the first instance of that letter in the list), may not present a problem with 5, 50, or even 500 items; but what if you have 50,000 items with similar beginnings – then you have a scalability problem.  Some user interface controls work well until you hit a particular threshold – here more than 500 items – and then suddenly, a simple act of choosing an item becomes an irritating problem. 

In complex products, it is sometimes hard to know when a problem begins.  Some problems might begin weeks before you actually encounter any negative event.  An example of this might be a choice in a properties dialog that you set during installation that you forgot about and then weeks later, when you are working against a deadline, doing something simple with your tool, BAM! CRASH! BOOM!   Your data are destroyed and you have no backup.  You talk about a crash as the problem, but it was really the result of an isolated change made earlier.

One of the conundrums of user experience design is that we have to design systems that accommodate different user groups, from first-time users to gurus. A system that delights the first-time user by guiding her through tasks (high learnability) might frustrate the guru who expects shortcuts, ways to combine functions through macros, or a command line. So user experience can influence whether a particular event is a problem or not.

Age makes a difference in determining whether a design feature is a problem or not.  I recently turned 59 and find it almost impossible to read the registration codes on those plastic containers for software DVDs.  Colleagues in their 20s and 30s may not find the tiny 7 point text a problem, but I surely do (and carry a magnifying glass around in my computer bag).

What other contextual issues affect whether something is perceived as a problem?  Weather can affect whether something is a problem.  My iPhone worked well for the first 5 months, but then one day I wore gloves and discovered that trying to dial a number with gloves didn’t work with the capacitive touch screen.  Apple has filed a patent for gloves that transmit a charge in cold weather to mitigate this problem (see http://www.appleinsider.com/articles/09/01/01/apple_files_for_patent_on_winter_friendly_iphone_gloves.html ) .  Years ago, I worked on an early computer system that police could use while chasing evil doers.  The system worked well in static testing, but when the laptop was used in simulated high speed chases on bumpy roads, it was a disaster.  The officer trying to operate the system had trouble typing in the license plate and reading the output (THIS MAN IS DANGEROUS AND HAS POWERFUL WEAPONS).

So, what constitutes a real usability problem is often dependent on context and perspective and what is a real problem to one user might be a delighter to another.  Some questions that you might consider to understand (and anticipate) the impact of context on how events in user research are labeled include:

• What is the range of environments in which your product will be used?
• How does performance interact with usability?
• What activities become difficult when you scale something up, like the number of collaborators working with the same data?  Something that works well with a few people may become a monster when you involve dozens of people working on the same thing.
• How do your data samples affect whether tasks are easy or hard?  Do you use large samples with dirty data (the real world) or toy samples with clean data?
• Did you consider the breadth of users in your research?  Did you focus too much on learnability to make your novice users happy, only to irritate your expert users?

For some other thoughts on this topic, you can read a related article that I wrote in 2007 for the ACM magazine, interactions:

Wilson, C. E. 2007. The problem with usability problems: context is critical. interactions 14, 5 (Sep. 2007), 46-ff.

Values in Software Design Practice

by John Schrag on July 10, 2009

Every user experience (UX) designer who practices in a corporate setting knows the breathless whirlwind that is modern business.  We designers manage relationships with developers, business managers, and customers, and still have a full-time production role researching, designing and validating features and interactions.  We rarely have enough time to do everything we should, and therefore have to carefully choose where to spend our time and resources.  Some designers don't even have the luxury of such choice, and instead are tasked with specific deliverables, which are sometimes not the best use of their time.  In the face of this industry-wide problem, it's no surprise that dysfunctional design practices can arise and even become so common that they are seen as normal, or at least inevitable.  But I don't believe they are.

In 2001, a group of software developers, development managers, and consultants created the Agile Manifesto, a document that outlined a set of simple values that they believed were the foundation of good development practices.  What is interesting about this document is that it makes no reference to specific practices or buzzwords -- instead, it outlines principles and values against which any particular practice can be evaluated.

In the same spirit, my co-workers and I spent several hours considering the question of UX design practice.  We weren't considering the question of what makes good design;  there are innumerable books on that topic.  We were considering the question of what separates a healthy, effective design practice from the horror stories we hear about when talking to others in the industry.  We came up with two lists;  the first list (below) talks about doing certain actions in the right order.  Reading the list, these orderings may sound obvious, but they are by no means standard practice:

Setting Goals before Taking Action
Understand Problems before Generating Solutions
Designing before Writing Design Documents
Validating Designs before Investing in Code
Steak before Sizzle

The second list is one of relative values.  While all of the items on both sides of this list have value, we value the items on the left more than the items on the right:

We Value:

Validated Data over Expert Opinion
Quality of Data over Ease of Data Collection
Complete Workflows over Long Feature Lists
Achieving Results over Writing Reports
Collaborative Design over Design by Referendum or Design by Fiat
Ease of Use over Ease of Coding
Well-designed Critical and Common Workflows over Complete Coverage of Every Possible Workflow

Of course, a few words in a list doesn't capture these ideas all that well.  In future blog posts, I'm going to talk about each of these items in more depth, show examples and talk about what can be done to keep your design practice healthy, even in the face of the whirlwind.

Do you have more items for our list?  We'd love to hear from you.

Why User Centered

by Douglas Look on June 26, 2009

Here’s a compilation of insights that I gathered from colleagues and friends put together as a Pecha Kucha style presentation.  I originally put this together as part of a lecture for a design planning workshop at the Institute of Design—tried to share some real-world insights around why user centered design is important, how to gain support, and various tips and tricks offered up by professionals in the field. 

Click here for the presentation: http://www.youtube.com/watch?v=tITPvAkfk2g