Designing the User Experience at Autodesk

Managing UXD and UA for Multiple Agile Teams

By Karen Smith & Joshua Ledwell

While the transition to Agile software development means big changes for everyone who’s “going Agile,” it has the potential to have the greatest impact on User Experience Designers (UXD) and User Assistance (UA) writers. Although Agile teaches that team members should be fully dedicated to one team, the budget constraints of many teams dictate that some people are shared across more than one team – and those people are often UXD and UA. As more project teams transition to Agile software development and strive to deliver more customer value in shorter, sprint-like chunks, sharing our experiences and best practices with others in UXD and UA is now more important than ever.

At the outset, participation on multiple Agile teams may seem overwhelming, perhaps even unmanageable. Over the last year and a half of working with a variety of Agile teams, we’ve learned that when UXD and UA work together, and have a shared vision with their Agile teams, the process becomes more manageable.

To ease the difficulty of working across multiple Agile teams and work more productively, here are some best practices for UXD and UA:

• Show your Agile team that you are a shared service and that your time is divided between multiple projects and assignments. This helps them understand why you may not be able to attend every standup meeting.
• Volunteer to help with other needs of the team. For example, offer to take notes during a user research or feature usability session, or offer to help with testing.  Agile is about breaking down traditional silos and accepting team responsibility for the success of the product.
• Even though Agile means a full schedule of meetings, UXD and UA should participate in non-Agile team activities, such a lunch outings, impromptu coffee chats, etc.  Getting to know your teammates pays off when you need help documenting a feature or understanding what design choices are possible.

In addition, here are some best practices specifically for UA:

• Prioritize the various Agile meetings. Daily standups, planning, and demos are the “must attend” meetings. UA should skip meetings that aren’t relevant or that don’t add value to the development of learning content.
• Track UA tasks separately from software development tasks. The biggest advantage to this approach is that the closing of a software development user story included in the definition of “Done” is not dependent on the completion of learning content. In addition, Agile team members can separately assign user stories and tasks to UA. If you’re using JIRA to track user stories, you can add links from the UA user story to related software development and QA user stories and tasks, making it easy to find the related information later on.
• Provide feedback to your Agile team on best practices. Getting involved in the team is the key to building trust and earning credibility. Set expectations for UA deliverables, identify where you can contribute to shared team objectives, and be clear about what you will contribute.
• When authoring learning content, don’t be afraid to write fiction. Then iterate, iterate, iterate.

 

To learn more about developing user assistance in an Agile development environment, check out these articles:

• Introduction to Agile Methodologies
• Agile for Technical Writers
• A Writer's Guide to Surviving Agile Software Development

 

Follow-up from "Agile UX Method Adaptation" half-day course at CHI 2012

by Desirée Sy and John Schrag

We would like to thank the attendees of the half-day course at the CHI 2012 conference, “Agile UX Method Adaptation,” for their energy, attention, and thoughtful questions.

As mentioned during the course, here are a collected set of references mentioned during the course, beginning with the refreshed series of slides:

• Slides from "Agile UX Method Adaptation"
  Desirée Sy and John Schrag, UPA 2011.
• “Adapting Usability Investigations for Agile User-Centered Design”
  Desirée Sy, JUS, May 2007. http://tiny.cc/agileUCD (or Sy_agile-ucd.pdf)
• “Case Study of Customer Input For a Successful Product”
  Lynn Miller. Agile 2005. 
• Agile manifesto
  http://agilemanifesto.org/
• Mike Cohn's site, http://www.mountaingoatsoftware.com/, has a lot of helpful information if you want to learn more about the Scrum flavour of Agile Development.
• “Interaction Designers and Agile Development: A Partnership”
  Lynn Miller. UPA 2006.  
• “Using Formative Usability Testing as a Fast UI Design Tool”
  John Schrag. UPA 2006
• “Using the RITE method to improve products; a definition and a case study”
  Michael C. Medlock, Dennis Wixon, Mark Terrano, Ramon L. Romero, Bill Fulton.
• “Formative usability investigations for  open-ended tasks”
  Desirée Sy. UPA 2006
• “Usability Testing with User Proxies: When is ‘Close’ Close Enough?”
  Desirée Sy. DUX blog.
• Recurring user studies
  “It's Not Rocket Surgery - First Fridays in the U.S. Government”
  Nicole Burton, Steve Krug et al. UPA 2010
• "Distributed Scrum Primer"
  Pete Deemer et al.
• "Taking Aim: The Power of User Experience Goals"
  John Schrag, UX magazine, Volume 7, Issue 1, 2008

 

Follow-up from "Agile UX Toolkit" tutorial at UPA 2011

by Desirée Sy and John Schrag

We would like to thank the attendees of our one-day tutorial at the Usability Professionals’ Association 2011 conference, “Agile UX Toolkit,” for their energy, participation, and terrific questions.

As mentioned during the tutorial, here are a collected set of references mentioned during the tutorial, beginning with the refreshed series of slides:

• Slides from "Agile UX Toolkit"
  Desirée Sy and John Schrag, UPA 2011.
• “Adapting Usability Investigations for Agile User-Centered Design”
  Desirée Sy, JUS, May 2007. http://tiny.cc/agileUCD [OR link: Sy_agile-ucd.pdf]
• “Case Study of Customer Input For a Successful Product”
  Lynn Miller. Agile 2005.  http://tinyurl.com/4xx8zle
• Agile manifesto
  http://agilemanifesto.org/
• “Interaction Designers and Agile Development: A Partnership”
  Lynn Miller. UPA 2006.  [link: Miller_AgileAndID_partnership.pdf]
• “Using Formative Usability Testing as a Fast UI Design Tool”
  John Schrag. UPA 2006 [link: Schrag_FormativeTesting.pdf]
• “Escaping the Design Traps of Creeping Featurism”
  Dong-Seok Lee, David Woods, Daniel Kidwell, UPA 2006.
• “Using the RITE method to improve products; a definition and a case study”
  Michael C. Medlock, Dennis Wixon, Mark Terrano, Ramon L. Romero, Bill Fulton. http://www.lutin-userlab.fr/gamelab/IMG/doc/Medlock-RITE.doc
• More on confidence levels at:
  http://www.measuringusability.com/wald.htm
• “Formative usability investigations for  open-ended tasks”
  Desirée Sy. UPA 2006 [link: Sy_OpenEnded_usability_investigations.pdf]
• “Usability Testing with User Proxies: When is ‘Close’ Close Enough?”
  Desirée Sy. DUX blog.
• Slides from "Big Picture Design without Big Design Up Front"
  Desirée Sy. Agile Roots 2010.

 

Agile UX: A one-day tutorial preview in Toronto, Canada

by John Schrag

Designers at Autodesk like to be involved in the larger design community, by attending and speaking at conferences. At the upcoming Usability Professional’s Association conference in Atlanta, Georgia, my colleague Desirée Sy and I will be presenting a one-day tutorial called “Agile UX Toolkit”. And because we believe in usability testing and iterative development, we are going to prepare by doing a dry-run of that course at our office in Toronto, Canada. We’re inviting any UX professionals in the Toronto area who are interested to attend, practically for free. If you are interested in attending, please send an email to me at john.schrag@autodesk.com and I will let you know how to register. Thanks!

 

Agile UX Toolkit:

As more organizations adopt agile development practices, UX practitioners want to ensure that the resulting products are still designed with users in mind. This tutorial teaches basic, proven methods to integrate user-centered design practices into agile teams.

The tutorial is for experienced UX practitioners and managers who work on agile teams, or who will be transitioning to agile. Prior experience with agile methods is not needed, and the course does not focus on a particular agile methodology.

The morning teaches new skills for planning design on agile teams, and the afternoon focuses on agile adaptations to usability testing and research methods. Research methods covered in this tutorial focus on eliciting observed user behavior (such as contextual inquiry, and formative usability testing).

 

Details:

Date: Friday, May 6^th, 2011
Time: 9AM to 5PM, with lunch and morning/afternoon breaks
Location: Autodesk, 210 King Street East, Toronto
All materials provided

 

Registration:

Although the course itself is free, we are asking attendees to chip in $20, which will be used to pay for lunch, and coffee and snacks at the breaks. To register and find out how to chip in, send an email to john.schrag@autodesk.com.

Usability Testing with User Proxies: When is "Close" Close Enough?

by Desirée Sy on August 14, 2009

How can we designers get valid feedback from more design iterations in less time? One bottleneck in the design flow is finding a steady stream of usability testers. Between the extremes of the perfect (an actual user, on site) and the unacceptable (the developer who's coding the feature), lies the grey zone of user proxies. Can you use internal employees with relevant domain knowledge to usability test your products, and still get valid data?

The answer, as with all design questions: It depends.

We have successfully used internal employees as usability test participants on products where we have an agile user-centered design methodology (see “Adapting Usability Investigations for Agile User-Centered Design” in the Journal of Usability Studies for a lot more detail on our agile methods). Since in agile design the prototypes are incrementally and progressively getting closer to the end product, we took the approach of using tasks and testers that also incrementally and progressively got closer to real work done by actual users.

With agile, our designs are being iterated in an incremental series of “mini designs,” or design chunks, so the earliest prototypes do not allow people to complete full workflows, but only separate operation-level functionality.  It is extremely hard to usability test these early “mini prototypes” externally because they do too little, have very little task-level context, and require extensive intervention from a facilitator to operate. 

An example of one such early prototype might test the dragging algorithm for a zoom operation (where users zoom towards the click-point as opposed to the centre of the screen). While this interaction requires a high-fidelity prototype to test it, it's also possible to create low-fidelity prototypes to test granular interactions.

We usability test these types of “mini prototypes” internally, with extremely artificial forced tasks. For example, to test a brush resizing algorithm for a 2D paint application, I asked internal testers to match the brush size of different brushstrokes I'd prepared on a canvas using different prototypes.  I switched between different prototypes for the testers since I was not testing discoverability of the function, but instead the smoothness and efficacy of the different algorithms. 

Recruiting internal testers requires just as much thought as recruiting externally. You need to understand what you are testing and select testers accordingly. 

We recruit internal testers from Quality Assurance, the Help Desk (Support), and our pool of subject matter experts.  We do not use developers or other UX designers.  All internal testers must work on a different product than the one being tested to minimize bias. Also, all internal testers must have particular qualities in common with our target users: in the brush resizing example, they needed to be people who sketched (so, for example, a QA person who was an expert in video rotoscoping but did not sketch was not recruited – just as an external person would not be recruited without hand sketching experience).

Our sessions with internal users are very fast, not conducted in a lab, and not recorded. Internal staff don’t want videos of them failing a task to be seen by people who might mistakenly see this as a sign of domain incompetence instead of a design failure.
 
Once the foundational interactions are locked down, we combine prototypes to enable more complete interactions that relate to a user activity. These more complete “workflow prototypes” are tested with external users. Initially, these users may be other user proxies (for example, students studying animation, industrial design, or other sketch-intensive fields who may not necessarily be users of the product). Eventually, we'll test refined workflow prototypes with actual users.

With some thought, I believe you can use internal employees and other user proxies successfully as part of an overall design validation strategy, whether or not you are using an agile development method.  The main things to consider are:

• your users' specialized characteristics or environmental contexts
• how different these are from those of your user proxies
• how relevant particular characteristics or contexts are to using a given prototype.

Of course, if what you are testing is heavily dependent on a particular characteristic or context, then you need to usability test with real folks doing actual work—sometimes in the field. But more often than not, there are design decisions that don't depend on all user characteristics. By using internal employees and other user proxies to usability test interactions in early design work, you can increase your design velocity without sacrificing the validity of the feedback.

Values in Software Design Practice

by John Schrag on July 10, 2009

Every user experience (UX) designer who practices in a corporate setting knows the breathless whirlwind that is modern business.  We designers manage relationships with developers, business managers, and customers, and still have a full-time production role researching, designing and validating features and interactions.  We rarely have enough time to do everything we should, and therefore have to carefully choose where to spend our time and resources.  Some designers don't even have the luxury of such choice, and instead are tasked with specific deliverables, which are sometimes not the best use of their time.  In the face of this industry-wide problem, it's no surprise that dysfunctional design practices can arise and even become so common that they are seen as normal, or at least inevitable.  But I don't believe they are.

In 2001, a group of software developers, development managers, and consultants created the Agile Manifesto, a document that outlined a set of simple values that they believed were the foundation of good development practices.  What is interesting about this document is that it makes no reference to specific practices or buzzwords -- instead, it outlines principles and values against which any particular practice can be evaluated.

In the same spirit, my co-workers and I spent several hours considering the question of UX design practice.  We weren't considering the question of what makes good design;  there are innumerable books on that topic.  We were considering the question of what separates a healthy, effective design practice from the horror stories we hear about when talking to others in the industry.  We came up with two lists;  the first list (below) talks about doing certain actions in the right order.  Reading the list, these orderings may sound obvious, but they are by no means standard practice:

Setting Goals before Taking Action
Understand Problems before Generating Solutions
Designing before Writing Design Documents
Validating Designs before Investing in Code
Steak before Sizzle

The second list is one of relative values.  While all of the items on both sides of this list have value, we value the items on the left more than the items on the right:

We Value:

Validated Data over Expert Opinion
Quality of Data over Ease of Data Collection
Complete Workflows over Long Feature Lists
Achieving Results over Writing Reports
Collaborative Design over Design by Referendum or Design by Fiat
Ease of Use over Ease of Coding
Well-designed Critical and Common Workflows over Complete Coverage of Every Possible Workflow

Of course, a few words in a list doesn't capture these ideas all that well.  In future blog posts, I'm going to talk about each of these items in more depth, show examples and talk about what can be done to keep your design practice healthy, even in the face of the whirlwind.

Do you have more items for our list?  We'd love to hear from you.

Autodesk User Experience Talks at UPA 2009

by Lynn Miller on June 4, 2009

Autodesk is happy to be participating in the Usability Professionals' Association 2009 International Conference, Bringing Usability to Life.  This year’s theme is particularly interesting to us since so many Autodesk products are used to design the objects that affect our lives daily – from the buildings we work in and the systems that supply them, to the cars we drive, and even the computer games we play.

We have nine different sessions this year, so there should be something for everyone.

Chauncey Wilson will be presenting his paper, The Art of Style Guides: Process, Politics, and Public Relations, as well as appearing on the panel: Heuristic Evaluation: Use and Abuse. Both of these have valuable insights for those fairly new to the field. For novices and experts alike, Chauncey is also on the panel, The Difficult Quest for a Formal Usability Body of Knowledge. This panel will debate the purpose and future of the UPA Body of Knowledge project.

On the agile development front, Lynn Miller will be presenting Traits of Successful Agile UX Designers, and facilitating the workshop Best Practices of UCD in Agile. Desirée Sy is presenting Coherent Agile User-Centered Design: Create a Big Picture in 2-week Chunks, a must-see for all designers on agile teams.

Autodesk also has three case studies in the program. Teresa Bleser and James McElroy will present Infusing UCD into Product Development While Driving Organizational Change. Rebecca Richkus and Gregg Stanley will describe the useful methods they found for working together in Design Partners: a Tale of UX and Subject Matter Experts Designing Together. And last, but not least, Erin Bradner and Melissa Dawe will share real-world lessons learned conducting a usability benchmark on a large, complex software application in their talk: Parts of the SUM: A Case Study of Usability Benchmarking Using the SUM Metric.

Drop by a session and say Hello!

Come See Us at SIGCHI 2009

by Lynn Miller on March 11, 2009

We at Autodesk are pleased to be having our largest presence ever at SIGCHI in 2009.  Not only are we a Champion Sponsor (more on that at a later date), but we have six people presenting this year on a variety of topics.

Lillian Smith and Greg Demchak are showing Project Chicago: Green Research in the Video Showcase on Tuesday, from 6:30 to 8:00pm.  Their video showcases a highly graphic, interactive technology concept for evaluating water and energy reduction, indoor environmental quality, and carbon footprint impacts and giving designers an immediate sense of the results of different building designs. This proposed sustainability dashboard concept is demonstrated on a 6’x3’ touch screen to explore its potential use as a collaborative tool. The dashboard is presented by Autodesk as a technology concept; it is not a commercially available product.

Lynn Miller and Desiree Sy are holding an Agile User Experience SIG on Wednesday from 9 to 10:30am. The goal of this SIG is to draw upon the shared experience of UX practitioners to uncover the best practices for agile user-centered design which facilitate optimal product development.

Lira Nikolovska, Greg Demchak and Matt Jezyk are presenting two papers as part of Designing for Discovery on Wednesday from 4:30 to 6:00pm.  The first paper, Simplified User Interfaces for Design and User Testing of Architecture Software Applications, describes the value of creating simplified user interfaces for use in early conceptual design phases. By simplifying the interface, the team was able to solicit specific feedback about new tools without the overhead or pre-conceptions associated with using an existing software platform. As a result, the team was able to iterate rapidly on specific problems. The second paper, Understanding User Needs for Conceptual Design Phases of Architecture Projects, describes design research methods used to understand user needs, identify user requirements and create new conceptual design workflows for an existing architectural software application.

And last but not least, Autodesk employee Chauncey Wilson is a member of the organizing committee for the User Experience and Usability community.

We hope to meet you at SIGCHI this year!

Lynn