Method 7 of 100 - Claims Analysis

100 User Experience (UX) Design and Evaluation Methods for Your Toolkit

 by Chauncey Wilson

This is the 7th in a series of 100 short articles about UX design and evaluation methods. Today’s method is claims analysis. Claims analysis is a simple method for documenting the impact of new features or different designs on product usability and user experience.

A claim is a hypothesis about the positive and negative consequences for users and other stakeholders of a design decision. Claims can be based on theories, principles, patterns, guidelines or practical experience (Carroll & Rosson, 1992; Carroll, 1995;). Claims for new products are based on possible scenarios or stories of use and document the product team’s assumptions about how a set of features affects users or other stakeholders.

For example, a decision about whether to disable or hide entries in a pop-up menu carries positive and negative consequences for different stakeholders. Here is a brief and simple set of claims – positive and negative consequences – for a pop-up menu design that hides rather than disables menu items. Note that the claims (3 positive and 2 negative) refer to different groups – new users, expert users, developers, and product managers (consistent across products).

Design Feature: Pop-up menu items that are not available in a particular context are removed (rather than disabled)

+ The menu is shorter and theoretically more efficient for expert, high-frequency users.

+ There is less clutter in the pop-up menu.

+ The code for pop-ups supports the removal of items that aren’t available.

- The menu will have reduced positional predictability – a problem for new and non-expert users.

- Other products in our portfolio disable items in pop-up menus so this design feature would be inconsistent with other products and frustrate users.

One approach that we’ve used at Autodesk for claims analysis is to issue members of a product team red (negative consequences) and green (positive consequences) Post-it® notes and ask them to write their claims for different designs and scenarios of use as shown below. We ask people to identify the class of stakeholder (e.g., developer, new user, expert users, disabled user, product marketing) on each note so the perspective behind the claim is clear. The use of the two colors, a sort of visual claims analysis, can give you a quick sense whether a particular design feature is feasible – too many negative claims (red stickies) might portend problems. Here is a conceptual example where it is clear that Approach 1 may not be the approach that should be considered.

Method 7 of 100: Claims Analysis

When to Use:

Claims analysis is useful when:

• You are considering (debating) different solutions to a design problem.
• You want to get at the assumptions behind particular design.
• You want to understand what impact particular design decisions have on the user experience for one or more classes of users.
• You want to try a new approach to a design problem that is not well understood.
• You want to involve the entire product team.

Procedure:

1. Consider a list of the key stakeholders: new users, expert users, designers, developers, product managers, sales, etc.
2. Develop a set of scenarios that describe how people will use a product.  
3. Extract key features from user scenarios that will have an important effect on your users. For example, if you were planning on using a new type of control to find objects in huge lists (e.g., an elliptical browser), you might want to do a claims analysis on this feature.
4. Prioritize your list of key features.
5. Generate positive and negative claims for high-priority key features from the perspective of the stakeholders you listed in Step 1. You can do this in several ways:
   1. Brainstorming
   2. Interviews with users and other stakeholders
   3. Reviewing literature and reviews on similar features in other products
   4. Using a set of questions that help evoke possible consequences
   5. Using the “Yes, but…” format where a feature is described and you say, “Yes, but would it work for users who are disabled?”
6. Take the most important key feature from your list of claims and consider design trade-offs that will decrease the impact of negative claims and increase the impact of positive claims.
7. Repeat this process for the next key feature.

Strengths and Weaknesses:

+Simple and low budget.

+Multiple stakeholders can bring up consequences that might not be obvious. 

+Highlights positive consequences as well as negative ones.

+Makes assumptions of product team members explicit.

-Deciding which design issues to consider for claims analysis is a craft since there are thousands of decisions and hundreds of scenarios involved in creating even simple products.

-Claims analysis does not provide specific solutions.-Requires mentoring of product teams on design principles, patterns, and cognitive theories.

References:

Carroll, J. M. (1995). Introduction: the scenario perspective on system development. In J. M. Carroll (Ed.) Scenario-based design: envisioning work and technology in system development (pp. 1-18). New York: John Wiley & Sons, Inc.

Carroll, J. M. & Rosson, M. (1992). Getting around the task-artifact cycle: how to make claims and design by scenario. ACM Transactions of Information Systems, 10, 181-212.

Inspection Methods – Claims Analysis. http://insidethefactory.typepad.com/my_weblog/2009/09/

Usability First Glossary – Claims analysis. http://www.usabilityfirst.com/glossary/claims-analysis/

What’s Next in the Series?
The next UX method posting will describe “Repeat Card Sorting”, a technique for eliciting underlying dimensions or distinctions among a set of items.