Method 13 of 100: Consistency Inspection

100 User Experience (UX) Design and Evaluation Methods for Your Toolkit

by Chauncey Wilson

This is the 13th in a series of 100 short articles about UX design and evaluation methods. Today’s method is “consistency inspection”.  This is a general method for examining one or more products for internal or external consistency. Every company I’ve worked for has had a goal to “make our products consistent,” but as I described in an earlier DUX blog, http://dux.typepad.com/dux/2009/03/the-consistency-conundrum.html, the concept of consistency is complex. The goal of this posting is to suggest a general method for finding and cataloging inconsistencies within a single product or between multiple products. Comments on this general method would be much appreciated.

Method 13 of 100: Consistency Inspection

When to Use:

Consistency inspections are useful when you get the “make things consistent” mandate from senior management, acquire new products that you will be integrating with existing products, or when your users are complaining about how consistency hampers learning and increases errors.

 Procedure:

1. Create a small team of “inspectors” with different backgrounds – design, usability, training, QA, user assistance, development, and product management. The rationale for a small team is that different perspectives will yield more consistency issues.
2. Define the target of the inspection. The target can be a particular area of a product, a product, or a set of products. Ask the inspectors to review specific areas of the target product against any consistency exemplar and references. In a major product with many features, you might assign sub-teams that will each look at a particular type of consistency.
3. Define an explicit list of what consistency attributes are the focus of the inspection. You may want to focus on a single attribute like icon/menu consistency or multiple attributes (e.g., terminology, layout, use of UI controls, workflow, work-context).
4. Provide the “exemplar” or reference against which you will compare within a product or between products. For example, you might have a style guide like the OSX or Windows 7 user experience guidelines with standards and guidelines that you use to determine where the product or products are inconsistent. You might designate a particular application as the exemplar – the product that you want all others to be consistent with – and note where related applications, like those in a suite, differ from the exemplar. You might also have a pattern library that you can use as a “consistency reference”.
5. Train your consistency inspectors on the following topics:
   1. The definition of consistency and the different levels (e.g., within a dialog, between dialogs, across one or more applications, with applications that are used with a particular application, with the Mac or PC operating systems) and attributes of consistency.
   2. Examples of “consistency bugs”. Here, you want to provide clear examples of items that you want to track as consistency bugs and have discussions to help the team grasp the range of consistency problems that will emerge.
   3. How to report consistency issues? A template for reporting might include: The product or products under scrutiny, the particular component (e.g., file open dialog or pop-up menu), the exemplar or reference that you using to determine if something is inconsistent, and the type of inconsistency.
6. Provide the inspectors with the target of the inspection, which can be a spec, a wireframe, a prototype, or working product. Early consistency inspections are highly recommended to avoid embarrassment late in development. For example, you might find that there are two different search engines in the same product that worked very differently – an embarrassing and costly consistency gaffe!
7. Categorize and prioritize the issues and work with the inspection team on a plan to improve consistency.

Strengths and Weaknesses:

+  This method provides some explicit guidance about what to look for and compare against when you are asked to review a product or set of products for the multifaceted concept of consistency.

+  Creating a team and training them on consistency can promote consistency across groups. For example, once developers or product managers do a consistency review, they will be sensitive to issues of consistency in the future.

-  Consistency is a complex characteristic since there can be many reference sources against which to make consistency judgments. In addition to this inspection method, you need to promote consistency in many ways (see the oldie, but good book, “Coordinating User Interfaces for Consistency” for many ideas on how to promote and improve consistency)

References:

De Oliveira, R. & Da Rocha, H. V. (2007). Consistency priorities for multi-device design. In Proceedings of the 11th IFIP TC 13 international conference on Human-computer interaction (INTERACT'07), Cécilia Baranauskas, Philippe Palanque, Julio Abascal, and Simone Diniz Junqueira Barbosa (Eds.). Springer-Verlag, Berlin, Heidelberg, 426-429.

Grudin, J. (1989). The case against user interface consistency. Commun. ACM 32, 10 (October 1989), 1164-1173.

Nielsen, J. (1989). Coordinating user interfaces for consistency. San Francisco: Morgan Kaufmann. [Note about the book at http://www.useit.com/jakob/constbook_preface_2nded.html]

Wilson, C. (2009). The Consistency Conundrum. http://dux.typepad.com/dux/2009/03/the-consistency-conundrum.html

---

What’s Next in the Series?

The next UX method posting will describe a method borrowed from improv called “Yes, and…” that you can use as an ideation technique for stories, scenarios, and requirements.