Method 9 of 100: Reverse Card Sorting

100 User Experience (UX) Design and Evaluation Methods for Your Toolkit

by Chauncey Wilson

This is the ninth in a series of 100 short articles about UX design and evaluation methods. Today’s method is called reverse card sorting. Reverse card sorting (also called inverse card sorting, reverse lookup, or tree testing) is a variation on closed card sorting where you place cards representing content, tasks, or navigation items onto a predetermined hierarchy (or other type of structure) and then assess how often users placed the cards into the “right” categories on the hierarchy.

Method 9 of 100: Reverse Card Sorting

When to Use:

Reverse card sorting is a variation on closed card sorting where a user is presented with a hierarchical diagram and a pile of cards representing categories and subcategories of information. Participants are asked to place the cards at the correct level of the hierarchy. The primary metric for this method is the mean percentage of cards that are sorted into the correct location on the hierarchy. This method can be used to validate changes to Web site navigation, hierarchical menus, or other task structures. In one study by Human Factors International, reverse card sorting was used to compare the navigational structure of a new Web site design to the old structure. The result of the study was that “…96% of the users understand the new site’s organizations and task groupings, compared with only 45% on the old design.” 

Figure 1: Sorting cards from a pile onto a hierarchy

 

Procedure:

1. Create a structural diagram and enter the labels for the high-level categories (see Figure 1). For example, if you are testing the main navigation for a new Web site, you would create a diagram with the top-level navigation labels filled in.
2. Create a set of cards that have questions, tasks, or names of the items that go into the sub-categories of the structural diagram. For example, you might create a card with a question like “Your wife has just purchased an iPad. Where would you go to buy her the leather cover that activates the device?” You would then place that card under the appropriate category on the tree diagram. The cards should not use the same language as the actual category labels. For example, in the iPad example, the actual category might be “Accessory” so you would not want to use that term in the question that you have on the card.
   1. You can use a tool like PowerPoint to create the structural diagram and online cards and send participants the PowerPoint file with instructions for the reverse card sort.
   2. You could also use collaborative graphics tools like Gliffy or Google Docs to conduct remote reverse card sorting sessions.
   3. There is a tool called “Treejack” that examines the “findability” of items in a tree structure by asking a question and then asking users to navigate through a tree structure until they find the right page. For more information on Treejack see http://www.optimalworkshop.com/treejack.htm
3. Shuffle all the cards that will be placed on the diagram or arrange them in a random order if you are using an online sorting tool.
4. Ask the participants to place each card in the lower level category on the diagram where they would expect to find the information on the card or get the answer to a question. Tell the participant that there is no wrong answer and you are interested in understanding how people would group information.

Strengths and Weaknesses:

+ The method is simple; it is like a visual quiz.

+ The method allows an evaluation of an information structure independent of the visual design or other confounding factors.

+ The method is useful for evaluating the effectiveness of an information architecture.

+ The method can be accomplished using a variety of online tools.

-  Reverse card sorting can be difficult if the information structure is complex.

 

References:
Human Factors International (N.D.). Case study – corporate intranets. Retrieved from http://www.humanfactors.com/downloads/documents/arinc.pdf on March 14, 2011.
Paul, C. L. (2008). A modified approach to a new card sorting methodology. Journal of Usability Studies, Volume 4, Issue 1, pp. 7-30. Retrieved from http://www.upassoc.org/upa_publications/jus/2008november/paul1.html on March 14, 2011.
Thurow, S., & Musica, N., (2009). When search meets web usability. Berkeley, CA: New Riders.

---

What’s Next in the Series?
The next UX method posting will describe perspective-based user interface inspections – a method where reviewers evaluate a product from different perspectives to broaden their problem-finding ability.