Method 8 of 100 - Repeated Card Sorting

100 User Experience (UX) Design and Evaluation Methods for Your Toolkit

 by Chauncey Wilson

This is the eighth in a series of 100 short articles about UX design and evaluation methods. Today’s method is called repeated card sorting. Repeated card sorting is a simple method for uncovering important dimensions of a set of related items.

Method 8 of 100: Repeat Card Sorting

When to Use:

Repeated card sorting is a variation on open card sorting that involves the repeated sorting of a set of items to understand underlying dimensions or characteristics of a product or service (Curran, Rugg, & Corr, 2005). One use of the repeated card sort method is to see if the dimensions that usability specialists believe are most critical to product success match the dimensions that are important to actual users. In common usability surveys, we might use “standard” dimensions like satisfaction, efficiency, and learnability that may or may not be what is important to users. Repeated card sorting can provide some validation of those standard dimensions and suggest new dimensions that might be even more important those our standard ones. Upchurch, Rugg, & Kitchenbaum (2001) used a repeated card sort to determine what qualities of a Web site influenced its perceived quality. In their study, individual participants came up with 4 to 11 sort criteria each. Over 80 sort criteria, some quite different from basic usability dimensions emerged from the study. If you were to do a repeated card sort of online banking sites, you might come up with some of the dimensions below:

• Trustworthy – Untrustworthy
• Ugly – Attractive
• Simple – Complex
• Dense – Open
• Informal – Formal
• Small Customer – Large Customer
• Colorful – Lacking Color
• Many Ads – Few Ads
• Scary – Inviting

 

Procedure:

1. Provide participants with a set of cards (or pages) that are thoroughly shuffled.
2. Ask each participant to sort the cards into two piles.
3. Have the participant name each pile and describe how the piles are different. For example, the participant might name one pile “Modern” and other pile “Old Fashioned” and describe what those names mean.
4. Shuffle all the items again and ask the participant to sort the cards in a different way.  
5. Name the new groups and once again ask the participant to describe how the piles are different.
6. Continue for 5 to 6 rounds or until the participant cannot come up with a new set of piles.
7. Gather sort data from 10-20 participants.
8. Organize the dimensions that you obtained from all the participants to see if there are any common themes.

Strengths and Weaknesses:

+      Simple procedure.

+      Useful for understanding “user-derived” dimensions that should be considered when evaluating a product or service.

-       Analyzing the dimensions that emerge across participants requires some background in qualitative analysis.

References:

Curran, M.J., Rugg, G., & Corr, S. (2005). Attitudes to expert systems: a card sort study.
The Foot, December 2005, 15(4), pp. 190-19

Upchurch, L., Rugg, G. & Kitchenham, B. (2001). Using card sorts to elicit Web page quality attributes. IEEE Software, 84-89.

---

What’s Next in the Series?

The next UX method posting will describe, “reverse card sorting”, a variation on closed card sorting that focuses on hierarchical structures like tree menus and Web navigation.