Method 3 of 100: Freelisting

100 User Experience (UX) Design and Evaluation Methods for Your Toolkit

by Chauncey Wilson

This is the third in a series of 100 short articles about UX design and evaluation methods. Today’s method is called freelisting (or sometimes free listing or free-listing). The technique can be used as a complement to group brainstorming or as an aid in understanding terminology, concepts, mental models, and beliefs. I first ran across the method while reading about methods in cultural and applied anthropology (Trotter & Schensul, 1998) that could be applied in the UX domain.

Method 3 of 100: Freelisting

Freelisting is a simple technique where you ask an individual or a group to “list as many ideas (or items) on [Topic X] as you can”. For example, you might ask a person to “list all the tasks that you perform on a project” or to “list all the aspects of [Product X] that frustrate you”. The technique can be used to understand users and their mental models by highlighting the frequency and order of answers to the freelisting question. The data from a freelisting exercise can be used to rank-order data by factors like job role and experience (assuming that you are gathering this data on your participants).

If you conduct freelisting exercises, you can assess the “coherency” of a domain or mental model by examining the statistical consistency of the frequency and order of items from freelisting participants. If your results are all over the board, then perhaps the domain or model is not too coherent.

When to Use:

This method can be used when you have limited time with a group. For example, I’ve asked my graduate students to “list all the prototyping methods that they have used” to get a sense of their experience levels and what I may have to emphasize in my class on Prototyping Methods at Bentley University. I’ve also used this method at conferences with hundreds of people – just hand out a sheet of paper, ask the freelisting question, and then collect the results. You get hundreds or thousands of responses which you can then organize by frequency and category.

You can also ask one or more freelisting questions to individual users during interviews as a way to get some detailed data on a topic. You can use online surveys, wikis, or even chat/IM systems to conduct freelisting sessions.

Procedure:

1. Consider what information would be valuable and can be easily listed in a short period of time (from a few minutes to say 15 minutes).
2. Develop your question using the “Would you please list all the [Item X] that you…?
3. Choose the media for the freelisting exercise. You might want to print a page with the freelisting question in large print at the top.
4. Develop a short set of instructions where you indicate:
   1. The purpose of the freelisting exercise
   2. How to use the media (e.g., paper, chat, survey)
   3. The time allotted for the exercise
   4. The importance of writing fast, but legibly
   5. What you will do with the results (and if the results will be made available)
5. Ask the freelisting question. If you doing this in a phone or face-to-face interview, consider the use of probes to help recall. There are two primary types of probes that can aid recall:
   1. Alphabetic probes where you ask the participant after listing items if they recall any more items starting with the letter “A”. If you were asking about UX methods, this might prompt the person to recall “A/B testing“ or “Audit”.
   2. Semantic probes where you ask the participant to review the list that he/she generated and “try to remember things that are similar to ones that were already listed. For example, if you were asking students to list all the usability testing methods and one of the ones listed was “heuristic evaluation” you might ask if there were any others similar to “heuristic evaluation” and they might recall that they did a “persona-based inspection” or a “structured heuristic evaluation”.
6. Collect data from multiple participants and then develop a frequency diagram that shows the items that were collectively most frequent. To see an example of a freelisting data analysis, check out Rashmi Sinha’s 2003 article on freelisting.

Strengths and Weaknesses:

+ Freelisting is simple.

+ Does not require a lot of training.

+ Can be used to compare different groups or cultures.

+ Can be used with different media (I’ve used Google® Spreadsheet and IM to do this)

+ Can be used to complement brainstorming or as a way to develop an understanding of users, domains, tasks, terminology, and mental models.

+ Can be done in group settings or face-to-face interviews.

+ Provides frequency, salience, and consistency data.

- Can be laborious to analyze for patterns with large sample sizes.

References:

Sinha, R. (2003). Beyond cardsorting: Free-listing methods to explore user categorizations. Boxes and Arrows. Retrieved January 10, 2011 from http://www.boxesandarrows.com/view/beyond_cardsorting_free_listing_methods_to_explore_user_categorizations

Trotter R. & Schensul, J. J. (1998). Methods in applied anthropology. In H. R. Bernard (Ed.). Handbook of methods in cultural anthropology. Walnut Creek: CA: AltaMira Press.

 

---

What’s Next in the Series?

The next UX method posting will describe “Reverse (Negative) Brainstorming”, an ideation method where the moderator asks negative questions like “What can we do to make our product frustrating to our customers?”