Method 26 of 100: The Critical Incident Technique (CIT)

by Chauncey Wilson

The critical incident technique (CIT) was first described by Flanagan (1954), as a method for collecting negative and positive incidents that contributed to the success or failure of a task.

Critical incidents are brief and memorable descriptions of actions that a person or group performs in particular situations that lead to either effective outcomes (successes) or ineffective outcomes (failures or near misses). 

Ideally, critical incidents are real incidents reported by actual users doing real work in their normal work environments.

The word “critical” often refers to a severe usability problem or a clear success. But sometimes a critical incident is something relatively minor, like an error message on a new system that forces you to waste time debugging when your other tasks are more important.

The CIT is useful for improving software documentation, software applications, web sites, aircraft controls, surgery, product requirements, and business practices.

  Information to include in a critical incident report

• A description of the task in which the critical incident occurs. 
• A description of the critical incident.
• A statement indicating whether the action that is performed during the critical incident is effective (a success) or ineffective (a failure).
• Optionally, a rating of how critical the incident is for completing a task.

 Reasons to use the CIT

• Understand less frequent events that are not always evident in usability testing or other forms of evaluation where a person is observed for a relatively short period of time.

• Collect critical incidents from remote users with early working prototypes, current products, or competitive products when you cannot view the users directly.

 Strengths and weaknesses:

+  The CIT is relatively inexpensive and provides rich qualitative data.

+  Data from a critical incident study can reveal system weak points, and system strengths.

+  The flexible method is adaptable for interviews, observation, surveys, think-aloud testing, online discussion groups, and other methods.

+  The CIT sometimes identifies rare events that are not picked up by other methods that focus on common or everyday events.

+  The CIT is applicable across a wide variety of products, environments, and users.

-   The CIT method focuses on critical incidents, and doesn’t capture routine task information that is often extremely important in system design.

-   The criteria that makes an incident “critical” is sometimes ambiguous. 

-    People are sometimes afraid to report critical incidents where their behavior contributed to or caused a serious problem.

 Procedure for using the CIT

1. Define the system of interest and the length of your study. If critical incidents are rare, consider running your study for days or week.
2. Determine the method for collecting critical incidents.  Use paper forms, online survey tools, or online forms that are suitable for display when an incident occurs.
3. Create the critical incident form.
4. Distribute the form with brief instructions to the target participants.
5. Run the study for the period of time that is necessary to capture of rare events.
6. Compile the data and categorize incidents into common themes.
7. Organize related incidents by frequency.
8. Brainstorm solutions to the most frequent or most severe incidents.

References:

 Flanagan, J.C. 1954. “”The critical incident technique.”” Psychological bulletin, 51(4), 327-358.

The classic article is at:  http://www.apa.org/pubs/databases/psycinfo/cit-article.pdf

 UXPA. Critical Incident Technique (CIT). http://www.usabilitybok.org/critical-incident-technique