Basics on how to conduct card sorting
And why it is one of my favorite methods.
There are a few reasons I love card sorting so much. The method is simple, effective, and fun to participate in (for both the moderator and the participant). I was surprised when I realized I hadn’t written an article solely on card sorting. It deserves it’s time to shine.
I first learned about card sorting about six years ago. I was talking to a colleague, and we wanted to redesign our platform for a different set of users. We were currently supporting hotels and hotel staff, but we wanted to see if we would also help staff working in residential buildings (think high-rise apartment buildings in New York City).
Now, of course, there was a lot more that goes into a conversation like that then merely bringing up card sorting. We had done qualitative research with the residential building staff already. We were trying to think of how to pivot the platform. How might we use the existing skeleton to support the differences in workflow?
I was still relatively new to the field of user research, and it was back when I believed there were only two methodologies: discovery research and usability testing. I hadn’t done too much experimenting with other methods for a few reasons. I hadn’t learned about theses methodologies, I didn’t feel confident with them, and I had no real guidance on how to implement this.
We set up our goals for the research session. We wanted to accomplish a few things:
- Understand what current features made sense to these users, and which were missing
- How the users categorized the different features and information on the platform
- How users would imagine the platform to look in terms of navigation and nesting
- The flow users went through in their day-to-day
Many great goals. After a week, we started usability testing. Spoiler alert: it didn’t work. I tried to ask them how they would navigate the current platform and stumbled around with future-based questions that were most likely leading. At one interview, I remember getting out a piece of paper and just having the person write down a list of what they wanted.
Major fail.
We wanted a mix of understanding users’ mental models and information architecture. That is where card sorting shines. After that last interview, I went back to the drawing board, Google.
At the time, I didn’t know what information architecture was, but I eventually put together enough of the right search terms to find card sorting. After reading a few articles, we went to work.
What is card sorting?
Card sorting is an activity in which you give cards to a participant and have them order the information in a way that makes sense to them. These cards can have information written on them, can be blank or a combination of the two.
These three scenarios line up with the three different card sorting techniques there are. Each of them has a time and place. My favorite is mixed card sorting.
Closed card sorting
Closed card sorting is when each participant gets a set of cards with information already written on them. They are limited to using these cards. This approach is very evaluative and is best when the terminology or concepts are well-defined and established. It can give you apparent patterns on the cards. The significant cons for this, however, is that you might not fully understand the user’s mental model, as they have to conform to what you wrote on the cards.
Open card sorting
Open card sorting is pretty much the opposite of closed card sorting. Participants create categories and concepts of their own and then order them. Open card sort is great for exploratory work, and understanding how users relate to, organize, and define different concepts. It can lead to a better understanding of terms and definitions. However, the con for this approach is that the patterns are usually not as clear as with closed sorting.
Mixed card sorting
Mixed card sorting includes cards with predetermined information, but allows the participant to create new categories or concepts that may be missing. When I do mixed card sorting, participants can “edit” the pre-written information. Mixed card sorting is my favorite because it allows for both evaluative and generative work, although you may still run into the same cons of fewer distinct patterns.
How do you choose?
First off, determine if you are in a generative or evaluative phase of the project.
Would you feel comfortable/confident in defining terms, concepts, and categories for your users? Have you conducted previous research that would help you correctly identify them? If yes, you can then use closed card sorting to evaluate the patterns better.
If you are starting from scratch, or don’t feel confident in creating cards, I would recommend going with either open or mixed card sorting. This way, you can better understand how users are defining these areas, and then go ahead with a closed card sorting after that.
To decide between open and mixed is up to you. If you know that some concepts have been validated, but others not, then go for the mixed, if you want to go all out discovery, open will suit you best.
Regardless, make sure you always capture the running commentary as the participants are placing, writing, or organizing cards.
How to run a card sorting test
Many different components go into a card sort, so I split them up into three main steps. I also referenced articles that go more into detail when it makes sense, such as with defining the content of the cards.
Before the card sort
Now that we’ve talked about this magical methodology. Let’s run one.
Define your research goals. As wonderful as card sorting is, it has its time and place (as does every other methodology). To be effective, you need to make sure your research goals match up to your methods.
The common goals that align with card sorting are:
- Evaluating a users’ mental model on the information architecture of a product/service
- Understanding how concepts relate to each other in the mind of the user and the hierarchy of them
- Uncovering definitions, terms, or ideas that might be missing or misunderstood
Decide what card sorting method you will be using. Depending on the phase of your project, choose between open, closed, and mixed. If you aren’t sure, I always recommend mixed. If you are super tight on time, you can do a closed card sort and then, in the end, ask users what they would change.
Writing the cards. If you are doing a closed or mixed card sort, you have to write the cards. Knowing what to include in a card sort is the tricky part. What you write on the card depends on the project. The most important part is not to include more than 40 cards in your card sort. Since deciding what to write for a card sort is an extensive process, I wrote another article on that here. That article is all about how to write cards for card sorting.
Setting up the card sort
- Usually, card sorts last 60 minutes, so plan for that amount of time
- Recruit 15–20 participants for this kind of research
- Get the cards ready
- Make sure there is enough space to spread out all of the cards on a table or ensure there is enough wall space if they are hanging them
- If you are conducting your card sort online, make sure you have sent the link to the participants and tell them they will need internet access
- Get someone to help you during the session to take notes and observe what you might miss. They can also help with recording the meeting
- Go through the session to practice before with a colleague to make sure everything makes sense before inviting the participants
Moderating the card sort
- Give the participant the set of cards (or have them out on a remote tool). Walk them through what the session will be like and explain what you are looking for them to do. In each scenario, mention that any cards that are confusing or they are unsure about can be put to the side.
- Closed/mixed card sort: Tell the participants you are Explain that you are looking to understand how all of the items on the cards relate to each other. Explain that they can group the information in a way that makes the most sense to them.
- Open card sort: Tell the participant that you are trying to understand what should be in your product/service. Then explain, once they’ve finished naming the cards, that you want them to group the cards in a way that makes sense to them. Once the cards are grouped, you will ask the participant to name the grouping. Then explain that you will ask for a name for each group of cards once the participant has arranged them.
- Request that the participant thinks aloud during the session, so you can fully understand their thought process behind their categorizations. If the participant struggles with thinking aloud, be sure to prompt them regularly
- Thank the participant, give them a chance to ask any questions, and provide your contact information. Be sure to mention the timeline for any incentive.
- Finally, email the participant within 24 hours to thank them for their participation in the study and with any incentive you had offered
Analysis
Once you finish the card, it is time to analyze it! To keep the length of this article reasonable, I will write a separate article on how to analyze your card sorting results, including some templates.
Recommended tools and resources
- Notecards and pens
- Trello
- Optimal workshop
- UXSort
For more engaging discussion, please sign up for the User Research Academy slack group! And check out User Research Academy for free resources.
