Which UX Research methodology should you use? [Chart included]

Start with this question, and all else will follow.

Source

When I first started as a user researcher, I believed there were two types of user research: usability testing and discovery research. I was right, those two types of research do exist, but they are not the only methodologies a user researcher can use to answer questions. But, alas, I was young in my career and believed every problem could get solved in one of these two ways. Am I needed to test a prototype? Usability testing. Am I required to figure out the content? Usability testing or discovery research (depending on my mood). Do I have to figure out how a user feels? Discovery research.

For a while, it was all very black and white. It was a simple time, and one where I believed I could unlock all the answers by using these two methods. I didn’t have to think about information architecture, long-term studies, testing concepts, surveys, clickstreams, A/B testing, the list goes on. Sigh, it was an effortless time.

However, like most people on their career path, I finally hit a wall. I encountered a research question I could not solve with these methods, despite my trying. I did try, and I failed miserably at this.

We were trying to redesign the navigation of our platform and where different features appeared on each page. Notice, we didn’t start with a problem statement or a goal. We started with a solution of a redesign. Also, we already had the redesigned screens. With this in mind, I figured a usability test could solve all of our problems, or validate our pre-defined solutions.

In the end, we decided to do a usability test with the new navigation and layouts. It was clunky, but our participants got through. However, when we released the changes (not too far after the user tests), we found a lot of issues and had to revert to the old designs.

So what could we have done better?

There were many things we could have done better, but I’ll focus on the three biggest ones we could have changed:

• Define why. Defining the why is step one of any research project or initiative. We have to understand why we are doing the research. I often include the why from the user perspective, as well as the business perspective. The why can be a simple paragraph explaining the reasoning behind the project.
• Start with a problem statement. When you start any user research project, it is essential, to begin with a problem statement. A problem statement is a central question that has to be answered by the research findings. In this case, we started with a solution, which led us down the wrong path. By starting with a problem, you are much more likely to solve an actual user need. Learn more about reframing a solution to a problem statement.
• Consider the objectives. By starting with goals, we can understand what we are expecting to learn from the research. We can ask ourselves what we are trying to learn and what we would like the research to achieve. Objectives keep your research on track and align everyone with an expected outcome. Learn more about writing objectives.

Since we skipped straight to testing a solution, we missed these crucial steps. By doing this, we ended up with a suboptimal experience for our users because we didn’t understand what their problems were and what they needed. It is an easy trap to fall into, but easy to avoid by following those steps. Instead, the process should have looked much more like this:

The overall process for choosing a research methodology

Defining the basics

With the knowledge from above, the outcome of that research project would have been much different. However, I still had the idea that there were only two types of user research: usability testing or user interviews. If only I could have told myself about all the different methodologies out there. Instead of time traveling, I will rewrite history to showcase the process I would go through today to choose a method for the same project.

The project was: Redesigning our hotel concierge platform navigation and information architecture of our features for hotel concierge.

Step 1 — Define why you are doing the research.
We have seen some users struggling when trying to navigate through our platform. Through user research, we have observed users hitting the cmd+f option to find what they are looking for to make the process faster. They are unable to find nested information as they are not sure where to look in our navigation. Also, users have been employing several hacks as opposed to using the features we have built.

Step 2 — Define a problem statement.
I am a hotel employee trying to fulfill a request for more towels for a guest, but I am unable to find that specific feature, which makes me feel frustrated. With this, I have to enter a generic request, and then write in what I specifically need and hope someone reads my notes.

Step 3 — Define the objectives.

• Understand the general workflow of users and when they need access to particular pages/features
• Discover how users are currently using the product
• Uncover the limitations and pain points users are facing in different situations with the platform
• Learn about any potential improvements in workflow, information architecture, or missing features

Choosing between generative and evaluative research

Now here is where things became fuzzy, and where I want to dive deeper. Although now I have a good understanding of which methodologies would have been best for answering the question and objectives, it didn’t always come so intuitively.

Since then, I have seen four significant types of objectives:

1. Understanding more deeply about people’s day-to-day lives, regardless of a product/service
2. Discovering how a person uses a product/service and how that product/service impacts their daily lives (positively and negatively)
3. Evaluating how a product/service works when a person is using it (pain points and joys)
4. Assessing how a product/service could be structured to impact a person’s daily life positively

By breaking these objectives down, we can then choose between generative research and evaluative research.

• Generative research allows a deep understanding of who our users are (inside and outside of a product/service). We can learn what they experience in their everyday lives. It allows us to see users as human, beyond their interaction with a product/service.
• Evaluative research is about assessing how a product/service works when placed in front of a user. It isn’t merely about functionality, but also about findability, efficiency, and the emotions associated with using the product/service. Many people think evaluative research = usability testing, but it goes much further than that
• Hybrid research is a combination of generative research and evaluative research. The example I’ve used above can very well end up being a hybrid method. Hybrid research helps us simultaneously understand our users, as well as how a product/service is performing. Now, this is not magic and, since it covers both spaces at once, it does not go as deeply into a place of understanding or evaluation. I will focus on hybrid research in a future article, as it is a more advanced technique.

Choosing between generative and evaluative research

Once we understand what type of research we are looking to conduct, we can better understand which methodology ties best to our objectives. I have included a chart of methods I have used, and how they relate to the above goals I have realized.

Methodology x Objective

If you are curious, for the research project I mentioned above, I would have suggested the following methods:

• Contextual inquiry
• Walk the store interview
• Card sorting

Now, none of this is an exact science, and your opinions may differ on this, which is fantastic! I would love to hear about how you approach this differently.

---

Interested in more user research? I teach an Introduction to User Research Course and am available for one-on-one mentoring. Check out the User Research Academy. Please join the User Research Academy Slack Community for more updates, postings, and Q&A sessions