2/17/20 2/17/20

Which UX Research methodology should you use? [Chart included]

Start with this question, and all else will follow.

When I first started as a user researcher, I believed there were two types of user research: usability testing and discovery research. I was right, those two types of research do exist, but they are not the only methodologies a user researcher can use to answer questions. But, alas, I was young in my career and believed every problem could get solved in one of these two ways. Am I needed to test a prototype? Usability testing. Am I required to figure out the content? Usability testing or discovery research (depending on my mood). Do I have to figure out how a user feels? Discovery research.

For a while, it was all very black and white. It was a simple time, and one where I believed I could unlock all the answers by using these two methods. I didn’t have to think about information architecture, long-term studies, testing concepts, surveys, clickstreams, A/B testing, the list goes on. Sigh, it was an effortless time.

However, like most people on their career path, I finally hit a wall. I encountered a research question I could not solve with these methods, despite my trying. I did try, and I failed miserably at this.

We were trying to redesign the navigation of our platform and where different features appeared on each page. Notice, we didn’t start with a problem statement or a goal. We started with a solution of a redesign. Also, we already had the redesigned screens. With this in mind, I figured a usability test could solve all of our problems, or validate our pre-defined solutions.

In the end, we decided to do a usability test with the new navigation and layouts. It was clunky, but our participants got through. However, when we released the changes (not too far after the user tests), we found a lot of issues and had to revert to the old designs.

So what could we have done better?

There were many things we could have done better, but I’ll focus on the three biggest ones we could have changed:

Define why. Defining the why is step one of any research project or initiative. We have to understand why we are doing the research. I often include the why from the user perspective, as well as the business perspective. The why can be a simple paragraph explaining the reasoning behind the project.
Start with a problem statement. When you start any user research project, it is essential, to begin with a problem statement. A problem statement is a central question that has to be answered by the research findings. In this case, we started with a solution, which led us down the wrong path. By starting with a problem, you are much more likely to solve an actual user need. Learn more about reframing a solution to a problem statement.
Consider the objectives. By starting with goals, we can understand what we are expecting to learn from the research. We can ask ourselves what we are trying to learn and what we would like the research to achieve. Objectives keep your research on track and align everyone with an expected outcome. Learn more about writing objectives.

Since we skipped straight to testing a solution, we missed these crucial steps. By doing this, we ended up with a suboptimal experience for our users because we didn’t understand what their problems were and what they needed. It is an easy trap to fall into, but easy to avoid by following those steps. Instead, the process should have looked much more like this:

The overall process for choosing a research methodology

Defining the basics

With the knowledge from above, the outcome of that research project would have been much different. However, I still had the idea that there were only two types of user research: usability testing or user interviews. If only I could have told myself about all the different methodologies out there. Instead of time traveling, I will rewrite history to showcase the process I would go through today to choose a method for the same project.

The project was: Redesigning our hotel concierge platform navigation and information architecture of our features for hotel concierge.

Step 1 — Define why you are doing the research.
We have seen some users struggling when trying to navigate through our platform. Through user research, we have observed users hitting the cmd+f option to find what they are looking for to make the process faster. They are unable to find nested information as they are not sure where to look in our navigation. Also, users have been employing several hacks as opposed to using the features we have built.

Step 2 — Define a problem statement.
I am a hotel employee trying to fulfill a request for more towels for a guest, but I am unable to find that specific feature, which makes me feel frustrated. With this, I have to enter a generic request, and then write in what I specifically need and hope someone reads my notes.

Step 3 — Define the objectives.

Understand the general workflow of users and when they need access to particular pages/features
Discover how users are currently using the product
Uncover the limitations and pain points users are facing in different situations with the platform
Learn about any potential improvements in workflow, information architecture, or missing features

Choosing between generative and evaluative research

Now here is where things became fuzzy, and where I want to dive deeper. Although now I have a good understanding of which methodologies would have been best for answering the question and objectives, it didn’t always come so intuitively.

Since then, I have seen four significant types of objectives:

Understanding more deeply about people’s day-to-day lives, regardless of a product/service
Discovering how a person uses a product/service and how that product/service impacts their daily lives (positively and negatively)
Evaluating how a product/service works when a person is using it (pain points and joys)
Assessing how a product/service could be structured to impact a person’s daily life positively

By breaking these objectives down, we can then choose between generative research and evaluative research.

Generative research allows a deep understanding of who our users are (inside and outside of a product/service). We can learn what they experience in their everyday lives. It allows us to see users as human, beyond their interaction with a product/service.
Evaluative research is about assessing how a product/service works when placed in front of a user. It isn’t merely about functionality, but also about findability, efficiency, and the emotions associated with using the product/service. Many people think evaluative research = usability testing, but it goes much further than that
Hybrid research is a combination of generative research and evaluative research. The example I’ve used above can very well end up being a hybrid method. Hybrid research helps us simultaneously understand our users, as well as how a product/service is performing. Now, this is not magic and, since it covers both spaces at once, it does not go as deeply into a place of understanding or evaluation. I will focus on hybrid research in a future article, as it is a more advanced technique.

Choosing between generative and evaluative research

Once we understand what type of research we are looking to conduct, we can better understand which methodology ties best to our objectives. I have included a chart of methods I have used, and how they relate to the above goals I have realized.

If you are curious, for the research project I mentioned above, I would have suggested the following methods:

Contextual inquiry
Walk the store interview
Card sorting

Now, none of this is an exact science, and your opinions may differ on this, which is fantastic! I would love to hear about how you approach this differently.

Interested in more user research? I teach an Introduction to User Research Course and am available for one-on-one mentoring. Check out the User Research Academy. Please join the User Research Academy Slack Community for more updates, postings, and Q&A sessions

2/7/20 2/7/20

5 ways to reframe a solution to a problem statement

And why this is essential for user research.

Very rarely do I get handed a problem statement. Whenever colleagues want to engage with me in a research project, they have already landed at a solution. I am not saying this is bad because solutions are a comfortable place to start. They are easy. However, they limit our potential and possibilities. I’ll give an example of something I hear frequently:

“We want to build a share feature for our users so they can share their trip ideas or details to friends and family. Can you research this for me?”

I’m sure this statement resonates with many user researchers. It is a tough situation to be in, and one that happens far too often. If this is the type of statement you are consistently getting from colleagues, you have an excellent opportunity to teach a new mindset. You can teach the mentality of approaching ideas from the problem-space before solution-space.

Why is reframing the solution important?

I’m going to pick on Google, for once, because I believe Google has enough self-confidence to take a small hit. Let’s talk about Google Glass.

To be upfront, Google Glass was a failure. The product did not succeed in the slightest. There are several different reasons for the flop, but we will focus on one in particular: Google Glass did not solve a problem for users.

There were many assumptions made about Google Glass, which were not validated with users. The conjectures led to many questions and, ultimately, to a product that did not correlate with the user’s needs or goals. There was a lack of clarity on:

Who would use Google Glass
When/where people would use Google Glass
Why people would use Google Glass
What value Google Glass brought to people

These are a lot of essential questions that were left unanswered. With this approach, there is a substantial likelihood that products will fail. If there is no value to users, they won’t use it.

And the best way to determine value? Start with a problem, not with a solution.

By understanding the problem-space before coming up with a solution, we allow for:

Understanding a user’s actual problem and need
Brainstorming ideas that are relevant for users
More possibilities to help with a particular problem
A higher likelihood of a successful product/feature

The best way to understand the problem space is by conducting generative research sessions with users. Although I won’t cover generative research here, please take a look at my comprehensive guide to generative research.

Picking apart the solution

We’ve established how important it is to start with a problem rather than a solution. However, many times, we are faced with a solution, as in the first example. Let’s take a look at that again:

“We want to build a share feature for our users so they can share their trip ideas or details to friends and family.”

Let’s assume there has not been any user research done on this topic of “sharing.” There are a few things that are not ideal with this statement:

There is no clarity on what the problem is we are trying to solve
We jump directly into a solution
Users wanting to share their trip ideas or details is an assumption
Sharing might not be the only or best solution
Our thoughts should come from generative research

But, lets base this scenario in reality. We’ve received this solution and need to test it. We can go in one of two directions:

Usability or concept test the solution
Take a step back and consider the problem

I know the latter isn’t always possible, which is why I included the first option. Sometimes the idea/feature/product is too far down the production line, and we have to do damage control. The best we can do is usability test the solution to understand what we can improve in further iterations.

However, sometimes it isn’t too late, and we can take a step back and consider the problem. We can ask the question: “what problem is this solution trying to solve?”

Tools to reframe the solution -> problem statement

This kind of question may be much easier for researchers to think about since we are used to doing this daily. For our colleagues, it might not come as naturally. Luckily, there are a few different methods we can employ to help us reframe the solution.

‘How Might We’ (HMW) Statements
This method is very trendy in the UX world, and for a good reason. ‘How Might We’ statements allow us to reframe our insights and thoughts into a broader context. We use these types of statements because they can get us thinking about the problem from many different angles, and they open the door to creativity. They also ensure we are thinking about an actual user need versus just coming up with cool ideas. Let’s take the sharing example and recontextualize it with HMW statements:

Investigative Stories
Investigative stories enable us to become detectives when thinking about a problem. With this method, you employ a journalist-type view of the situation. By looking into all of the different aspects of the problem, you can get a holistic picture of what you are trying to tackle. After you answer all of the questions, you can create a story of your user, which helps to understand the actual user need/problem.

Unpacking Assumptions
This is a technique I have been practicing and teaching for many years now. It is one of my favorites as it plays very well into my Buddhism practice. Before I begin thinking about solutions, I list all of the different assumptions I/we have about the user. It consists of everything you think you know. If we take the example above, the list might look like this:

Users want to share trip details with friends and family
Users are booking travel for a group of people
Users need other’s opinions on their trip options/details
Users find their current method of sharing details painful
Users want a new way to share trip details with others

The Six Thinking Hats Model
Dr. Edward de Bono originally introduced the ‘Six Thinking Hats’ model. This method examines problems/concepts from many different viewpoints to get a holistic understanding. Getting colleagues to engage in this kind of exercise can be complicated, but also extremely rewarding. The thinking hats/roleplaying game gets people into the minds of users. Each person wears a hat:

White hat: Facts
- What information do we have?
- What hasn’t worked in the past?
- What information is missing that we need?
- What are the weaknesses?
Green hat: Creativity
- What are the other angles we are missing?
- What are the alternatives?
- What are the next steps?
Yellow hat: Benefits
- What are all the benefits of the different options?
- What solutions would work?
- What is the best-case scenario (for users and the business)?
Black hat: Risk
- What are the different risks of each option?
- What solutions would not work?
- What is the worst-case scenario (for users and the business)?
Red hat: Feelings
- How do the options make you feel (from the user’s perspective)?
- What do you like about the options?
- What don’t you like about the options?
Blue hat: Process
- Where are we now?
- What other work needs to be done?
- What is the next step?

Research Plan Template
One of the most significant pieces of advice I can give is to create a template that prompts colleagues to think of the problem statement before they even come to you. An additional way I do this is by sending a research plan template for colleagues to fill out before we meet to discuss features. This template starts with the problem we are trying to tackle, rather than the solution. With this, you can start discussions from the problem-space. Check out my research plan template.

Writing the problem statement

Once you have done these exercises, you can write some problem statements. I mentioned the ‘How Might We’ formula above, but I also use some other problem statement formulas:

I am (persona/role) trying to (do X) but (barrier/problem) because (x), which makes me feel (emotion)
I am a mom trying to book a flight ticket home for my daughter, but I don’t know her college schedule, so I am unsure which dates to book for her, which makes me feel frustrated.
I am (a persona/in a situation) who needs a way to (user need) because (current problem)
I am traveling with a group of friends and need a way to coordinate everyone’s schedules so that I can pick the best date/time for everyone

You don’t have to use the exact formula (I deviated above), but it is an excellent framework to keep in mind. Try to avoid proposing solutions during this phase, as it is an easy trap to fall into. Keep your focus on the problem and facts.

Overall, getting people to shift from the solution-space to the problem-space is a considerable challenge, but one worth confronting. If you are continually urging others to think about the problem before the solution, they will start to adapt to that mindset. By instilling these practices, and with some time, you can transform the mindset of an organization to approach ideas from the problem-space. Not only does this make your life as a user researcher easier, but it also turns the company into a much more user-centric culture.

If you liked this article, you may also find these interesting:

Interested in more user research? I teach an Introduction to User Research Course and am available for one-on-one mentoring. Check out the User Research Academy.

Please join the User Research Academy Slack Community for more updates, postings, and Q&A sessions

1/7/20 1/7/20

Should user researchers give feedback to teams?

Where does user research feedback start?

The other day, I saw a bad prototype. It came up during a design critique. We were going to start user testing that very prototype the next day. I wrote down my notes and mentioned it to the designer.

The problems I saw were:

Typos
Simple design flaws (different colors for the same hierarchical information)
Information that made no sense in the context

For me, these were easy things to fix. I wanted to give the designer this feedback for two reasons:

To make the design cleaner, and to encourage attention to detail
To ensure we were getting the right feedback from the users

The second reasoning behind this, for me, is more important. When I present a prototype to users, I would rather not spend the precious time we have with them confused about easily changed information. Or information that doesn’t matter in the test. It can be a big waste of time, and participants can get stuck on these small inconsistencies.

What happened in this case? Unfortunately, the designs didn’t get changed in time. We tested the older designs with the issues I mentioned. While we did get some great feedback on the user experience, there was some time wasted on those smaller inconsistencies. I had to explain them and wave them off as small mistakes that don’t matter. However, it made the interviewing experience seem less productive and prepared. Almost every user recognized the changes I had asked to be made.

Regardless, again, we still received great information from users, but it felt clunky to explain away the prototype. Now, I know prototypes are supposed to be far from perfect, but this felt beyond the usual prototype spiel I give.

So, my biggest question in this case: is it okay for the researcher to request the changes I did? Or is it on the researcher to perform the interview in a manner where these inconsistencies don’t matter to the user?

When should user researchers give feedback?

This particular case made me question when, and at what level, user researchers should give feedback to our teams. I generally give feedback during the following opportunities:

During the idea/concept phase
During the prototype phase
After a design is completed
Synthesis from research sessions

I’m not entirely sure if this list encompasses every opportunity, but it was my starting off point. I don’t want designers or other team members to think I am an expert in UI/UX (or any other field, for that matter) or that I am overstepping boundaries.

Here is how I give feedback at each of these steps:

During the idea/concept phase. I do my best to ensure teams come to me with ideas very early on in the development process so we are able to test the viability of the idea with users. When they come to me with ideas, they are generally solutions, rather than problems. I ask them to come up with the problem they are trying to better understand and the questions they would ask during to find out more information. Some teams have come to me with a fully developed idea, which I knew would not stick with users, or solve any pain points. In this case, I was new to the company, so I was forced to test it with users. It proved the point that it is important to do some upfront user testing before we come with fully built solutions based on assumptions. Now, when people come to me with solutions, I request they go back to the drawing board and start with a user problem, and questions they would like to ask.
During the prototype phase. This is similar to the example I gave above with the prototype. I try to get a look at all prototypes before we put them in front of users. I will have the designer walk me through each screen, and I will point out any small inconsistencies. This gives the designer a second pair of eyes on the designs and helps ensure the design and flow make sense. Prototypes can still be “messy,” as in low-fidelity, but they need to make sense. We don’t want to waste time having users comment on small things that are insignificant to the usability test.
After a design is completed. This is where the whole feedback concept starts to get tricky for me. Once a design has completed user testing and is off into the wild world of being “live,” what do we do? Since it was already user-tested, do we have the right to give additional feedback? For this stage, I will wait a bit and then follow-up with any feedback we are receiving on the particular design (or feature). If the design did not go through user testing, I will test at this stage and do a heuristic evaluation to give some additional feedback to the designer.
Synthesis from research sessions. And finally, synthesis. Maybe for some, this is the most straightforward but, for me, it can get complex. We all talk about how synthesis is one of the most important parts of the user research role, but rarely is it discussed in full. At this point, how I understand synthesis is as follows: we digest and analyze the research sessions, and then give “actionable recommendations” on what should come next. What does this mean? Are we telling people what to do? What are we recommending? This brings me to my next point…

At what level should we give feedback?

Since the first three opportunities for feedback are much more straightforward, I will focus on synthesis for this particular case. I have the following questions when it comes to giving feedback (specifically targeting synthesis):

What should we be producing? Action items/recommendations?
What are the action items/recommendations?
How far should we go with action items/recommendations?

I truly believe user research isn’t about giving people answers, it is about giving people tools to better contextualize something. We aren’t meant to “tell people what to do.” So, with this in mind, what are we supposed to be writing in terms of recommendations?

When I tested the aforementioned solution (which was a huge feature), it was glaringly obvious our users would not find it helpful or useful. And they would not pay for it. There were a few aspects of it they liked, but, by in large, it wasn’t sticking with them. They simply were not interested, and would rather have the company work on other features or improvements.

When I received those results, I wasn’t entirely sure what to do. The solution was already half-way built, and the team had spent a good amount of time working on it. I decided to give my honest recommendation: stop working on this immediately and pivot to working on other, more impactful, areas. I gave some ideas on how we could change the idea to suit our users better, but it should not be a high priority. I was lucky to have enough people on my side to be met with little resistance.

However, when it comes to these tests, I always wonder what level we should give this feedback and these recommendations? I often state the recommendations as problems instead of solutions:

“User is unable to locate the “pay” button or move to the next step” versus “move the “pay” button to higher on the page or make it a bolder color.”

This still gives enough flexibility for someone to make a better decision, without me telling them exactly what to do. However, as I mentioned, I also made an honest recommendation of not continuing on with a product. I’m not entirely sure what the balance is, but I am sure there is one. But I would still love to know…

How do you give feedback as a user researcher?

If you liked this article, you may also find these interesting:

If you are interested, please join the User Research Academy Slack Community for more updates, postings, and Q&A sessions :)

12/13/19 12/13/19

Jobs to be done personas

Focus on the job, and all else will follow.

There are so many different types of personas, it can sometimes make your head spin. And, oftentimes, people don’t generally like any of them. When describing personas, people use “useless,” “confusing,” “stupid,” and many other negative words. I have had people tell me to use them and others who tell me to avoid them…and, like most, people who throw their hands in the air, unsure of what to do.

Regardless, I have a soft spot for personas. I truly believe, with the right lens and mindset, they are a great tool to be used for innovation and decision-making. Personas are fictional characters that embody the typical characteristics of different user groups. See here: they are a fictional embodiment. We have forgotten that personas are a tool to be used, not the answer to all of our questions, or the solution to all of our problems.

How I use Personas

The reason I create personas is to help my teammates understand the story and the relevant context of our users. I’m not just trying to shove as much information as possible on to an A4 poster. I want to help my team make decisions with the information I give them. I believe, if we are mindful of how we use them, they can be excellent tools that can help guide teams towards better decision-making and innovation

What are the benefits?

Set the focus
Empower product and design to make better decisions
Builds empathy for the users and promotes a user-centered mindset
Make our assumptions explicit
Keeps users at the center
Show the realistic
Helps to inform user stories
Aligning the organization on the same language

Now on to Jobs To Be Done Personas…

First off, let’s start off with a small explanation of Jobs To Be Done:

Jobs To Be Done encompasses the concept that users are trying to get something done, and that they “hire” certain products or services in order to make progress towards accomplishing goals. Usually, we all want to be better than we are in a given moment. We all have goals that lead us closer to our ideal self. So, we will look for, and purchase, certain products or services when our current reality does not match what we want for the future.

With Jobs To Be Done, we are completely solution-agnostic. We aren’t even thinking about our product or any products in general. We are focused solely on people’s motivations and how they act. We try to identify their anxieties and the deeper reasons of why they act the way they do, and why they respond in certain ways.

For instance, if we have a travel product that allows people to buy all types of travel tickets online (bus, car, plane, scooter, you name it), we focus our research on how people are using this website/app and how they buy tickets online. However, when you think about it, people didn’t buy tickets online in the past, so this kind of platform didn’t exist back then. And in ten years, there will probably be a new way to buy tickets we don’t even know about yet.

The point is, people still need to travel for various reasons. That job remained stable throughout the years, and the solutions created for it have changed. People will use different products that allow them to get their job done in a faster, better way.

So, if you are just focused on the best ticket buying experience, you will be relevant for some time, but, eventually, competitors will overtake you. Instead, we need to focus on the job people are trying to accomplish: getting from one place to another in the fastest and easiest way possible.

If you look up Jobs To Be Done Personas, you will often find articles called: “jobs to be done versus personas.” When you put these two concepts together, it is usually an either/or conversation. They don’t seem to be able to exist together, in harmony. Most people believe you either do JTBD research or you have personas. I, however, disagree. I believe there is space for a JTBD persona.

How to Create JTBD Personas

Creating JTBD personas is fairly similar to the process of creating other personas. There is a lot of research, analysis, and synthesis. Below is how I created our first JTBD personas.

Do JTBD research. The first, and most important step, in creating these personas is to do the foundational research. I interviewed around 20 people in this style of interviewing before I started to come up with the concrete personas. Some people say you can make them with less, some only with more. Either way, it is whenever you feel comfortable enough with the data you have gathered. Keep in mind, these will keep changing, so nothing is set in stone. If you have never done JTBD research, feel free to read my guide: https://dscout.com/people-nerds/the-jobs-to-be-done-interviewing-style-understanding-who-users-are-trying-to-become
Define the goal of the persona. What is it that you are trying to accomplish with your persona? There are several different goals you can have when creating a persona. Below are the various reasons I have created them:

Show colleagues who our customers are
Highlight the behaviors of user groups
Present high-level motivations of users to highlight the why behind certain behaviors
Tell a story of a user’s life and how they operate when they want to accomplish a certain task

3. Decide on the information to include. I always do this before synthesis so I know what I am looking for when I comb through the data. JTBD personas include slightly different information than what I would put in more “traditional” personas.

Motivations. I believe motivations are the most important part of the JTBD interviews, and the most important information to include in the personas. JTBD highlights the motivations behind our actions. I separate motivations into three different categories: ideal self, experiential self, and social self.

Ideal self motivations: What is getting the person closer to their ideal self? What are they doing to get there?

Experiential self: What is that person trying to experience in order to get closer to the ideal self? How do they want their environment to impact you?

Social self motivations: How is the person trying to be seen by others? How do they want their relationships to function and get them closer to the ideal self?

Values. What does the person, at a high-level, value in day-to-day life? What are the concepts or ideas that are important to them?
Needs. What does the person need on a day-to-day basis? What is important for them to have to accomplish their goals?
Pain points. What is a painful experience for that individual? What do they try to avoid? What causes them anxieties?
Goal (or job). What is the goal (or job) they are trying to accomplish? What is the outcome they would like to achieve?

4. Pull together the information. Read through all of the transcripts and notes that you have gathered from the research. One thing I do is tagging each important transcript line with one of the above points (motivation, pain point, goal, etc.). See an example below:

I then will create an affinity diagram with all the research I conducted and once I am done tagging. This is where I group all of the similar patterns I hear and cluster them into categories, such as motivations, values, pain points, etc. See an example below:

5. Outline the persona. Create version one of the persona. Include all the information you think might be relevant. Again, we are trying to help teams understand motivations of why someone acts the way they do, why they might use a product, and also tell a story of that person.

6. Design the persona. Putting this information into a visual format is the last step. Try not to do this before you have all the information together. Additionally, you don’t have to go crazy with graphics, as they usually aren’t helpful for a team. There are these bars that indicate people’s personality traits on a spectrum, and they are infuriating to me. They aren’t beneficial at all to a team making a decision. Keep the persona simple.

My most significant piece of advice when creating a persona: don’t just blindly use one of the many templates available. Although some may be beautiful (while others are undoubtedly ugly), they won’t serve you. First, come up with your goal and the relevant information needed for that goal. Then start creating the visual.

My second piece of advice: don’t make anything up. Delete all false or dramatized information. It will only get in your way when making decisions and could lead the team astray. You want your personas to be based on real user data.

I will do a more in-depth case study highlighting what this might look like with a real-life example. Until then, I hope this was helpful!