How I got into User Research

From the first time I heard about user research to my first internship.

Source

Since I was young, I had my sights set on becoming a vet or a psychologist. My parents laughed at the thought of me being a vet. If I saw an animal even stub their paw, I’d be in tears. I spent a brief time shadowing a vet when I was younger. I lasted about 2 hours before I ran out crying, swearing off my future as a veterinarian. I then dove into my second option, psychology.

During my Master’s program in Psychology, I worked in a mental hospital a few days a week. The patients I longed to work with had mental illnesses such as schizophrenia, bipolar disorder, and dissociative identity disorder (it exists). I worked with these patients in the hospital and criminals in a neighboring hospital to get this experience. Over the two years, I completely burnt out and withdrew my Ph.D. applications.

I was terrified. I had some debt from school and no viable career ahead. All of my plans flew out the window. I grasped at straws to find the right job for me. Could I be an interior decorator? Or a motivational coach? Or maybe I could start my own business. I spoke to a few career coaches and still felt lost.

By some crazy stroke of luck, I user experience fell into my life. I was at a party (wallowing, I’m sure) when someone mentioned user experience. They told me it was about making website and app experiences better for whoever uses them. This idea was intriguing. I went home early and looked up this potential new career.

I fell in love

The instant I Googled user experience, I couldn’t stop reading. Everything about the topic spoke to me. There was something about user experience. It felt like a beautiful mix of psychology and technology. I believed I could use what I learned during my Master’s program and apply it to user experience. I thought it would be an excellent fit for me. Plus, it gave me a sense of control. I finally knew what I could do for the rest of my life.

The only problem was I cannot draw. Not for the life of me. To this day, I can’t complete a circle (it always overlaps) or draw easy shapes. Visualization is one of my weakest points. I can see what I want in my mind, but it is almost impossible for me to put it on paper (physical or digital).

I ignored this small hiccup and figured I could learn how to become a designer. Spoiler alert: that didn’t work. However, I am so grateful for moving forward since it brought me to user research.

How did I start?

I immediately filled my shopping cart on Amazon with as many books as I could find, including:

• Interviewing Users: How to Uncover Compelling Insights by Steve Portigal
• Mental Models: Aligning Design Strategy with Human Behavior by Indi Young
• Measuring the User Experience (2nd Edition): Collecting, Analyzing and Presenting Usability Metrics by William Albert and Thomas Tullis
• Handbook of Usability Testing: How to Plan, Design and Conduct Effective Tests by Jeffrey Rubin

And many more!

I then scoured the internet for any guides or blogs on the subject of user experience. I sat on Dribbble for ages, scrolling through people’s designs; I found designs on Pinterest (I still have a UX Pinterest Board — although now renamed to User Research); I read guides on how to be a user experience designer. Anything user experience, I ingested.

I then found General Assembly. General Assembly is an education platform geared towards tech- and business-related careers, such as coding, designing, digital marketing, and data analytics. They had two options for a user experience course: a full-time immersive course and a part-time course. Since I was working, I chose the part-time course. I started counting down the days until it started.

What was it like?

The General Assembly course primarily focused on user experience design. We learned about design thinking, prototypes, and different tools such as Sketch. Finally, we came to the section on user research. User research fascinated me. I wanted to know everything about the topic. While my design skills were lacking, my background in Psychology bolstered my interviewing skills. I felt so comfortable conducting user research (although I had no idea how much I needed to grow).

That course rocketed my interest and love for user research. I didn’t get quite enough training in research, but it motivated me to continue my learning and quest to become a user researcher.

I want to make a quick note here: you don’t have to take a boot camp or expensive course with a certificate to get into user research. You don’t need a MA or Ph.D. (although for some companies such as Facebook, it makes it easier to get an interview). All you need curiosity, compassion, and a drive to learn and practice user research.

That course did not (I repeat, did not) get me my first user research job. Hard work and practice made this career a reality, and I will outline each step I took to get into the field.

What happened next?

If only the world were magical and I landed a job right after finishing the General Assembly course. That was not the case. General Assembly was great for understanding that user research was a viable career and the basics. I had a lot of work to do after that course. My immediate next step was to start applying to jobs.

But, before that, I noticed a small caveat present on almost each job description: portfolio required. My heart sunk once I figured out what a portfolio was. I had my General Assembly project, but that wasn’t going to help me with user research jobs. I knew it wouldn’t be easy, but here is what I did:

1. I volunteered for a pet adoption agency and did some user research for them. I had already been volunteering for them, so I knew who to reach out to. I also did this for a few other pet adoption agencies where I didn’t have any contacts — I do so without them knowing, but it served as a portfolio piece. To get participants, I used guerilla research by sitting in a Starbucks with a sign that said, “If you have adopted in a pet in the last six months, talk to me for twenty minutes, and I will buy you a coffee and treat.” It was surprisingly successful

2. I picked two different areas I felt strongly about and researched them. The first was a video game app, and the second was a writing app. I had friends/acquaintances I could use to conduct real user research on both topics. These two topics were passion projects, and my ability to research with friends of friends helped my interview skills.

3. I joined a few different hackathons to see what it was like working with other people (and the networking was great). For one hackathon, I was able to work with a product manager, product marketing manager, UX designer, and developer. Although we didn’t win the hackathon, I felt to a degree what it was like to work in a tech company for that entire weekend.

4. I offered to do small freelance projects for friends and family. I also asked my network if anyone needed help on projects. With this, I was able to work on three small projects for others. I only added one of them to my portfolio because of the size, but it was nice to have the potential to talk about multiple projects.

With all of this work, I created a portfolio of a few different studies. I felt good about my portfolio. Was the portfolio perfect? No. Did it include everything a portfolio should consist of? No. Everyone starts somewhere. But I had case studies. I had something to present and talk about during my interviews.

Are you trying to break into user research?

These are the steps I encourage you to go through:

1. Find a non-profit, charity, or something similar that you can create a project around. You don’t need to know any contacts at the given organization, nor do you have to tell them you are researching their website/app/platform. You can always send them any recommendations after you are done and see what happens!
2. Volunteer for a local small business or organization to see if you can help them. I went to a few different local shops in New York City to see if they needed user research help. I had a few leads on this but, ultimately, spent my time on the non-profit organizations. Due to COVID, here is an alternative: You can also find Facebook groups on specific topics and reach out to people via Facebook or LinkedIn.
3. Pick a true passion project and create a case study around this topic.
4. Join hackathons (even virtual) to experience working with others you would work with as a user researcher.
5. Reach out to friends/family, and friends of friends/family to see if you can volunteer your time to research their websites/apps/platforms.

These are actionable steps you can take to get more experience in user research before your first role, or even between roles!

If you’re looking for some guidance on creating a case study, check out my user research case study outline freebie.

—

Interested in all things user research? Sign up to my newsletter and join the slack community! And check out User Research Academy for freebies, courses, and more!

The UX Collective donates US$1 for each article published in our platform. This story contributed to Bay Area Black Designers: a professional development community for Black people who are digital designers and researchers in the San Francisco Bay Area. By joining together in community, members share inspiration, connection, peer mentorship, professional development, resources, feedback, support, and resilience. Silence against systemic racism is not an option. Build the design community you believe in.

I sat down with a bunch of product managers for a conversation

And the questions I received were fascinating.

Source

The other day I was invited to an informal product meetup as an informal speaker. My fiancé (yes, we got engaged!) is a product manager in Berlin, and he meets about once a month with a group of about ten other product managers. They swap stories and experiences, discuss tools, and seek advice from one another on all things product management. I, personally, imagined meetings filled with JIRA talk and phrases like, “it depends…”

A few weeks ago he asked me if I would like to come in and talk with the group about user research. He asked me if I’d be willing to answer as many questions as possible and be an open book about user research during those two hours. I said yes, but I was a bit apprehensive. A group of ten product managers and one user researcher.

I was mostly nervous that I would be unable to answer their questions well and that I would give them the impression that user research was not valuable. Imposter syndrome crept in on the day of, but I swatted it away. As researchers, we can struggle so much with defending and explaining our craft. I didn’t want to leave a bad impression or mistakenly convince people that user research is unnecessary.

We arrived at the apartment. Cheese and wine were served as everyone settled into their seats. Soon, heads turned to me. Someone asked me if I prepared anything (spoiler: I hadn’t). I explained that I wanted this to be an open forum for them to ask any questions they wanted to and I would do my best to answer.

It was over two and a half hours later that we noticed the time.

What questions did I get?

I had quite a lot of questions from the product managers:

• When do I reach out for help on user research?
• When should I hire a user researcher?
• What should I look for when hiring a user researcher?
• How do I pitch hiring a user researcher to the company?
• What should I expect from a user researcher?
• How should I work with a user researcher?
• Should a user researcher know how to do both qualitative and quantitative data?
• How should I get started with some user research (if I don’t have a researcher our the team)?
• How should I pitch the value of user research to a company/organization?
• What questions should I ask a user researcher?

What surprised me?

There is bad user research conducted out there. Just like in any profession, it can go wrong, and this can leave a bad taste in people’s mouths. A few product managers brought up some difficult times they experienced with researchers. I am sure you can think of some tough situations with product managers as well.

Things can go wrong in all professions and you can have less than ideal interactions with anyone. However, user research is still relatively new as a profession and a role. We have had to push for user research to be done, constantly pitch the value of user research, and defend the validity/reliability.

Based on these experiences, product managers were curious about what user researchers could do that they couldn’t. When they’ve seen user research done badly, they think, what is the point of a user researcher?

Other topics that surprised me:

• Product managers want to work with us.
• Product managers don’t necessarily know how to approach us. They deal a lot more with business problems versus customer problems. As researchers, we need to help our stakeholders translate the business into customer-centric thoughts
• We don’t have enough open communication with product managers. We can’t take these questions too personally. If product managers don’t see the value in user research, it is much more helpful they are honest and ask us, rather than just thinking this to themselves. If we are willing to hear the hard questions and give meaningful answers, we can educate in a positive way
• We have a lot more in common than we think. Product managers and user researchers want the business to succeed — we both want to solve problems and create a positive impact. If we work together better, we can do this much more effectively and efficiently.

What is next?

I left the meeting buzzing, despite drinking water all night and abstaining from the wine (I wanted my mind to be sharp). I wondered to myself, why don’t we have these open and honest communications more often?

We constantly work with product managers, among many other stakeholders, all the time, but we don’t seem to have much open communication. My biggest takeaway is this:

Speak with product managers as if they were your users, stakeholders, and peers. Be open to their questions (even if they hurt), and be honest and kind with your responses. Educate and learn. Together, we can truly work so well.

Here are some additional action items I will take away from this:

• Meet more often with product managers — I am going to try to get back into the group as a follow-up but, aside from that, I will start attending product management conferences, webinars, and meetups to better understand them
• Take these questions and write down answers for my stakeholders, so they can easily reference them. For example, the “when should I reach out” question can be answered in a Google Slides presentation that stakeholders can later reference!
• Writing an article about user research geared toward product managers that answers these questions, and has additional tips

Take some time to sit down and chat with your product managers. Let them as you the hard questions and give you feedback. We can continue to educate each other and work as a team.

The UX Collective donates US$1 for each article published in our platform. This story contributed to Bay Area Black Designers: a professional development community for Black people who are digital designers and researchers in the San Francisco Bay Area. By joining together in community, members share inspiration, connection, peer mentorship, professional development, resources, feedback, support, and resilience. Silence against systemic racism is not an option. Build the design community you believe in.

The life and timelines of research projects

Source

When I was first starting as a user researcher, I had no basis of understanding how projects worked in tech companies. The only mental model I had of research was academia. I was unable to find examples of research projects online and had no idea how they worked. Eventually, I learned that once I saw research in action. However, it would have been highly beneficial for me to understand the overall process beforehand.

For those who have not yet worked in a tech or product company, it can be hard to envision what projects would look like. Previously, I wrote an article about a week in the life of a user researcher. With this article, however, I will shine some light on a seemingly mysterious process only available behind closed doors.

Let’s begin

Imagine you are working at Best Friends Animal Society, an animal welfare organization in America dedicated to ending companion animal deaths in shelters.

Sidenote: if you have a chance to volunteer there, I highly recommend it!

Second sidenote: tears came while writing this article due to my absolute love for animals. If you are looking to do a portfolio piece, consider volunteering for a charity or non-profit organization!

You are a user researcher at this organization. You work with different departments, but primarily tech, product, and marketing. You are continually working with product managers, UX designers, and developers to empower teams to make the best decisions for the organization.

Usually, your research projects come to you in two different ways:

1. From precious research insights
2. From an internal stakeholder

For this project, we will focus on the second. A research project lands on your desk from one of the product managers. Recently, we have seen a drop in the usage of the “adopt” area of the website. The product manager looked at Google Analytics and saw a considerable decrease in time on page and click-thru rate. The team in charge of that functionality wants to understand what is going wrong.

What would you do?

The life of a research project

Before we dive into the timelines with this project, it is essential to look at the holistic life of a typical user research project.

The steps a user researcher generally takes for a research project

I want to stress that this is a typical journey and research won’t always happen in such a linear (and clean) fashion. Often, you have to skip straight to recruitment because of timelines or conduct research while recruiting. This overview is simplified but gets the overarching message across of the steps a researcher generally takes during a project.

As you can see, before you jump right into research, you go through the steps of really understanding and defining the problem and goals of the study. You are meeting with stakeholders to ensure alignment and answer clarifying questions.

Arguably, steps 1–6 are the most important as they drive the project in a particular direction. For example, the research statement and goals help define the methodology, recruitment, and the discussion guide. This building block effect is why it is vital to take these steps before diving into research.

For the ease of this article, let’s come to some conclusions:

Research statement:

• We want to better understand the user’s mental models when it comes to animal adoption to improve our current website

Research goals:

• Understand how users are thinking about adoption, in general
• Uncover how users are currently interacting with the adoption feature on our website
• Identify pain points in the adoption process, in general, and on our website
• Discover other apps/websites users are interacting with to adopt animals and their experience with them

Now comes the picking of methodologies. I have previously written on this subject, and it is a difficult one to explain. However, you use your goals to help guide you to the write methodology, which lands in one of three buckets:

1. Purely generative research
2. Purely evaluative research
3. A hybrid between generative & evaluative research

Here is a high-level and simplified process I go through when deciding:

1. Are we looking to understand mental models, frustrations with current methods, or how users think about concepts in general? Are we looking agnostic of our product? -> Generative research
2. Are we trying to evaluate how users are interacting with a prototype or live product? Are we looking to see pain points associated with our particular product? -> Evaluative research
3. Is there a mix of both? -> Hybrid research

Once I place the goals into one of these three buckets, I can decide on the methodologies that would help answer these goals. The most common associations are:

Generative research

• 1x1 in-depth discussions (aka interviews)
• Contextual inquiry
• Mental models
• Customer journey interviewing
• JTBD

Evaluative research

• Usability testing
• A/B testing
• Competitive testing or analysis
• Benchmarking
• Surveys

Hybrid

• 1x1 in-depth discussion + usability testing
• Card sorting
• 1x1 in-depth discussion + follow-up survey

The different types of user research sessions

Timeline of a typical project

The timeline of a typical project can be over three to six weeks. As mentioned above, it can include just generative research (which is about four weeks total) or just usability testing (which is about three weeks). Another standard timeline is conducting generative research to understand the problem, and then creating insight-based prototypes to usability test.

The timeline of a typical user research study

As you can see, in the above image, you decided to run two separate studies. First, you run a 1x1 in-depth discussion to understand the following goals:

1. Understand how users are thinking about adoption, in general
2. Identify pain points in the adoption process, in general

After, you run a usability test to cover the following goals:

1. Uncover how users are currently interacting with the adoption feature on our website
2. Identify pain points in the adoption process on our website
3. Discover other apps/websites users are interacting with to adopt animals and their experience with them

In this evaluative study, you may also offer prototypes for users to test in addition to looking at the current live product. Or, you may have to run a separate evaluative study to test those prototypes.

Timeline of a hybrid project

Slightly different from a typical project is the hybrid project. I love hybrid projects; they are my absolute favorite types of studies to run. I feel like I get much more accomplished in each session than with the typical project timeline. I also use hybrid projects to highlight the importance of generative research.

The timeline of a hybrid user research study

In this instance, you run a more extensive study, including generative and evaluative research. In one session, you are conducting a 1x1 in-depth discussion followed by a usability test. In this study, you can answer the following goals:

1. Understand how users are thinking about adoption, in general
2. Uncover how users are currently interacting with the adoption feature on our website
3. Identify pain points in the adoption process, in general, and on our website
4. Discover other apps/websites users are interacting with to adopt animals and their experience with them

While I prefer hybrid projects, they are more advanced than the typical approach where you tackle one method. Hybrid projects take more careful planning and time management. While you are practicing and honing your craft, I recommend starting with a regular timeline and making your way to hybrid.

These timelines are not rules!

As I mentioned, these steps and timelines are ideal. You may have to do lean research in two weeks, or you might not have the budget to run two or three separate studies. In this case, you will have to choose between generative and evaluative research. Additionally, you may already have previous research the team could look into to answer the questions.

I offer these graphs and steps as guidance towards your next research project. They are not hard and fast rules to live by, and I encourage you to find the best process for yourself and your organization.

---

If you’re interested in joining an awesome community of user researchers, join the User Research Academy Slack. If this article piqued your interest, and you would like to learn more, check out the courses and webinars I offer.

The UX Collective donates US$1 for each article published in our platform. This story contributed to UX Para Minas Pretas (UX For Black Women), a Brazilian organization focused on promoting equity of Black women in the tech industry through initiatives of action, empowerment, and knowledge sharing. Silence against systemic racism is not an option. Build the design community you believe in.

How to interview as a user researcher if you aren’t (yet) a user researcher

When 1–3 years of experience is necessary for an internship

Source

A company’s expectations of user researchers are high. It is a competitive world out there, especially for people looking to break into the user research field. My heart sinks when I view intern or junior positions that require experience (stop doing this!) or act as though applicants should already be experts. While I believe we need to revamp our hiring process, this is the current situation.

I get so many questions from people trying to understand how to navigate what should be a simple interview process. Let me tell you. It is not easy. How are you supposed to have real-life experience and know everything as an intern or junior level? I have been in this field for over seven years and am still learning.

What I can offer are a few steps and pieces of advice to feel more prepared for your first user research interview. They won’t answer all the questions, but they might make you feel better walking into the session.

Have a story in mind

My first and most significant piece of advice is to have a story in mind before starting the session. This story should be an example of a project you completed. You want to have this story to help you answer the questions they will ask you. Ideally, in this story, you will have faced challenges and can reflect on what you would improve next time.

A great way to understand how to create a story is to look at other researcher’s portfolios. These portfolios will help you know how researchers work in a company.

Before I started working as a user researcher, I knew I had to create a portfolio piece. With this in mind, I volunteered to understand animal adoption and how people interact with animal rescue websites. I chose a company I volunteered with in the past. I conducted generative research to understand pet adoption in general and usability testing of the website. I also did a card sorting exercise to see the impact on navigation. I brainstormed some metrics I imagined the company would be tracking, such as click-through rate, in-person visit rate, contact rate, etc. Although I didn’t have access to their data, I wanted to show my understanding of business metrics.

I then used this story throughout my interviews as an intern and junior researcher. This story helped form my process and, although it was outside of a typical company, I had answers to why I chose the methods, how I recruited, and how I conducted the research. I also prepared reflections and challenges:

• Working alone is very difficult, and I was lucky to get some input from my friends who are designers
• I developed a report with findings to pitch to the company, but ultimately, did not have the opportunity to work with them
• I wasn’t working on a strict timeline, but I tried to simulate the timeframe in a company setting (I allocated five weeks for the entire project)

Understand how the company works (and company jargon)

It is challenging to demonstrate how you would fit into a team without a general understanding of the basics of how tech teams work. And without the jargon or terminologies that are common in the tech and user research industries. I recommend researching how tech teams work, how businesses work, and how user research can fit into organizations. This will make you stand out to recruiters and companies, as someone with a real passion and motivation to go above and beyond.

When I was newly interviewing, I found it very important to understand the timelines at a company. I delved into understanding how scrum, sprints, and agile worked since the majority of companies I applied to used those methods.

I read everything I could about how user research might work in these confines. I also learned about how long specific user research methods generally take. Although timelines can differ (through recruitment availability and tools the company has), here are the basic guidelines:

• Generative research methods can take 4–6 weeks
• Evaluative research methods (usability testing) can take 2–4 weeks
• Surveys can take 1–2 weeks
• Recruitment can take 1–2 weeks
• Alignment meetings can take up to 1 week

Some of these methods can overlap, but keep these in mind when determining a timeline. As a note, you can start researching before you finish recruiting to cut down timelines!

With this in mind, I tried my best to simulate how I would research projects at a company. When people asked me how I would deal with tight deadlines, I had mentioned my ideas on methods I could overlap and ways to speed up recruitment.

I wrote an article on the essential concepts, processes, and jargon to understand. Check it out here.

Know who you will work with and how you can collaborate

Understand the different roles you will work with. Some of the most common for a user researcher are:

• Product managers
• Designers
• Developers
• Marketing
• Account management
• Customer support

It is helpful if you have never worked in a typical company setting to research each of these roles. Understanding their day-to-day and their goals will help you know how you would work with them.

Let’s go back to the example I used above. I had never worked in a tech company. I had no idea what a product manager was.

Knowing the roles I was going to be working with, I could answer questions about when I would engage with them. I also felt more confident about how I would pitch user research value to these roles since I could speak to the goals they were trying to accomplish.

For example, even though I had never researched content copy, I knew that was something marketing worked on. That way, I could talk about how I might interact and help the marketing department. Additionally, I knew customer support had incoming calls about customer problems. I could then talk about how important it was for me to meet with them regularly to understand top issues, which would reduce their workload if fixed.

If you aren’t able to talk to the different people in these roles, look up job descriptions. These descriptions should help you understand what the responsibilities are and what they are trying to achieve.

Be prepared for a whiteboarding challenge

The whiteboard challenge for a user researcher can be a daunting task.

Your interviewers will give you a vague prompt and want you to provide them with an idea of your process. This exercise allows them to evaluate how you go about approaching a problem in a “real-life” setting, beyond your curated portfolio. Companies are increasingly valuing employees’ thought processes over their qualifications.

Understandably, this can be a challenging experience, which is why you should practice beforehand! Whiteboarding your approach is a great skill to have in general.

Check out my whiteboarding approach in this article. And practice!

Get ready to answer tough questions

There will be some questions that you might not be able to answer. Keep in mind, it is okay to say, “I don’t know.”

A lot of your answers will be hypotheticals. For instance, since I didn’t work with product managers in the past, I could only say how I might engage with them. By knowing the different roles, you can better hypothesize what you would do in various situations.

When I had to talk about how I would work with product managers, I answered based on my research. I knew I would have to involve them early, that I might have to explain the value of user research, that I would have to meet with them frequently to ensure user research was correctly involved. With this knowledge, I brainstormed ideas on how to answer those questions. I read articles on how people have tackled these issues in the past and used that as my hypothetical answer.

Here are some questions you should be prepared to answer:

1. What is your user research process?
2. How do you manage difficult stakeholders?
3. How do you convince stakeholders about the value/importance of user research?
4. If you had X goal, which methodology would you choose?
5. How do you select the right method for a project?
6. How do you turn insights into actions a team can take?
7. How do you ensure your deliverables are acted upon?
8. Talk about a challenge you have faced. What was it, and how did you overcome it?
9. What is the most challenging part of your research process? (For you and others)

The best approach for interviewing for a user research position is preparation. Take your time in understanding how user research works at companies, and think about how you would positively impact that organization through user research. If you are feeling stuck, read through another researcher’s portfolio or ask a fellow researcher to talk you through their process.

Torch the user research reports

Creative ways to share user research insights

Source

The first time I was asked to create a research report, I was lost. After spending my life writing academic reports, I was used to the long-winded, endless vocabulary in these presentations. Producing a report like that, as a user researcher, would prompt a lot of blank stares. And when I tried it, it did. After (many) failed attempts, I have finally found three ways to effectively share user research that is tangible, creative, and realistic for a user research team of one or fifty.

After creating and presenting my first user research report, I received many blank stares. I stood in front of the CEO, CTO, and head of product — shuffling through a pretty dull, and visually unappealing, deck. (To my humiliation, I’d used a preset gradient background). Each slide was filled with information I gathered from research sessions. And while it was helpful information, it was a lot. It wasn’t distilled down into anything digestible by upper-management.

That was when I learned my first lesson in presenting research: It doesn’t matter how useful the findings are if you aren’t able to communicate them effectively with your team. I also realized how easy it was to feel like an imposter in this field.

The worst thing you can do when you present research or create a research report is to have it sit in a folder somewhere, collecting dust. Insights need to be infused with excitement and urgency to promote action, and it is our job, as researchers, to do just that.

Now, whenever I share user research, I use the following formula:

Informative + actionable + fun = solid user research shareout

After many attempts at successfully sharing user research in an energetic and knowledgable, I have finally found five ways to effectively share user research that are tangible, creative, and realistic for a user research team of one or fifty.

Different ways to share insights

Demo desk: A “Demo Desk” is an area that allows employees to stop by and play with the current prototypes/ideas you are working on and give feedback. This will give other employees the chance to understand what the product team is working on and also give their input to impact your ideas. It is a great way to understand what user research is, how it is conducted, and how it affects your product/service.

Usability movie night: Usability movie night is an event you can host showing back-to-back video clips of usability tests, which brings everyone closer to the customers’ experience. Nothing makes us empathize with our customers more than watching them use our product. Whether they’re struggling to understand the UI, or they’re thrilled to interact with a new feature, observing the customer experience first hand is incredibly eye-opening. It helps teams plan where they should next focus their efforts.

Choose Your Own Adventure: The entire point of a CYOA is to bring someone through a story in a way that they feel very connected to the material. I have used CYOA to connect the product and tech teams to our users in a creative and digestible way. I craft stories and scenarios for our personas and allow the teams to make their way through these stories that reflect actual choices users must make. Shopify wrote about how they have had much success with this type of deliverable.

Research session snapshots: This type of summary I use is full of factual information and consists of direct recommendations or the next steps. I create a research session snapshot after a usability test, or when I am synthesizing a few generative research sessions for a team. Generally, in this scenario, we might not have time for a group synthesis session, or this could be a summary of the group synthesis session.

Canva infographics: Qualitative research summaries and reports can get very word-heavy. By breaking the words into infographics, we give a visual representation of what we learned, which is more engaging for those receiving our reports.

How to use these methods

Now that I have explained some of the different techniques I use to share user research insights and ideas, I will take you through how I set each of them up.

Demo desk

Necessary equipment includes iOS and Android test devices, Windows test device, Mac test device, desk area, feedback sheets, candy (for a reward!)

1. Find a central space in your office space where you have space to set up the various test devices relevant to your product/service
2. Speak with the designer(s) to understand any upcoming prototypes or concepts they want to test. The feedback you can generally expect from a demo desk is visual appeal, usability, typos, missing information, confusing information, and what is going well
3. Set up the prototypes of the test devices
4. Write up brief descriptions of each prototype, so people are aware of the context and a feedback sheet.
5. These are the steps I use to explain:

• Choose your preferred device
• Open up the prototype (all the current ones will be on the homepage/desktop)
• Read the brief context about the prototype you choose to test
• Be aware, it is a prototype, so things might not work as they do on our platform
• Play around with the prototype for 10–15 minutes (more is even better!)
• Fill out a feedback form
• Leave your feedback form in the demo desk feedback jar
• Take a piece of candy and have a wonderful day
• If you are having any trouble, come to Nikki’s desk and annoy her until she fixes it 😊

6. Write up a feedback sheet. Here is what I include:

• Date and prototype being tested
• What was confusing about this prototype/idea?
• What didn’t work?
• What was missing?
• What would you change to improve it (and a short answer on why)
• What was working well?
• What did you like about this prototype/idea?
• Any other feedback, comments, questions, or suggestions

7. Aggregate the input every week (or few days) and present to the designer(s)

Usability movie night:

1. Choose a theme for the night, such as a pattern you have seen in generative research, or a series of usability tests on a particular prototype
2. Create an introduction to the movie night, which gives the audience a context of the goal and objectives behind the research you were conducting
3. Find short and compelling video clips (you can do this while you synthesize) that clearly show user issues, confusion, unmet needs, or goals they are not able to achieve. I usually pull about 7–10 video clips, two minutes each. Put these together in succession to show the pain points or problems users were having. I also include two or three short clips at the end of positive remarks users had about the product/service.
4. Engage the audience in a game after they view the videos. This could be a role-playing game where people write down ideas on how to alleviate these pain points, and the top three ideas win a prize. Or, you could set up a scavenger hunt around the office, in which the clues correlate to problems or pain points the users had.
5. Order snacks, pizza, and beer for the team for the usability night and have fun!

Choose your own adventure.

This is an excerpt from an article I wrote for dscout, which gives more information on the CYOA model. As mentioned, Shopify also went into detail about how they create CYOA.

1. Choose a problem you want to bring to light. For instance, I wanted to highlight the non-linear path that comes with planning travel and all the nuances involved. This will give you a good understanding of the type of story you want to create, and which part of the journey to focus on. I always ask myself: What is the goal of the user at this point? Where does that goal start and end? I now have a story starting point and endpoint and need to include the details in between. I then choose the most common pain points, needs, and tasks to include in the story.
2. Create a user flow. Once I had an idea of the overarching story, I broke it down into a flow or journey the user experiences. Mostly, these are steps the user has to go through to achieve that goal identified above. At this point, most of these are linear, task-based steps, without any emotion. It’s okay if they are a little disjointed, and some parts don’t connect correctly with others. This stage is really for outlining the basic story.
3. Infuse emotion. Once I have a basic outline of the story, I begin to add emotions to the context. Instead of simple text, I form scenarios. For instance, “select a date to travel” becomes “Jonas isn’t sure which exact dates he will travel. On the website, he sees the option to select a set date range, which annoys him, seeing as he wants to explore several different options.” Adding these intricacies allows readers to understand pain points in a way that is not solution-oriented and is product-agnostic.
4. Add choice. Now comes the most complicated, but interesting, part: adding options. This is where it is very beneficial to get into the mindset of the user and relive the research sessions. Where do your users have to make decisions? What are the triggers that cause them to think: “Forget this website/app, I’m going somewhere else”?
5. Read your story. Go through the CYOA, ensuring it makes sense, and you have included the different possibilities in an easy-to-understand narrative. The value of a CYOA is that it can continuously evolve and grow as you learn more about your users.
6. Send it out. I first send these stories to the UX team, so they can glance over to see if everything makes sense. I then roll it out to the broader company. Often these stories are based on personas, so I also make sure to socialize those with the story to give more context.

Research session snapshots

These snapshots are a quick way for the team to easily visualize what important information has come from the research sessions. My snapshots include:

1. Which participants faced this issue (ex: P1, P4, P5, P7)
2. The insight/problem
3. The actions/recommendations
4. A quote
5. 1–2 video relevant video clips that highlight the issue
6. If it is a usability test, an annotation of the screen and what happened

I hope these help diversify the way you share your research insights and give you some creative ideas. Remember: food and drinks really help to motivate people to join events — so keep that in mind. Please let me know if and how you use these, and any adaptations you do!

---

If you are interested, check out more free content and my offerings on www.userresearchacademy.com and join our slack channel

How to ask for feedback

And why it is so important.

Source

No matter where you are in your life and your career, there will come a time where you will either ask for feedback from someone, or you will be asked to give feedback. Feedback is an amazing tool and is one of the best ways you can learn from others. Without continuous feedback, it is almost impossible to improve.

Learning how to properly ask for feedback, and how to give feedback, are two huge milestones in your career. By asking for feedback, you admit you don’t always have the correct or best answer, and you open yourself up to a world of opportunities. You show people that you aren’t afraid to be wrong, and you also show yourself that it is okay to ask questions. And, by giving feedback, you help others grow personally and professionally.

Without asking for and giving feedback, how would you be able to move forward? How would you know ways you can improve for next time? Albert Einstein was attributed (although many would argue otherwise) with saying:

The definition of insanity is doing the same thing over and over again and expecting a different result.

Without feedback, we would continue doing the same thing over and over again, hoping for a better result. Continuous feedback breaks these toxic cycles.

However, this can go horribly wrong. On both sides.

---

Let’s start with the concept of asking for feedback…

It is wonderful when people ask for feedback. I absolutely love it. However, sometimes, the questions can miss the mark. I believe the right intent is there, but it can leave the person receiving these requests either frustrated, confused, or tired.

For example, I spoke with a few friends who get asked feedback frequently for portfolios, resumes, jobs, etc. Both of the below scenarios are, of course, exaggerated to get the point across, but they are still representative. Here are three examples:

**I do understand we are in a time where people may really need jobs, and we are emotionally exhausted. So if you receive a message such as the below, be kind and try to explain that you simply need more information to give the best feedback/do the best networking possible

Example 1 (networking):

• Person: “Hey, can you help me find a job?”
• Feedback giver: “Hey, let’s see if I can help. Can you tell me more?”
• Person: “I’d like a job in user research.”
• Feedback giver: “Okay…where are you looking for a job.
• Person: “Anywhere, I can work remotely or move.”
• Feedback giver: “okay…where are you currently located?”
• Person: “Boston.”
• Feedback giver: “And what kind of role are you looking for? What level? What type of company…”
• Person: “Anything user research related.”

Example 2 (networking):

• Person: “Hey, can you give me some feedback on my portfolio piece/resume?”
• Feedback giver: “I’d be happy to, what kind of feedback are you looking for?”
• Person: “If it is good and if it will get me a job.”
• Feedback giver: “Sure, but what is your experience level”
• Person: “Mid-level.”
• Feedback giver: “Okay, and what types of jobs are you applying to? This will help me with giving you the best feedback.”
• Person: “Mid-level user research jobs.”
• Feedback giver: “Can you give me an example of a company?”
• Person: “UK Gov.”
• Feedback giver: “Okay, and what is your biggest strength and your biggest weakness in your portfolio?”
• Person: “I don’t know.”

The thing is, we are all learning how to network and how to get through these situations. Some people have been networking for years, and some people are naturally good at it. But others are still trying to navigate the digital world of networking and asking for feedback, where we lose important indicators such as body language and tone.

Example 3 (workplace, over slack):

• Researcher: “What do you think of this research report I just put together?”
• Colleague: “It’s nice…but I am not really sure how to use it…”
• Researcher: “What do you mean, you don’t know how to use it?”
• Colleague: “Maybe it is just me, but how should I use this for roadmap or planning?”
• Researcher: “Well, it isn’t as if I can give you all the answers, but it shows the number of participants who were confused on slide 7.”
• Colleague: “Sure, I guess I’m just not sure what the next steps should be. Really, maybe it is just because I’m not used to working with a researcher!”
• Researcher: “Okay, then what can I do?”
• Colleague: “Maybe you could put the next steps or recommendations you have…”
• Researcher: “Okay, sure.”

It can be difficult to hear negative things about work we’ve put time and effort into. Sometimes the feedback will come out in kinder ways (such as above), and sometimes not. We have to learn how to take this feedback and make it as constructive as possible, to learn how to do better next time.

Some guidelines on asking for feedback:

1. Think about both sides. The best advice I can give when asking for feedback is to think critically about your message. Would you like to receive it? If you received it from someone else, could you help them?
2. Mean it. Don’t ask for feedback if you aren’t going to use it or are just going to defend whatever you currently have or are doing. This isn’t helpful for anyone. It is totally fine to disagree and have discussions (that is another way we learn), but don’t just ask and ignore.
3. Give more context. As mentioned above, more context makes communication more clear for everyone. It also makes everyone’s job a lot easier. It is really difficult to authentically reach out to help someone find a job if you aren’t feeling motivated to do so based on the message you received.
4. Try not to get defensive. As I mentioned, it can be hard to not get upset when someone doesn’t understand something you have worked on. Remember, you are all working together towards the same goal within a company. The best thing you can do is understand someone’s perspective and then decide what to do with the feedback. Maybe there is a misunderstanding and you can make things more clear, or maybe something is confusing that you can change.
5. TALK to the person first. If you are connecting with someone you don’t know and asking them for feedback, create some conversation. Show them you did some research and why you want to get their feedback. Maybe they wrote an article or gave a talk you loved — let them know! Try not to just lead with asking for things.
6. Give people the benefit of the doubt

---

Better feedback conversation examples:

I just quickly wanted to give some good examples that are similar to the ones above. These are also based on real-life experience! It just takes time to learn these skills, but it is important in advancing your career.

Example 1:

• Person: “Hey! I was wondering if you could help me with finding any opportunities. I am a mid-level user researcher located in Boston, MA (but willing to relocate or work remotely). I have worked as a researcher for 3 years, mostly at B2C companies. I am looking to move to a smaller sized company (under 500 people) and in the B2C space. I would love a role in education tech or medical tech, however, I am also happy to work in the e-commerce space (where I have more experience). I’ve attached my resume if you want more details. Please let me know if there is anything else you need from my side. Thank you!”

Example 2:

• Person: “Hi! I am currently looking for a new job in user research. I created this portfolio based on my recent work at a B2C company. I am starting to apply to some junior and mid-level roles (I’ve only been in the field for about 1 year). I am really struggling with the layout and making sure my content is engaging. I have some nice deliverables, but I’m not sure if everything makes sense when I put it together. Would you mind looking through it to see if it tells a story? And also if there is any content I am missing that you think would be applicable to a hiring manger? Thank you and let me know if there is anything else you need!”

Example 3:

• Researcher: “What do you think of the research report I sent you a few days ago?”
• Colleague: “Oh, it’s nice, but I’m not really sure how to use it…”
• Researcher: “That is such great feedback, thank you for sharing. Can you tell me more about what is confusing or missing? I’d love to make it more actionable for you, and be able to improve the next reports! Maybe we could have a meeting to discuss it?”

All of these are just simply models based on some experience I and others have had. These aren’t rules, by any means, they are simply some guidelines you can use when thinking about asking for feedback. Again, I’m not saying the first examples are wrong, but they might come off as difficult to engage with to some. Remember, you are asking favors of people when you ask for feedback. You want to make sure you come off as clear and understanding.

We all can come together and think about how our words impact each other, on both sides, and become better at communication. As the world evolves, online communication will start becoming more and more inherent in our lives. It is best to think about having a conversation online as you might in person.

Stay safe and healthy

---

For more engaging discussion, please sign up for the User Research Academy slack group! And check out User Research Academy for free resources.

Seven steps to writing a screener survey

How to make sure you are always getting the best participants.

Source

Screener surveys are an excellent tool for the beginning of each research study. Before you jump into recruiting, you can send out this survey to assess potential participant’s backgrounds, behaviors, habits, and other essential characteristics.

There are a few reasons we write screener surveys:

1. Find and talk to the most relevant participants for your research study
2. They help you guarantee you are getting high-value participants
3. Ensure return on investment for your research will be as high as possible
4. Avoid wasting time/money on suboptimal participants
5. Avoid wasting the time of others (ex: users who might not be the best fit)
6. Dodge awkward research sessions where the participant cannot meaningfully answer the questions you are asking

I learned about the importance of screener surveys the hard way: through first-hand awkward experiences with the wrong participants for my study. I was researching the process that occurs when people move to a new home or apartment. I wrote my screener survey, which was full of demographic information, and asking a few questions about whether they had moved recently.

We received a good number of responses quickly, and I was happy. They were matching the demographic data we wanted from our target participants. Nothing ticked in my head, saying, “how are we getting so many participants so quickly?” Not to spoil the plot, but it had to do with the screener survey. It was too focused on demographic data and not focused enough on behavior.

So, what happened? I had participants come in, and I asked them my default starting question, “tell me about the last time you moved.” They gave me an overview, and I set to dig into the details:

• Me: “Okay, great, now walk me through the step-by-step process you went through.”
• Participant: “Oh, I wasn’t involved in the process. My partner/real estate agent took care of that. You should probably talk to them.”
• Me (internally): Sh*t.

Some of the people had lied on the screener survey. They responded saying they had moved in the past year. However, some had moved a few years ago. But they knew what I was looking for and the correct answer to participate for the incentive.

I had recruited the wrong participants because of my screener survey. Not all the participants were a bad fit, but an overwhelming majority ended up not being super helpful. I had to redo my screener survey and start recruiting again. That is when I learned, screener surveys:

• Are not all about demographics
• Should focus on behavior and habits
• Need to elicit specific information
• Can’t give away which answer is correct

After that experience, I took my screener surveys much more seriously and learned how to write them correctly.

However, they can be tricky to write

Like most things in user research, we have to consider how we construct screener surveys seriously. Certain questions can backfire on us. You have to achieve a balance within a screener survey:

1. Get the right information to qualify the best participants
2. Make sure you are not asking obvious questions that would lead people to exaggerate their responses to participate

Tips for writing effective screener surveys

After a few years, I have done my best to achieve this balance. I always use the following steps before writing and sending out a screener survey.

The seven steps to writing an excellent screener survey

1. List the criteria of your ideal participants. Before you even think about recruiting, it is beneficial to list out all of the different criteria an exemplary participant would have. And think about the information you need from them. For instance, if I had genuinely thought about needing information on the step-by-step moving process, I would have (hopefully) included questions about whether or not they took part in this. It is beneficial to consider:

• Who are your target personas or proto-personas
• What behaviors do you want to understand more?
• What habits are you trying to target?
• What are the goals the user is trying to accomplish?

2. Write one question for each criterion you identified

• For each ideal standard/behavior, write one screener question

For example, if it is essential someone has wanted to move for 60 days; write a question about this behavior

• Make sure to write precise questions that ask for particular behavior patterns or timeframes:

Imprecise criteria: People who have wanted to move for 60 days

Precise criteria: People who have visited 3+ apartments in the past 60 days

3. Focus on how potential participants feel and HAVE behaved

• Focus on their past behavior, as it is the best predictor of future behavior

Future question: Will you look at apartments in the next 60 days?

Past question: Have you looked at 3+ apartments in the past 60 days?

• Ask for only the most basic demographic information that you need, since it makes the survey longer

4. Order your questions carefully (use logic!)

• Ask questions that will quickly weed people out in the beginning
• Prioritize the essential criteria you need the participant to align with before you ask specific questions

For example: Make sure you ask IF someone has thought about moving before asking them how often they have viewed apartments

• Ask about location first if you need in-person interviews
• Use a funnel approach

For example: Start with more broad questions, move into specifics, and then broaden out again with questions like demographic data

5. Avoid leading questions

• Leading questions will influence people to answer in particular ways, and skew your data
• Leading: How great was our coffee?
• Fixed: How would you describe our coffee?

6. Include a mix of open-ended and closed questions to avoid obvious answers

• Using a combination of open and closed questions helps to make sure you are getting the best participants. A 60/40 mix is safe, where 60% of questions are closed, and 40% are open-ended questions.
• Open-ended questions help you see how a participant would answer in their own words, without priming them to respond in a specific way
• Using closed questions in a non-obvious way can help you understand behavior patterns (ex: asking how many times someone viewed apartments in 3 months, and giving a range of answers)
• Adding an open-ended question to your screener survey can give you clues as to how much insight a participant will provide in your actual study
• One word responses, illogical rambling or cagey answers can all indicate that a participant may provide a low return on investment

7. Always include an open text field

• Always make sure you include an open-text area in each multiple-choice question, as it is impossible for you to know you include all options someone might need
• Ex: “other,” “not applicable,” “I don’t know,” or “none of the above”

Let’s go through an example:

We own a plant shop in Brooklyn, New York, where we sell groups of exotic plants. We have been posting our plants on social media, and people from around the United States are contacting us and asking if we have an online shop. We want to consider this as an option but aren’t sure what people would want or expect. We want to conduct user research to understand the following:

1. How people would like to purchase plants online
2. What do people expect of an online plant store
3. What do they think of other online plant stores
4. Get their opinion on a prototype of the online plant store

Our screener survey might include questions like:

1. Have you purchased plants online? (Yes/No)

If yes:

1. How often have you bought a plant online in the past six months? (Multiple choice)
2. How many plants have you purchased online in the past six months? (Multiple choice)
3. Where did you purchase from? (Short open text)
4. How was your experience with buying plants online? (Long open text)

If no:

1. Have you considered buying a plant online? (Yes/No)
2. If yes, why have you not purchased one online yet? (Open text)
3. If no, why have you never thought about it? (Open text)

Feel free to add more thoughts on how you would screen!

---

For more engaging discussion, please sign up for the User Research Academy slack group! And check out User Research Academy for free resources

If you liked this article, you might also be interested in:

• Moving Beyond the NPS to measure experience
• Recruiting the right participants
• Writing a UX Research survey
• Elevating your survey questions

Recommended resources for User Researchers

February 2021

Source

I get asked quite often about any resources I can recommend for someone new to, or even seasoned, in the field of user research. Since I have found myself writing the same recommendations constantly, I decided to compile these into a list, which I will come back to and update!

Last updated, February 2021

Podcasts:

Since I have a dog, I spend a lot of time listening to podcasts while we go on long walks through Berlin. During this time, I love to listen to podcasts about user research. Not all of them allow me to come away with tangible action items, but they allow me to understand a different or new perspective. I love hearing about different ideas or challenges other researchers have faced. Here are my favorites:

• The Conversation Factory (my new favorite)
  A fantastic podcast, hosted by Daniel Stillman, where he explores the edges of Conversation Design: the application of Human-Centered Design principles and Experience design to human discourse.
• Dollars to Donuts
  A wonderful podcast, hosted by Steve Portigal, where he talks with people who lead user research in their organization about all things user research.
• Mixed Methods 
  A thought-provoking podcast, hosted by Aryel Cianflone, interested in the hows and whys of user experience research. Through interviews with industry experts and hands-on trial and error, they indulge and celebrate curiosity.
• Awkward Silences 
  A podcast, by User Interviews, where they interview the people who interview people. Listen as they geek out on all things UX research, qualitative data, and the craft of understanding people to build better products and businesses. Hosted by Erin May and JH Forster, VPs of growth/marketing and product at User Interviews.
• Aurelius 
  By Aurelius labs, and hosted by Zack Naylor, a podcast that talks to industry experts, discussing design and product strategy. Hear from leaders how they are solving the right problems and building products and features that matter most.

Slack channels:

There is nothing I love more than seeing a community of UX’ers helping each other. I truly believe a community is one of the best things for a field, and there are several amazing slack groups growing. I highly recommend joining these communities, as they are some of the best support out there.

• Mixed Methods 
  A great community for all things UX, both research and design. Join the growing community to post questions and help others in the field.
• ResearchOps
  A global community who’ve come together to discuss the operations and operationalization of user research and design research — also known as ResearchOps. ResearchOps includes the people, mechanisms, and strategies that set research in motion.
• User Research Academy
  A bit of a shameless plug here, since this is a slack channel I am growing. A community to be a support system, a go-to place for user researchers to find advice, and a community for an engaging discussion on important user research topics
• Ethnography Hangout
  A global community created for conversations about ​ethnographic methods. This is an interdisciplinary group wearing many hats from design to tech and research, so you don’t need to have any formal background in ethnography to participate.

Books:

There are some basic books I always recommend for people to read, which have a wealth of information about user research. Although these are more static than articles and podcasts, they serve as wonderful foundations and building blocks. When I get stuck on a problem, I still go over to my bookcase to pull out these books. These books include information on, both, qualitative and quantitative user research methodologies.

• Research Practice: Perspectives from UX Researchers in a Changing Field by Gregg Bernstein (I am a contributing author!)
• A Practical Guide to Measuring Usability by Jeff Sauro (one of my favorites)
• Just Enough Research by Erika Hall
• Think Like a UX Researcher: How to Observe Users, Influence Design, and Shape Business Strategy by David Travis
• Observing the User Experience (2nd Edition): A Practitioner’s Guide to User Research by Elizabeth Goodman, Mike Kuniavsky, and Andrea Moed
• Interviewing Users: How to Uncover Compelling Insights by Steve Portigal
• Mental Models: Aligning Design Strategy with Human Behavior by Indi Young
• UX Research: Practical Techniques for Designing Better Products by Brad Nunnally and David Farkas
• Measuring the User Experience (2nd Edition): Collecting, Analyzing and Presenting Usability Metrics by William Albert and Thomas Tullis
• Handbook of Usability Testing: How to Plan, Design and Conduct Effective Tests by Jeffrey Rubin
• Practical Empathy by Indi Young
• Quantifying the User Experience: Practical Statistics for User Research by Jeff Sauros

Blogs/Resources:

I am often scouring the internet for information on user research. After all, searching the internet is how we learn now. I love reading about different perspectives and thoughts on user research through people’s words. Instead of finding particular articles, I wanted to recommend the websites I reference the most.

• Neilsen Norman Group
  One of the top websites for all things UX and a great place to find relevant information. One of the articles I send around the most from NN group is their article on only needing to usability test with five users
• UX Collective
  My go-to when searching for UX articles on Medium. They have a great selection of UX articles written by people in the field. What I love most about these pieces is that they are written by people who are experiencing the problems in real-time, and can give tangible advice/examples on how to handle them
• dscout People Nerds
  dscout is constantly putting out relevant information into the world of UX, and they touch upon many important topics, such as accessibility and repositories. They take real-world problems researchers are facing, at a global level, and provide content to help them overcome these hurdles. PS: I write for dscout, but find them extremely relevant regardless :)
• Usability.gov
  A bit old school, but a really comprehensive website all about usability and improving user experience. They have guides, templates, and documents to help you with better understanding, and testing, usability.
• User Research Academy
  Another shameless plug. I founded User Research Academy back in 2019 to help people get into the field of user research. I offer courses and mentorship in user research.

Facebook:

I only have one recommendation for Facebook, but it is such a wonderful community, so I wanted to make sure it got proper recognition.

• User Research Collective

I hope you find these helpful! As I mentioned, I will be coming back to add and update these!

---

If you liked this article, you might also be interested in:

• Which UX Research methodology should you use? [chart included]
• 5 ways to reframe a solution to a problem statement
• How to Break into User Research
• Assessing your user research career level, seniority, and path

5 Steps for Setting Up In-depth Interviews

In-depth interviews are gold. This user research method allows us to genuinely understand who our customers are and what they experience in their everyday lives. It’s a style of interviewing that goes beyond the customer’s interaction with our product. Of course, we still want to know how a product or service impacts them. However, understanding our customers as real people, with lives outside of our product, should affect the way we build our product or service.

How do you set up these research interviews?

Here are 5 steps to get started:

1. Define your problem statement. This is the central statement the research seeks to understand. It is WHAT you will be studying.

Example: Choosing a software product is a very complex process; we seek to deeply understand how people make these decisions.

2. Define your research objectives. They should address HOW you are going to study the problem statement. Do this by breaking the research problem down into several objectives:

Ask yourself: “What am I trying to learn?” and “What must the research achieve?”

Example objectives:

• Understand the end-to-end process of how participants are currently making decisions on choosing a software product.
• Uncover the different tools participants use to make software product decisions.
• Identify any problems or barriers they encounter when trying to make decisions on software products.
• Learn about any improvements participants might make to their current decision-making process.

Identify what stage of the idea is being researched:

• A concept or idea: You need to understand the process people are currently going through (generative research)
• A prototype: You need to uncover what people think about the prototype and how they expect to use it (generative + usability testing)
• Live code: You need to evaluate the performance of the product and what people think about it (usability testing)

3. Target the correct participants for the study. Before you start recruiting, you have to understand who your users are so you can optimize recruiting efforts. Talking to the right people is a fundamental part of effective research. What are some ways to do this?

• If you haven’t already established personas or a target audience, take a day to define your target user. Do they work in marketing, sales, customer support? Probe internal stakeholders for key insights and come up with proto-personas.
• Scope your competitors, and recruit based on their audiences. Bonus: You can even recruit people who use the competitor’s product, and during the interview, ask them how they would make it better.

4. Write a research plan. Research plans contain all of the necessary information about the research project. Use it as a kick-off document to distill core goals and align the whole team.

5. Align the team. Once you have finished the research plan — or even before — sit down with the team to discuss the research problem and objectives. By doing this, the whole team can brainstorm questions they would like answered or concepts they would like to understand better through the research.

With these five steps, you should be able to start generating critical insights from discovery research!

Why you should always aim for open-ended conversations

It isn’t all about asking questions.

Source

The always in the title might be a little bit of a lie and slightly misleading. I must admit, not every conversation I have is open-ended. There are times when I am human (other times, I am a robot) and ask a yes or no question, or even a leading question. However, I do my best to have the most open-ended conversations as often as my brain can keep up with my mouth.

Open-ended conversations are the magic of user research interviews. They allow you to understand users’ mental models and dig into the lives of your customers.

Discovering how and why users think a particular way is key to delivering great insights to your teams. With just a list of basic interview questions, it is nearly impossible to uncover the mind of the person sitting next to you.

There is one particular framework I use to ensure I do this as much as possible. Unfortunately, I can’t remember exactly where I learned this from (it was years ago), but if anyone recognizes it, please tell me.

The TEDW Framework

I have used this framework since I can remember, and it has helped me become a better researcher. With this, I spend less time thinking about what to say next. Additionally, the TEDW framework means I am not always asking, “why, why, why” repeatedly, but I am still able to dig deeper into what the user is saying.

Another reason I love this framework is that it turns interviews into conversations and storytelling. What you want and need from users is to have them recall specific memories and tell you stories about those previous experiences. The TEDW framework enables you to get into this mindset. It takes the pressure off of the researcher and the interviewee.

Another cool thing is that the TEDW framework is not about asking questions, but about having open-ended conversations. Instead of asking people direct questions, you are using active listening and open-ended statements to extract the stories from them. Using this technique makes it a lot easier for participants to give you reliable past data, as it reduces biases that come through in research.

TEDW framework

Some examples

To demonstrate this further, here are some ways I would reframe questions into the TEDW format. Since Netflix is such a famous app/service people use, I will use this as an example:

• Original question: “When was a time you Netflix?”
• TEDW question: “Tell me about the last time you watched Netflix.”
• Why I rewrote this: If you start with a question like “when was the last time…” you are not prompting the user to tell us any stories. You are asking a specific question. Let’s start broad, and then you can use the “what, when, where, how, why” to dig deeper into setting the scene.
• Original question: “Were you doing other things while watching Netflix?” -> follow-up: “Why were you shopping online while you were watching Netflix?”
• TEDW question: “Describe what else you were doing while watching Netflix.”
• Why I rewrote this: The original question is direct and a technique I will sometimes fall into. The person might not know why they were shopping online while watching Netflix, but it is more important to know first what they were doing and to have the participant set the scene. Then, if you are a team that is trying to understand the way people go in and out of a show, you can dig deeper into that area. You might not get as reliable of data if you are asking these direct questions.
• Original question: “Why were you feeling frustrated with Netflix when you couldn’t log in?”
• TEDW question: “Explain what you mean by frustrated.”
• Why I rewrote this: There are two reasons to rewrite this question:

1. You have to understand what that person means by frustrated. People often have different perceptions of what emotions mean, and it is essential to understand that from a user’s perspective, not from your assumed mental model of the feeling. My frustration might be your anger or sadness.
2. Again, this is a direct question. The participant might not know why she was frustrated. If you leave it more open-ended, she will end up recalling the scene, which gives you ample opportunities to dig deeper into what is most important to the participant. And, you should care about what the participant cares about.

• Original question: “What made you turn on Netflix last time?”
• TEDW question: “Walk me through the last time you started watching Netflix.”
• Why I rewrote it: Again, you are getting at a story with the TEDW format. You are asking them to recall a memory of what was happening, instead of asking them to answer a specific question on what made them turn it on. With the context of the story, then you can drill down into subjects such as, “what were you feeling?” “When was this?” “Describe the scene for me.” You want to focus on the story the participant is telling you, not the answer to a question.

Using this framework doesn’t mean that you never drill down and use questions like, “why did this happen,” or “when were you doing this?” Instead, TEDW is a way to start conversations, and a way to begin the storytelling process. Once the participant starts giving you nuggets of the story, you can dig deeper into the critical parts based on our research goals.

Give it a try and tell me how it goes!

Want to share with others? Feel free to use this condensed slide deck.

---

Interested in more user research? I teach an Introduction to User Research Course and am available for one-on-one mentoring. Check out the User Research Academy. Please join the User Research Academy Slack Community for more updates, postings, and Q&A sessions.

Is it time we move beyond the NPS to measure user experience?

Why the NPS sucks and what to use instead.

Source

I get an overwhelming sense of grief and anger when I see the Net Promoter Score (NPS) being used as a beacon of light for companies. I have heard so many people praising the NPS, and using it as a legitimate way to measure customer satisfaction and success of a company. In fact, sometimes, it is used as the only way to measure these metrics. But, I really believe, the NPS should not be the basis of making critical decisions.

As user experience and user research gain traction in the tech/product field, there are so many additional ways to gather feedback, measure satisfaction, and pinpoint where you are disappointing your users. The NPS feels like a very archaic metric that stuck around simply because it is easy. And, to be honest, the other suggested metrics in this article are more difficult than the NPS, but there is a reason behind that.

What exactly is the NPS?

If you have ever tried out a product or service, you have probably seen this question before:

On a scale of 0–10 how likely are you to recommend <product or service> to a friend or colleague?

Your NPS is the accumulation of people who use your product or service taking the time to rate whether or not they would recommend your product or service to others. Based on the score they give your product or service, they fall into one of three buckets:

Source

With this information, you are able to calculate a score that you can measure across time. The formula to calculate the score is:

Net Promoter Score = % of Promoter respondents minus % of Detractor respondents

Remember, passives are kept out of calculations, so any scores of 7–8 essentially do not exist in terms of scoring.

As an example, let's say you received 100 responses to your survey:

• 40 responses were in the 0–6 range (Detractors)
• 30 responses were in the 7–8 range (Passives)
• 30 responses were in the 9–10 range (Promoters)

When you calculate the percentages for each group, you get 40%, 30%, and 30%.

To finish up, subtract 40% (Detractors) from 30% (Promoters), which equals -10%. Since the Net Promoter Score is always shown as a number (not percentage), your NPS is -10. NPS scores can range from -100 to +100.

In addition, the NPS appears to do things that other business metrics don’t or can’t. It is:

• A single question that is easy to understand
• Easy to calculate and measure
• Easy to track any changes over time
• Widely accepted in business

Why the NPS is a suboptimal metric

There are a few reasons why I think NPS is not the most ideal metric to use as a single metric. I am in no way saying, “never use the NPS again!” If you want to continue measuring NPS, just understand what you are really measuring and consider additional metrics to track alongside the NPS.

1. Why is recommendation a sign of satisfaction? There are so many different reasons why one might or might not recommend a product/service to a family member, friend, or colleague. Sure, we could assume the mental model of “I am satisfied with this product/service, therefore I would recommend it,” but this still leaves many situations unanswered. What if it simply doesn’t make sense to recommend the product/service to these people because they never use anything like this? Or maybe you are unsure about the value since you haven’t used it a lot. Maybe you hate it, but someone else in your life would benefit from the product/service. There are many different contributing factors to recommendations, and they don’t always lead to product success and customer satisfaction
2. Recommendations are contextual and subjective. Similar to the above statement, whether or not someone gives a recommendation can be completely subjective and based on context. Would I recommend a movie I just saw to others? Maybe. It depends on the person. If it was a comedy, I would potentially recommend it to my friends who like comedy, but not my friends who like action or thrillers. I could have also had a terrible night with bad popcorn, and lumpy theater seats, causing me to dislike the movie because of circumstance, rather than quality. Or I could have simply been having a bad day, and that impacted the experience I had with a product/service, causing me to give a bad score. There is a multitude of reasons why recommendations are extremely contextual and subjective
3. NPS calculation overshadows success. The weird calculation of the NPS score does a great job of hiding improvement and success. The above example gave us an NPS of -10. Now, let’s say we work super hard to make some great changes to the product/service. We know we are on the right track because we are using user research insights to help give direction to these changes (biased, I know). So, it turns out we have increased those scores, and we receive many more 6’s and 7’s. For some reason, the NPS counts the 6’s as 0’s (detractors) and the 7’s don’t count (as they are passives). So, ultimately, our NPS is still -10, despite the improvements we have made. This simply does not make sense
4. You do not know what type of user is responding. The NPS does not care or categorize the respondents to the survey. So, we don’t know if the people who are responding are power users, new users, or even our target personas for the product. If we have spent time building for certain types of personas but are getting scores from other people out of our scope, that can completely skew the results. It is also easy to skew the results in the other direction — you could be getting good reviews from people who you are not at all trying to target
5. Cultural differences could impact your score. If you are a global company and are collecting responses from many different countries, it is extremely important to study the NPS data per country. There are vast differences in how cultures respond to these types of surveys, which can really mess up your data if you are lumping them together. It is best to keep an eye on the country or region your responses are coming from and to separate that out
6. An 11-point scale is really large. The NPS gives a very large range of responses as an 11-point scale. In fact, it is one of the largest scales. The distinction between the numbers is not at all clear, and most likely does not make sense from person-to-person. You and I could have the same exact experience but I would give the product a score of a 7, and you give the score of a 6. What is the difference between 6 and 7? It is very hard for respondents to understand this difference, and to choose a meaningful response. It is almost the same as closing your eyes and picking between two numbers
7. There is no data on ‘why’ someone gave a particular rating. Aside from all of these, my biggest gripe with NPS is that there is no understanding of why someone gave a particular score. Receiving a rating can be rather useless if you don’t understand the motivation behind the score. Without the ‘why,’ there is no way for a company to determine how to improve a product or what direction to move in. Perhaps it was an error beyond your control that led to a lower score. Without understanding the ‘why,’ you can spend much time and energy trying to guess what went wrong and how to fix it
8. It is completely future-based. I do my best to avoid asking future-based questions. We want to focus on what people have done in the past, as most of us cannot predict our future behavior (think of all the gym memberships that go unused).

What to do instead (or in addition to)

Now that I have bashed the NPS as a single-source metric, I would love to give some alternatives to the NPS or to measure alongside the NPS. We all know people love numbers and measurements and there are some other questions we could be asking (hint: they might not be as easy, but could be more effective). You can even continue to include the positive/negative aspect of the NPS.

Frustration:

• How delighted or frustrated were you today? — Extremely delighted (+2), delighted (+1), meh (0), frustrated (-1), extremely frustrated (-2)
• In the past X weeks, have you felt frustrated with our product/service? — Yes (-1), Unsure (0), No (+1)

Helpfulness/happiness:

• Have we been helpful today? — Yes (+1), Unsure (0), No (-1)
• Have we made you happy today? — Yes(+1), Unsure (0), No (-1)

Loyalty:

• In the last X weeks, did you recommend us to a friend or family member? — Yes(+1), No(-1)
• Have you ever recommended us to a friend or family member? — Yes(+1), No(-1)
• Were you recommended to us by a friend or family member? — Yes (+1), No (-1)
• In the last X weeks, have you considered [canceling your subscription, switching to another provider, etc.]? — Yes (-1), No (+1)

Satisfaction:

• How satisfied are you with [company X]? — Very dissatisfied (-2), Dissatisfied (-1), Neither (0), Satisfied (+1), Very satisfied (+2)
• How easy was it to complete your order online? — Very easy (+2), Easy, (+1), Neither (0), Difficult (-1), Very difficult (-2)

Finally, always leave an open-ended text field asking something along the lines of “how could we improve?” This adds to the qualitative input you can receive to give you a better indication of what went wrong and the context of the situation. This will help in providing more clear direction towards product/service improvements.

Ultimately, user experience and satisfaction cannot be boiled down to one singular number. Let’s keep an open mind about what we measure, how we measure, and why we are using specific metrics, so we are best able to make better, well-rounded data-driven decisions. And, don’t forget to add in some qualitative research as well!

---

If you liked this article, you may also find these interesting:

• Asking about the future in user research
• A case for beta testing
• First-person surveys in user research
• Benchmarking in user research

Interested in more user research? I teach an Introduction to User Research Course and am available for one-on-one mentoring. Check out the User Research Academy.

Please join the User Research Academy Slack Community for more updates, postings, and Q&A sessions

Assessing your user research career level, seniority, and path

Source

Career paths are extremely important topics to think about, especially in the context of a more niche field, such as user research. While there is some information online, it is primarily for the broader field of UX and much more focused on the UX design career path. Planning the next 1, 3, or 5 years of your career is never easy. But trying to do so without a lot of guidance or mentorship can be even more difficult.

The reason I know this is because I have recently been trying to figure out what my next step is, and have also failed to plan effectively in the past. This lack of planning has led to confusing and less than ideal roles and situations. Knowing what your current level is, and where you want to go with your user research career, really helps you make sure you are in the best role for growth.

User research career levels

There are a few different levels of user research, and (fortunately) they generally follow the same kind of trajectory as other careers. I have also put a general amount of years of experience but this, of course, can vary. I have seen people in certain roles with much less, or more, experience than is generally “recommended.” I believe an employee’s skillsets and level of maturity are far more important in determining a career level than the number of years someone has been in the field.

I have also measured these against the following areas of impact a researcher could have on a company level:

• Operational: Deals with the day-to-day function of user research
• Organizational: Handles how the company understands and ingests user research
• Strategic: Aids the company in making strategic decisions based on user research

User researcher levels

• Research coordinator: Supports the product team throughout the research life cycle, including scheduling, recruiting efforts, participant communication, streamlining research operations and team communications
• Junior user researcher: Embedded in a team to carry out user research activities. They have some practical experience but need regular guidance and training to produce their best work and develop their skills. They generally work in combination with a more senior user researcher
• Mid-level user researcher: Embedded in a team and responsible for planning and carrying out user research activities. They are able to work independently on a team, without too much guidance
• Senior user researcher: Able to plan and lead user research activities in larger teams and on more complex services. They build user-centered practices in new teams and align user research activities with wider plans to inform service proposition. They may supervise and develop other user researchers to assure and improve research practice
• Lead user researcher: Leading and aligning user research activities across several teams. They ensure that teams take a user-centered, evidence-based approach to service design and delivery. They develop and assure good user research practices
• Head of user research: Leads user researchers in an organization and attracts and builds talent. They are an expert practitioner who can define and assure best practice, influence organizational strategy, and priorities, and collaborate with colleagues across a company.

Download this sheet here

Different career paths

There are two main career paths for user researchers, and they are similar to those of other industries. In general, you can go one of two paths:

• The individual contributor
• The manager

The biggest question I would ask when determining one of these two paths is to understand: “do I want to help others develop the skills I have?” or “do I want to continue to hone my skills as a research practitioner?”

As a manager, of course, you may still be able to engage in the more tactical side of the job (ex: actually conducting research), but usually, you operate as a people manager, mentor, and strategic partner. As an individual contributor, you will most likely be able to contribute at an operational, organizational, and strategic level. The biggest difference here is whether or not you are mentoring and managing others.

If you feel okay stepping away from the day-to-day and are looking to mentor others in the field, management might be your best bet. If you want to become a super expert in your field, I would stick with being an individual contributor. Of course, the best way is to simply try. As of right now, I have been a senior-level individual contributor and have finally decided to make the leap to management. Let’s see :)

How to assess your level

Now, these descriptions and charts are all generic. They have to be. There is no one description that will be perfect for all. Many user researchers have unique paths and experiences, so it is hard to make generalizations. However, for the purpose of this article, I do my best to help you categorize and see where you can grow.

Here are the steps I take when assessing my level:

1. Audit my own skills. I always start by listing out all of my skills and my level of confidence in these skills (“low, medium, high” is totally fine). I also list the skills I would really like to learn next, that I find important for moving to the next level
2. Look at the skills in the different levels. I then look through the skills in the level I think I am, and then the levels directly below/above. I also stalk people on LinkedIn at the level I think I am, and the surrounding levels. I see what their experience and skill sets are, and then compare that to mine. I also do a lot of networking and talking to researchers at all levels to get a more concrete idea. Actually talking to other researchers is the best way to do this!
3. Consider my past experience. As I mentioned before, skills are one thing and experience is another. I always consider my past experiences, roles, and responsibilities. For example, when I was starting out, I was a UX Research intern but was expected to operate as a more junior/mid-level. With this, I was able to gather different skills and experiences that pushed me almost straight from Intern to mid-level. It is important to take this into consideration.
4. Think about my level of maturity. This is especially important when you start to get into the senior/lead positions in the field. It is extremely important that you strongly consider how mature you are as an individual. More and more people will be depending on you on a consistent basis, and you will have much more pressure and feedback coming your way. It is okay if you have not yet had the training to best handle these situations, but they are very critical to consider when you move into these roles. You will also have a level of responsibility for others if you choose the management track

I recommend doing this at least twice a year, as it will give you a better understanding of where you are currently and where you can go in the future. Not only is this great practice to do for yourself but, if you are looking to go towards the managerial track, it will be great practice for your future reports!

What is another aspect of your career that is great to assess regularly? Defining your user research philosophy!

---

If you liked this article, you may also find these interesting:

• How to Assess Your User Research Interviews
• How to Break Into User Research
• A Week in the Life of a User Researcher
• How to Write a Generative Research Guide

If you are interested, please join the User Research Academy Slack Community for more updates, postings, and Q&A sessions :)

A week in the life of a UX Researcher

The Atypical Week of a Researcher.

Source

I am constantly getting asked about what a typical day is like as a user researcher. And, to be honest, my answer is always “it depends.” I wanted to answer this question the best I could, so I decided to outline a week of my life as a user researcher. To be clear, not every week is like this, and they can vary from this.

Monday

8:30 — Leave for work and practice some German on the train

9:00 — Arrive at work and grab a tea

9:15 — Check and respond to any emails, such as recruiting participants confirming, and slack messages from team members.

9:30 — Squad meeting number one. We meet to discuss what is being worked on and anything that is coming up. The squad lets me know what research they will need in the upcoming weeks.

10:30 — Squad meeting number two. We meet to discuss what is being worked on and anything that is coming up. The squad lets me know what research they will need in the upcoming weeks.

11:30 — I create research plans for any upcoming research needed, and share it with the relevant squad so they are able to comment on the document, and add anything else they are interested in learning about.

12:30 — Lunch

1:00 — Squad meeting number three. We meet to discuss what is being worked on and anything that is coming up. The squad lets me know what research they will need in the upcoming weeks.

2:00 — Based on the needs from the squads, I will pull a customer list of our email subscribers and reach out to them, one-by-one, to see if they are interested in participating in a user research session.

3:00 — Generative research session with a customer

4:00 — Research synthesis for the generative research session, in which I review the session, record detailed notes and create a research summary to send to the relevant squads

6:00 — Check my email to respond to and schedule any new participant sessions.

6:30 — Leave work to go home.

Tuesday

8:30 — Leave for work and practice some German on the train

9:00 — Arrive at work and grab a tea

9:15 — Check and respond to any emails, such as recruiting participants confirming, and slack messages from team members.

9:30 — Answer any questions from the squad about the previous research sessions

10:30 — Half-day research synthesis session with a squad: we have finished speaking to seven participants for a usability test we have been working on for the past two weeks. In order to make sure we are all on the same page with next steps, we sit in a room together to discuss the different results, and decide on an outcome. I facilitate the workshop, bringing the team through the research insights, and helping them sort through the results with an affinity diagraming exercise, and HMW statements

4:00 — Generative research session

5:00 — Research synthesis for the generative research session, in which I review the session, record detailed notes and create a research summary to send to the relevant squads

6:30 — Quickly check email and leave work to go home

Wednesday

8:30 — Leave for work and practice some German on the train

9:00 — Arrive at work and grab a tea

9:15 — Check and respond to any emails, such as recruiting participants confirming, and slack messages from team members.

9:30 — Answer any questions from the squad about the previous research sessions

10:30 — Present the research findings and next steps to upper-management

11:30 — Chase the squads who haven’t responded to research needs to make sure they tell me what research is coming up, so I have enough time to recruit (it takes me about one week to manually recruit enough participants)

12:30 — Lunch

1:00 — Recruit more participants to make sure we have a backlog for any “surprise” projects

2:30 — Usability test with external participant

3:30 — Synthesize the usability test results

4:30 — Prioritization brainstorm: sit with one of my squad teams to prioritize upcoming ideas for the backlog

6:45 — Quickly check email and leave work to go home.

Thursday

8:30 — Leave for work and practice some German on the train

9:00 — Arrive at work and grab a tea

9:15 — Check and respond to any emails, such as recruiting participants confirming, and slack messages from team members.

9:30 — Office hours: I am free for any squad who needs questions answered, or wants me to help with research planning

11:30 — Team meeting

12:30 — Lunch & Learn: What is User Research — I help educate the company and any new hires on what user research is, and how they can get involved

1:30 — Internal usability tests on a concept we will test

3:00 — Budget proposal: I propose next year’s annual budget for research to upper-management

3:30 — Generative research session

4:30 — Research synthesis for the generative research session, in which I review the session, record detailed notes and create a research summary to send to the relevant squads

6:45 — Send a few more recruiting emails and leave work to go home.

Friday

8:30 — Leave for work and practice some German on the train

9:00 — Arrive at work and grab a tea

9:15 — Check and respond to any emails, such as recruiting participants confirming, and slack messages from team members.

10:30 — Full-day workshop on generative research sessions. During this workshop, all the squads get together to discuss the previously done 10 generative research sessions. I facilitate the workshop in order to help people more deeply understand our users, and also to help each squad prioritize their work. We make sense of the research through affinity diagraming, how might we statements and empathy maps. I follow-up with the designers in order to start building/iterating on personas and customer journey maps.

6:30 — Leave work to go home and enjoy the weekend.

All of this is based on a random sampling of a week to try to encapsulate what a user researcher does on a weekly basis. It doesn’t always look like this, and sometimes I have 2–3 generative research sessions a day, or even 5 internal usability testing sessions!

My step-by-step user recruitment process

And don’t worry, I don’t have a high budget.

Source

Honestly, recruiting sucks. It is one of the most challenging, time-consuming, and tedious parts of my job. Every time a research project comes across my desk, I sigh internally thinking about the recruitment process, especially if there is a tight deadline.

Since the responsibility of recruiting will never entirely disappear, I decided it was time to implement a process. I wanted to make recruitment as easy as possible through a standardized method and with as many templates as I could. My goal was to make recruitment mindless, and something I could do on the fly, in between meetings.

Of course, not every part of recruitment can be streamlined, especially the first parts of brainstorming about which participants are best. I tend to take more time in the beginning stages to ensure the right participants are selected to get the most optimized research results. Below is my thought process before I even begin writing an email to potential participants:

Before recruiting

Approximate your user
Before we even start recruiting, we have to understand who our users are so we can optimize our recruiting efforts. Talking to the right people is a fundamental part of doing proper research. If you don’t end up talking to the “right” people, your research may result in a complete waste of time and effort. What are some ways to do this?

• If you don’t already have personas or a target audience, take a day to sit in a room and define your target user. Bring in internal stakeholders that may have a good idea of who the target user will look like (such as marketing, sales, customer support), and come up with proto-personas. These are, essentially, wireframes of personas that consist of hypotheses about who your user is, and are a great starting point for who you should be talking to
• Look at competitors similar to you, and recruit based on their audiences. You can even recruit people who use the competitor’s product, and during the interview, ask them how they would make it better — a bit of a bonus!
• Sit down with the team who requested the research and ask them what type of participants and information they need. Is there a particular behavioral pattern? Is it necessary they have used your product (or a competitor’s product)? Do they need to be a certain age or hold a specific professional title? Gathering this information will ensure you write a great screener, which will collect you the best participants

My Step-By-Step Recruitment Process

Once I have my screener survey written, I am off to the races. Well, the slow races. I will take you through a sample project I recently, to illustrate how I am currently recruiting participants. And, as I mentioned, the budget is small.

Sample Project: We want to test a potential product concept idea with 7–10 users of our platform. Ideally, they will have traveled three or more times in the past three months, and have bought tickets through our website or app. Ideally, this research will be completed and reported on in three weeks.

1. Approximate the user. Since this particular project came with predefined criteria, it was easy for me to set the underlying demographics. We needed a mix of our current users for a product idea. Recently, we had decided to focus on a slightly younger demographic, so I included that strategy in my decision as well.
2. Create a screener survey. Once I had the basic idea of who I wanted, I went ahead to start forming questions to get me the right participants for the project. I knew it had to be people who traveled three or more times in the past three months, and who have used our website/app to do so. I asked for demographic information, such as age, gender, and income, to understand how different users interact with our website/app. I then put this on Google Forms to easily send a link for people to fill out, which is free! See my example screener here.
3. Pull newsletter subscriber emails from the CRM. Since I now work in the EU, GDPR rules are more strict than in the US. I can’t merely spam people who use our website/app. To get around this, I work with the email marketing team to pull newsletter subscriber emails from our CRM. When users sign up for our newsletter, they give us the right to contact them for user research sessions.
4. Individually email users screener survey. With these emails in hand, or rather, anonymized on my computer, I set aside a large chunk of time to email each user. Since we can pull up to one thousand emails at once, I usually leave this task for the later afternoon, when my brainpower is already going downhill. In this email, I include a link to the screener survey, why we are contacting them, and, of course, compensation.
5. Post a recruitment survey via HotJar. We were fortunate enough to get a business subscription to HotJar finally. They have a portion of their product dedicated to recruitment surveys, which pop up on the website. I include the screener survey there.
6. Respond to any email inquiries. My Google form responses come in nicely to Google sheets, where I can see all of the information beautifully laid out. For everyone who “passes” our criteria needed for the study, I reach out to them with the session information, a consent form, and reiterating compensation. For those who do not “pass” the criteria for the particular study, I try to fit them into another research project. Also, keep an eye out for any questions potential participants might ask.
7. Email calendar invitation. I use Google Calendar to send requests. I always double-check the timing and keep a consistent format for the subject of the calendar invite: Participant name x My name: company name feedback session. Within the calendar invitation, I reiterate the vital session information (how long the session will be, my email, etc.) and include a link to a Zoom meeting. I make a separate calendar invitation in which I book a conference room and invite all my colleagues.
8. Email a session confirmation 12–24 hours before. The probability a participant will attend a research session goes up if you email them a reminder 12–24 hours before the meeting. In this email, I reiterate the necessary information, including a link to the Zoom conference room, and the consent form if they have yet to fill it out.
9. Always send a thank you email. Regardless of compensation, or anything really, I always send a thank you email to the participant no more than 24 hours after the interview. In this email, I include specific topics we discussed, as well as the compensation voucher. Additionally, I encourage them to reach out to me with any questions or feedback in the future.

See all of my recruiting, scheduling and thank you email templates here.

Although it is still time-consuming, I have managed to decrease the amount of time I spend recruiting. In my case, the email templates have been especially helpful in speeding up the process. In the time since I started, I have managed to convince the team to invest in some great time-saving (and free/cheap) recruitment tools:

• Calendly
• Doodle
• Google Forms
• Typeform
• Zoom video conference
• HotJar

As tricky as recruiting can be, it isn’t impossible with a tight budget, or as a team of one. Just hunker down, create some streamlined templates, and enlist your patience.

How to write a generative interview guide

A Valuable Cheat Sheet for Moderating Interviews

Source

If you’ll be talking to your users in real-time, an interview guide is a valuable cheat sheet. It reminds you of which questions will help you meet your objectives, and can keep your discussions on track. I usually include these interview guides in a more extensive research plan. Check out my full research plan template, as well as other free content, here!

Even if you don’t actively refer to your interview guide, writing one ensures everyone else on the team has a place to input their questions. And if you’re outlining questions or prompts for unmoderated research, making those questions public for reference gives your team a chance to alert you if something is unclear.

For moderated, generative research, my interview guides consist of the following sections:

1. Introduction
2. Questions
3. Wrap-up

Introduction

The introduction details what you will say to the participant before the session begins, and serves as a nice preview of all the different points you’ll be discussing. It’s especially helpful if you are nervous about going into a session.

Example introduction:

Hi there, I’m Nikki, a user researcher at a takeaway delivery company. Thank you so much for talking with me today; I am excited to have a conversation with you! During this session, we are looking to understand better what makes you order food from our service. Imagine we are filming a small documentary on you, and are trying to understand all your thoughts. There are no right or wrong answers, so please talk freely, and I promise we will find it fascinating. This session should take about 60 minutes. If you feel uncomfortable at any time or need to stop/take a break, just let me know. Everything you say here today will be completely confidential. Would it be okay if we recorded today’s session for internal notetaking purposes? Do you have any questions for me? Let’s get started!

Questions

This portion of the interview guide is the trickiest to write. In this section, we’re writing down some of the open-ended questions we want to ask users during the session. For most types of qual research, you won’t always have a long list of detailed questions, since it’s more of a conversation than an interview. But readying a few open-ended questions you can then follow up on can serve as useful prep.

---

Pro tip: Questions to avoid in your interviews and interview guides

• Priming users — which will force the user to answer in a particular way
• Leading questions — which may prohibit the user from exploring a different avenue
• Asking about future behavior — instead of focusing on the past/present
• Double-barreled questions — asking two questions in one sentence
• Yes/no questions — which will end the conversation. Instead, we focus on open-ended questions

Examples of priming/leading questions:

• Priming: “How much do you like being able to order takeaway online?”
• Leading: “Could you show me how you would reorder the same order by clicking on the button?”

---

I always outline my interview guide questions with the TEDW approach. TEDW stands for the following structures:

• “Tell me…”
• “Explain….”
• “Describe….”
• “Walk me through….”

In addition to TEDW, I have recently started framing questions using the Taxonomy of Cognitive Domain, which explains how certain verbs target particular thought processes. Using this Taxonomy is a great way to expand your questioning sin order to help trigger specific responses from participants.

Taxonomy of Cognitive Domain

Beyond that, one neat trick for question generation is to use your research objectives. Your questions should be able to give you insights that answer your goals. So when you ask a participant Ta question, it is ultimately answering one of the objectives. Turn each objective into 3–5 items.

Objectives Informing Research Questions

So, let’s take a central research problem and objectives and form some research questions.

Central research problem: to understand the reasons why individual customers are reordering at a higher frequency, as well as the barriers encountered by customers preventing them from reordering on the platform.

Objectives:

• Discover users’ motivations behind reordering, both inside and outside of the website/app
• Uncover other websites/apps customers are using to order takeaway
• Learn about any pain points users are encountering during their process, and what improvements they might make

Research questions:

Objective 1: Discover users’ motivations behind reordering, both inside and outside of the website/app

• Think about the last time you ordered takeaway on our website/app. Walk me through the entire process, starting with what sparked the idea.
• Explain how you decided to reorder food on our particular website/app.
• Who were you talking with?
• What time of day was it?
• How were you feeling?
• Did you have other websites/apps open?
• Describe why you decided to reorder takeaway rather than cooking your dinner and/or going out to eat.

Objective 2: Learn about any pain points users are encountering during their process, and what improvements they might make.

• Describe the last time you struggled with reordering food, what was that like?
• How did you solve the problem?
• What would be the ideal scenario for reordering takeaway from the website/app (crazy ideas included!)?
• How would you change or improve the process of reordering food outside of our website/app? Inside our website/app?

Objective 3: Uncover other websites/apps customers are using to order takeaway.

• Talk me through the other websites/apps you have used multiple to order takeaway (or even groceries).
• Describe your experience with these other websites/apps.
• What are the other websites/apps you use to help you decide whether or not to order takeaway?

Each of these research questions is a jumping-off point for a more open conversation. They get at the core of your objectives, which in turn gets to the heart of the central problem you’re trying to solve.

Wrap-up

The wrap-up is a reminder of all the items to mention during the end of an interview. Generally, you cover information such as compensation, asking if they would be interested in future research, and assuring them that you’re thankful for their time.

Example wrap up:

Those are all the questions I have for you today. I appreciate you taking the time. Your feedback was extremely helpful, and I am excited to share it with the team to see how we can improve. Since your feedback was so useful, would you be willing to participate in another research session in the future? If you have any problems with the compensation or have feedback, please feel free to email me at any time. Do you have any other questions for me? Again, thank you so much for your time, and I hope you enjoy the rest of your day!

How to assess your research interviews

A framework to continuously improve research sessions

Source

As researchers, we are hyper-focused on the actual research interviews we conduct. We spend a substantial amount of time prepping for each meeting to give it our all when we face each participant. Right after, we rush to gather our insights, to remember all the juicy details, and to inform our teams to the best actionable steps. We strive to make a positive impact.

However, we spend so much time prepping and synthesizing and tend to evaluate how each research interview went significantly less. I still hate listening to myself, and it sometimes makes me cringe when I hear my voice. The art of reviewing our interviews and judging how well we did as a moderator, during the interview, is simply something that doesn’t happen often enough. I’ve found reviewing interviews and writing feedback has helped me to evolve my researching skills immensely.

How to evaluate your research sessions

After each interview I conduct, I listen again to the full session and make notes on how I can improve in future interviews. I’ll especially look for times when I make a glaringly obvious mistake, such as asking a leading question, assuming what a participant meant or asking a complicated, double-barrelled question.

In addition to keeping these basic best practices in mind, I have also been using an adaptation of Steiner Kvale’s criteria of a good interviewer to give myself a more accurate assessment. For each of these topics, I give myself a rating of 1–5: one meaning I did not fulfill this principle at all during the interview, and five meaning I went above and beyond to satisfy that principle. I then include some notes in each one of what I did well, and what I could do better the next time.

Ten Principles of a good interview

1. Familiarity with the topic of the conversation. You have immersed yourself in the problem space and have researched the domain you are about to enter, including industry trends and jargon, as well as being aware of potential competitors. If applicable, and you are conducting usability tests, you have a functional knowledge of the prototype or product you will be testing. You are familiar with the concepts, interactions, patterns of a product and know what to expect when a user clicks on specific elements. For example, I didn’t do enough due diligence, as I was juggling a few usability tests at once. I ended up just as confused and surprised as the participant when the prototype wasn’t working in the way we expected.
2. The interview was structured. You start the conversation by explaining what the participant can expect and layout the purpose of the discussion. I tell my students to write down introductions and try to memorize them before starting a research project. The beginning of a research session can make or break the entire interview. If the participant feels you are robotic and reading from a script, they may have a hard time opening up to you. On the flip side, if you don’t adequately explain what the research session is about, you leave the participant in the dark, which can feel very unnerving.
3. Everything was clear. The questions you asked were simple, easy, and short, which is especially crucial in usability testing. But first, let me go into question structure. We want our questions to be as open-ended as possible. I use the TEDW method, “tell me about…/explain…describe…walk me through.” These open-ended questions lead to stories and conversations, which can give us much needed context and right insights, versus asking them a continuous stream of yes/no questions.
   With usability testing, tasks need to be clear and directive. I create small scenarios behind each task. Here is a great example to illustrate this concept: Ikea wanted to understand how people found products on their website. They were usability testing by asking users to “find a bookcase.” Virtually every user went to the search box and typed in the word “bookcase.” They weren’t getting any valuable information, so switched the task to a scenario-based request: “You have over 100 books strewn around your apartment, in boxes, and cluttering various shelves. Find a way to organize them.” Participants responded in a very different way: clicking through various categories, searching for terms like ‘storage solutions’ or ‘shelves.’ Very rarely did someone go to the search bar and type ‘bookcase.’ This scenario is much more natural and relatable than “find a bookcase” and results in innate action.
4. The tone was gentle. You allow participants to finish what they say. My number one pet peeve is when people interrupt participants during research sessions. There is nothing you can do to make someone stop talking more than continually interrupting them. I often wait three seconds after the interviewee has stopped talking, ensuring their thought is finished. I find these three seconds especially important on remote video conferencing, as there is often a delay. If you must interrupt the participant, you can do so gently, with apologies. The only time I will interrupt is if we got utterly off-topic, and I am concerned about time. In this scenario, I will say: “I’m so sorry for interrupting, and I am super interested in hearing more about this topic. However, in the interest of time, could we switch back to this other topic, and I will try to leave time, in the end, to come back to what we are talking about.”
5. There was a sense of empathy. You listened to the interviewee with all the attention you have. For those 30, 60 or 90 minutes, whatever the participant says and does is the most crucial thing in the world. We must mirror this feeling in our responses. When a participant is going off-topic but is passionate about the idea, you can allow them to vent, while acknowledging their feelings. Even if what they are saying seems mundane, it is still imperative for us to care about the person in front of us. In a way, we assume the role of a therapist, by being understanding and warm. When we build empathy for participants, we can walk in their shoes, and understand what is significant for them. With this, we can give context to the insights we gain, and appreciate the person we are speaking. On this level, we can create truly impactful solutions to their problems.
6. Responses felt aligned with the flow of the interview. You responded to what is important to the interviewee. While we want to stay true to our research objectives, it is also essential to respond to the participant’s thoughts. I find that this is especially relevant for generative research, and is why I tell students to write a very sparse interview guide. Participants will always bring up what is most important to them when you ask a proper, open-ended question. Your job is to follow them on that path and allow them to be the leaders in the conversation. We help create boundaries with a strong introduction. The best thing you can do is respond to what the participant just said, by asking them why or asking them to explain further. I feel user research is a bit like improv — you are thinking on your feet and base your next question off of what that person just said. We aim to dig further into what a person has just said, especially with words you don’t understand or are highly subjective. Which is pretty much any descriptive word (easy, flexible, difficult) and any feelings a user expresses (frustration, sadness, etc.). If participants mention a potential solution, understand the why behind it and what problem it would solve for them. A great example is from one of my students — the participant kept saying that she used reviews or recommendations from other parents to help find after-school activities. This opinion was the perfect point to drill deeper: how did this impact her? What kind of information was she seeking? How did this help her? An example of the last time this happened. By staying on the same path the participant is walking down, we can understand the most critical information, which can result in more meaningful insights.
7. References to the participant’s language. You acknowledge and refer back to what has previously been said, using the same style the participant is using. At times, moderators will ask the same question over and over again to get an answer they want, which ends up giving false data. If you find yourself asking questions multiple times, reassess the flow of the interview. Frequently, you will ask the first question, and the participant will respond with a long answer, with multiple paths to go down. I write down keywords, in the user’s language, that help spark my memory later in the conversation. After the user has finished their thought, I will go back to the first keyword and go through one-by-one in asking the user to explain more about what he/she meant.
8. Correct interpretation. You were able to summarize the conversation, using the participant’s language, and did not assume, or put words into the participant’s mouth. For example, a participant said he would “appreciate more on-the-go information on his phone in terms of news.” The moderator responded, saying, “you would want more content tailored to your iPhone.” That response was a paraphrase of the participant’s thoughts. Another example is when a participant explains a feeling such as, “it made me feel upset, and I was yelling at the screen…” and the moderator says, “it sounds like you were angry.” The participant did not state that they were angry, and now we have put words in their mouths. This experience can be uncomfortable for participants and makes them feel unheard. Ensure you are using the participant’s exact words when reiterating what they just mentioned.
9. A sense of curiosity. You show a natural inclination to understand people and dig deeper. No matter what the subject is, or what the person is talking about, you have the motivation to understand them. You crave the ins-and-outs of why a participant is saying or doing something, and you dig deeper past the surface-level of explanations.
10. Silence was utilized correctly. You have used silence to your advantage. This last point can be related to being gentle and sensitive during the interview, but I think it deserves a separate point. Being silent, and allowing the person to talk is an extremely beneficial skill. Deep listening is one of the best ways to get people to talk to you.

While I am giving myself a score on the above principles, I also keep other questions in mind to ensure the overarching flow and structure of the interview went well. So, I am not only evaluating myself in core areas, such as empathy and silence but also how the conversation went. I keep the following questions in mind and write down short answers to each.

Additional questions for evaluating how an interview went

• To what extent are the participant’s answers spontaneous, vibrant, specific, and relevant?
• Are the interviewer’s questions shorter and the participant’s answers longer?
• Does the interviewer follows up and clarify the meanings of the participant’s answers, particularly emotions?
• Did the interviewer interpret the meaning of the participant’s answers throughout the interview without asking?
• Did the interviewer attempt to verify his/her assumptions of the participant’s answers?
• Was the interview self-communicating through natural, story-like answers?

When we take the time to evaluate our interviewing techniques, we can level-up our user research skills beyond what we can imagine. It is a great practice to get in the habit of, and can be a humbling experience!

Benefits of Internal User Research

Why we should validate ideas and concepts with colleagues

Source

User testing with employees gets a really bad reputation. For years, I refused to test anything on internally, with the fear of skewing data, integrating biases and considering inauthentic behavior as a genuine reaction. There are many arguments against user testing with employees:

• Employees are (generally) loyal to a brand, which may make them feel strongly positive (or negative) during the user test, regardless of what is actually being put in front of them. They are less focused on the prototype, or potential feature, and are more likely assessing their feelings of the company, as a whole
• Internal participants may know, or be friends with, those involved in the study. This can heavily influence behavior during a test. You don’t want to tell someone you are friends with that his designs suck. Instead, you may participate in a white lie for fear of insulting or disappointing colleagues
• People working for the company have their own internal motivations and opinions on the best course of action for the business. If you are testing a feature that is aligned with those motivations, there may be more positive feedback, and the opposite can be true as well
• Tech knows tech, so sometimes, developers might give very technical feedback, or may say something is not technically feasible. It may be difficult for employees to step out of the company box they are stuck in and give innovative feedback
• Employees may have prior knowledge about the study, and definitely have prior knowledge about the company. They know the company’s jargon, and the way the company thinks about ideas, which could make them preform or understand better than an “outsider.”

I am absolutely not arguing with the above, as they are all really great points against user testing with employees. I know, for a fact, it is significantly harder to observe authentic behavior when testing internally, which is the entire goal of user research.

However, I would like to provide a counterpoint. Just like everything else, internal user testing does have a time and place, and can offer some benefits.

Benefits of Internal User Testing

Of course, internal user testing cannot be viewed in the same way as user testing, and the “findings” need to be taken with a large grain (oxymoronic, but true) of salt. When used in the right circumstances, internal testing can be extremely helpful, if you know what to expect and how to interpret what you hear.

Here are some (unexpected) benefits I have come across when I finally gave in to internal user testing:

1. Internal user research is free!
This is probably the most common pro of conducting internal user research. Conducting research takes a lot of time and money, from recruiting, planning the research, talking to users and compensation, there is a solid amount of budget you need to consider. There are definitely ways to lower a user research budget, but very rarely are interview sessions free. When dealing with internal research (under 99% of circumstances), you are able to get this free feedback. You aren’t going to get all the insights you may normally get with users, but you are getting feedback, nonetheless.

2. People are easier to recruit
Recruiting can take a lot of time. Obviously, employees are much more accessible than users, and are more willing to give their time to the company (especially when there are cookies and candy involved as “compensation”). With internal tests, I have posted a message to our general Slack channel and have scheduled 7 tests over the courses of the following two days.

3. It promotes UX research to the company
Inviting employees to try out prototypes or concepts increases the visibility of user research in the company. Although I have given many presentations on what user research actually is, people are much more interested when they are actually taking part in the research. It opens up the opportunity for employees to understand what we do as researchers, and why it is important for us to talk to our users. In the past, this has led to more UX support, in general, and a larger budget.

4. Employees are able to give feedback and help the product team
I have seen product teams completely siloed in companies, and that makes me really sad. To me, everyone who works in the company should have the ability to influence the product in a positive way, and internal user testing is a great outlet for that. When I first started trying internal user testing, we put something in front of an employee (an account manager) and they said, “wow, you are really going to make our users do that?!” That moment opened my eyes to a whole new (and different) world of feedback that I could gather internally. Not only that, but employees feel more empowered to make a difference, and contribute!

5. You test your user testing protocol before speaking with users
I would have to say, this is my favorite reason for conducting internal user research. I can run a pilot test of my research script, and see which of my own questions don’t make as much sense, and where the flow maybe gets confusing or awkward on my side. This gives me a small heads up to where the conversation might get stuck, and gives me some pointers of how to work through these before I head into a user testing session

When to do internal user testing

I definitely don’t use internal user testing for everything, but, whenever I do, it always compliments actual testing with real users. I usually focus on user testing in these scenarios:

1. Feedback on a prototype before putting it in front of users
   Internal testing is a great way to find bugs, typos and generally confusing information before you put something in front of actual users. The number of times I have found a issue with an Invision prototype, or confusing copy during an internal has been numerous, and allows for better feedback once you are speaking with your users
2. Testing out a research script
   As I already mentioned, I really appreciate running through my user test in a semi-realistic scenario before stepping into a user research session. Not only can I find problems with my interview flow, but it makes me feel more confident when it matters most
3. If you are building a product for employees
   Some products are actually built for your employees, which is the perfect time to run internal testing — in fact, this changes it from internal testing to user testing
4. When there is internal resistance to UX research
   Whenever I have started at a company who is resistant to the value of user research, I always start by running usability tests internally. Although the information is not the same, it can still highlight issues with feature or concept ideas that were not thought of upon conception. Testing with stakeholders can break down the barriers and open up the possibility of communication and discussion when it comes to “gut decisions”
5. Gathering basic feedback on a very confidential concept
   Sometimes conducting tests on a confidential subject can cause some issues with recruitment and user testing, especially if it is highly sensitive and the company is worried about others finding out. In this case, you employees have already signed NDAs, and are less likely to cause confidentiality problems than external participants

Internal user testing will not get you the same results as testing with your actual users. You can’t expect to walk out of an internal user session with the same, deep findings you would had you spoken to a real user. It is great if you were able to get some great, innovative ideas out of the session (especially if your internal stakeholders use the product), but I always make sure to tag those insights as “internal” and see if I hear these same ideas in external user testing.

Some tips to maximize internal user testing

With all this being said, there are ways to maximize your internal testing results, and here are a few tips I have to offer:

• Recruit a diverse pool of employees
  Make sure you recruit people who have been there for a long time, but also those who have joined recently; and be sure to pick from different teams. I always try to include the most “diverse” mix of employees possible
• Incentivize with food
  I use food as an incentive for everything. If I know people will not want to be in a meeting, I will bring cookies and candy to help get people excited about what we are about to do. Since you aren’t compensating your employees for their time, I always like to offer them some sweets as a thank you
• Maintain your expectations
  Ensure everyone involved in running the test understands that this is not the same as speaking with an actual user, and that the results are geared more towards making sure the prototype or concept makes sense, and is acceptable to put in front of users. You will probably get more than that type of feedback, but to always remember that feedback came from internal sources
• Let employees know what type of feedback you want
  I generally preface these sessions letting the participant know what kind of feedback I am looking for, such as feedback on how the prototype looks, the content of the prototype and, of course, whether anything is generally confusing. I will also ask usability-related tasks and for opinions, but I do want to make sure to maximize the type of feedback given the method type

Remember, internal user testing is not a “get out of jail free” card from speaking with your actual users, so please refrain fro only talking with employees. When used in the correct circumstances, and in conjunction with user testing, doing user research with your employees can be a really helpful experience!

Non-User Research

Why it is important to talk to people who don’t use your product

Source

We talk a lot about the importance of user research, but I don’t think there is enough focus on non-user research. What I mean by this is talking to people who don’t currently use your product, and have never used it in the past. When we only talk to users, I believe we are only getting half of the story. In a way, our users are biased, whether that is negatively or positively biased, they are still going to respond in a way that is significantly different.

I’m not sure when it became such a habit to only talk to users (at least for me and for some of the projects I have seen others go through). By focusing solely on only our users, we are potentially missing other important information that can make the experience better. We may be assuming that our non-users don’t want to use our product, but that is not necessarily the case. We might not be designing an experience accessible to these users, or we may have a key gap in our products. When we are only talking to users we are encouraging confirmation bias, even if we have the best of intentions.

Ultimately, when we talk about user research, we tend to refer to the holistic picture of an experience. But, what is a holistic picture without the other half of the story?

When and why talking to non-users is valuable

As you can probably deduct, I am a huge fan of talking to non-users, although I probably don’t do this nearly enough. It can be hard to argue talking to non-users when everyone on your team is focused on improving the product they work on everyday. However, there is so much more than just that product, and speaking with non-users really enables you to see a unique perspective!

I’m a big advocate of talking with non users in mainly three different situations:

• Foundational/competitive research
• Understanding how to approach/onboard new users
• Empathy

Foundational research
When it comes to talking to non-users for foundational research, I am typically using either Jobs To Be Done (JTBD) or Generative research sessions. This is because I can use these methods to deeply understand the person in front of me, completely separate from the product.

In particular, I find the JTBD approach important because you might discover new tasks people are looking to accomplish, or unknown barriers they are coming up against. These non-users will be using other products or services to accomplish the job, as opposed to your solution, so you can really understand what your product is missing. You can discover so much information about competitor products and service, in order to pivot in the best way that aligns with people’s expectations and needs.

Not only can these methods help you understand how users are currently accomplishing their goals with other products, but it can also give you insight into how to innovate. A lot of times I fall into the same habit of talking to users through generative research of JTBD research in order to find innovative avenues to explore, however, talking to non-users can give you an entirely different perspective. It allows you to explore the market in a broader sense, challenge assumptions and enter new areas.

Approaching new users/onboarding
Although less common, I do put screens, prototypes or concepts in front of non-users. Putting your current product (or a future state of a product) in front of non-users allows you explore how users relate with your product. I will often ask them what adjectives immediately come to their mind when they see the product. This helps to give an idea of what the immediate impact is of your product, which can also really help the marketing and branding teams (as a sidenote, I recently got the words “mundane” and “corporate” for a website I was testing, and it was incredibly helpful to get this outside perspective as we (including our users!) can be way too close to the designs).

Showing non-users the initial screens of the product also helps us gain their perspective when they are approaching the product and seeing the first screens. Not only can this indicate how the user got to a website, app or product (what they searched in Google, what they expected versus reality), but it can also really help us in understanding onboarding. People bounce from websites, apps and products so fast now, if they don’t see an immediate value, they will be gone. Showing this part of the product can allow you to understand what users would like in those important few seconds, and how to keep them engaged. You can also learn if anything presented in front of them would compel them to switch from their current solution.

Empathy
Ideally, teams are focused on their users and are trying to build technology to make user’s lives better. However, we can easily get stuck in a rut of thinking everyone is the same, and it is really important to remind ourselves to empathize outside the box. Non-users can often provide insights into critical issues such as accessibility, understanding or demographic factors (think more social influences) impacting engagement and usage of certain products. By speaking with non-users and users, not only are you breaking down barriers, but you are also understanding the struggles and reality of people you aren’t always thinking of.

When is talking to non-users not as valuable?

Although it is clear that interviewing non-users is super valuable in identifying new opportunities and perspectives, there are certain times when it is less ideal to speak with non-users:

• If the current priorities are around improving the experience for customers with a certain level of product experience
• If you need to choose between users or non-users to speak to, always start with improving your current product
• Finding bugs or other issues with the current user experience of a product
• Niche products that only certain groups use (ex: recruiting a teacher for a software made for surgeons)
• If you are prototype testing something that requires a certain level of understanding from users

When deciding whether or not to speak with non-users, just make sure to think about the goals of the study, and what your objectives are!

How to recruit non-users

Recruiting and talking to non-users can be much more challenging than finding people who currently use your product, however, it is definitely possible. The most important first step is to define who these non-users are, which I have given a rough list of below:

1. People with loyalty to another brand
2. People who are not interested in your product
3. Bad previous experiences with similar products
4. Misinformation of your product
5. Life circumstances

Sometimes it really can be impossible for a certain person to use your product, and that is why we still need to choose and prioritize our non-users. I would much rather speak to people with loyalty to another brand than people who are simply uninterested in the topic my product covers. Also, both people with bad previous experiences and misinformation are great candidates to better understand the gaps and negativity they perceive in your product.

As mentioned, it can be tough to recruit for non-users, but here is how I approach this problem. First, I set my criteria for recruitment:

1. Unfamiliar with the brand/product
2. Not using the product OR using a competitor product
3. Recently stopped using a competitor product OR never have used a product in the space
4. Bad experiences with competitor or similar products
5. Any demographics I am looking for (location, age, gender — but normally least important). This can also include jobs that may be really important to factor into your study, such as making sure you are recruiting the correct role for your software

Then comes the more difficult part: actually getting in touch with these people. I use a few hacky ways to get in touch with these users:

1. Posting the screener link (or just a description) on LinkedIn, Twitter or Facebook
2. Posting on a forum, such as Reddit
3. Reaching out to slack communities
4. Asking friends of friends (friends and family can be too close)
5. Using a recruitment agency (which can also be very expensive)

Just as a caution, you might get some less than ideal (I’m being nice here), participants through some of these methods, but that is part of the process. The same thing can happen with your actual users as well.

Another VERY important note — if you are recruiting using these methods for in-person tests, always try to meet and conduct the test in a public area, and never go alone.

What to do after you talking to non-users

After running a study with non-users, there is typically a lot of information to sort through, and a lot of ways you can highlight what you found. With this information, I will generally:

• Review the sessions and see if there are any trends in what non-users need that we could build in our product
• Compile why non-users are using (or switching) to certain competitor products
• Map the journey of a non-user, especially the journey of a non-user using a competitive product
• Encourage teams to get creative in coming up with ideas for non-users
• Create a non-user persona or visual breakdown
• Prototype a similar, but new type of product or service that would appeal to non-users

While talking to non-users might not be on the top of your priority list (we can’t always do everything at once!), it is still an important topic to consider. You have to make a decision whether doing some research, even with non-users, is better than doing no research at all. Maybe it is better to do nothing if you have a strong sense that it may lead you astray. Or maybe it’s better to talk to non-users if it will help you make something better, even if it’s not necessarily the biggest pain point for your current users.

User research plans: who cares and how to write one [with template]

Source

Research plans are something I absolutely swear by. I have been writing them for years and, at this point, I don’t think I could actually start a project without first creating a research plan (that may be a slight exaggeration, but it is mostly true). For me, they contain all of the necessary information about the research project, and really ensure the whole team is aligned in the right direction. In the past, they have really acted as a North Star in my projects.

What is a user research plan?

A user research plan is a form of documentation, which is why some people may shy away from it. I can honestly say, it is not referenced or used often, after the project has started, unless to point out a specific reason why we are conducting the study, or why we recruited certain types of participants.

Overall, they are an explanation about the research initiative taking place, serving as a kick-off document for a project, and really boil down to what the goal of the research project is.

Why do we create them?

User research is super exciting, to me at least, and it can be really easy to get off topic during a research project. Say you start a 3 week project with the intention to better understand your users through generative/discovery research and, suddenly, two weeks in, you realize the majority of the time is spent asking questions that the team is interested in and tacking on prototypes to the end of these sessions. Soon enough, your generative research hour-long session has turned into a mash-up of different subjects and Invision links. I’ve seen this happen a lot.

This doesn’t mean we can’t answer all these exciting questions, or test all these prototypes. What I do mean is we can do them, just separately. I have watched researchers try to mash too many things together at once and the results are confusing, at best. Albeit, sometimes I will add a simple prototype to the end of a session, but never more than two. It becomes extremely disorienting for your users to test a bunch of different concepts at once. This is where a well-crafted research plan comes into place,

Research plans keep everyone on track and make sure the overarching goals are well-defined and will get answered by the research. They keep the entire team aligned on the outcome, the bigger picture, instead of getting bogged down in the details, or switching the goal of the research in the middle, by mistake. Most importantly, they allow researchers (or whoever is doing the research) to really focus, and make sure the objectives of the research plan will be answered in the most effective and efficient way possible by the end of the project. We want to make sure we are actually answering the questions we set out to uncover, and research plans enable us to do so.

How to structure a research plan

For a while, when I first started writing these, they felt way too formal and too much like all the papers I used to write in academia. They can definitely have that sort of feeling about them, but are much lighter on the information. Usually, I don’t have more than a few sentences or bullet points for each section. The only time-consuming section will be the interview guide, which we will explore more.

• Background — a few sentences on what the research is about and why it is happening, which helps get people on the same page as to why this research is recurring, and what we hope to understand at the end. Arguably, this part is the most important for teams to align on first
• Stakeholders involved — the relevant internal team members who will be impacted by this research, and, potentially, the teams that initially came up with the research initiative
• Research objectives — what the teams want to learn from the research, or what they would like the outcomes to be. I like to put objectives this way: we should be able to answer all the objectives at the end of the research project, so, whether we are looking to understand an end-to-end journey or evaluate a new set of features, we should be able to come to conclusions about these objectives at the end. Read about how to write and elevate your research objectives here
• Methods — choose which research methods should be used in order to answer the research objectives. Mostly I am choosing between more generative methods in order to uncover motivations or broader insights, or usability testing to evaluate a current product or prototype
• Business objectives (optional, but recommended) — this is where you can discuss what business KPIs might be impacted through this research project, such as revenue, retention, acquisition, etc.
• Participants — who are the participants you are targeting for this particular project, such as demographics, geography, non-users versus users
• Interview guide — a cheat sheet containing topics/questions to follow during the interview
• UX Metrics (more applicable for usability tests) — what criteria would determine success, such as time on task or task success for a usability test
• Timeline — a general overview of how long the research will take

Gathering this information will require you to talk to other people, like product owners, developers or designers, which I always highly recommend doing. I ensure we are creating the plan together so that, before we start the project, I am convinced everything makes sense, and we are on the same page. Oftentimes, I will gather what they are interested in, or the questions they want asked, and turn them into more unbiased alternatives.

An example (and template)

Although I have a template available, I will use an example for context and to make this less abstract. Imagine you worked at a Homemade Dog Food App Company, and recently discovered that the team was interested in learning more about how people use the app to order dog food, and why they use this app over others.

Background:
For this particular research project, we are very focused on two main areas: 1. We want to understand the motivations behind why our customers order dog food on this app, and on others, and to gather a deep understanding of the journey people go on from when they think of this need, to when they are feeding their dogs; 2. We are interested in learning how customers currently use our product, and any pain points or problems they are experiencing

Stakeholders involved:
Retention team: product owner(s), designer(s), developer(s)
Acquisition team: product owner(s), designer(s), developer(s)

Method(s):
20 60 minute interviews, split between:

• Usability discovery (‘walk the store’) sessions
• Jobs To Be Done sessions
• Post-qualitative surveys on importance of features and concepts found in qualitative data

Research objectives:

• Discover people’s end-to-end journey when considering, purchasing and using dog food
• Uncover the current process people undergo when considering, purchasing and using dog food on this particular app
• Identify the competitive landscape of what additional products customers are currently using, and why
• Evaluate the current product, and any immediate pain points, or bugs, customers are facing

Business objectives:

• Customer retention rate
• Customer satisfaction rate (measured by: System Usability Scale)
• Users creating an account & logging in
• Understand competitors

Participants:
n=20

• 10 current users
• 10 non-users
• Male/female mix

Interview guide
This is it’s own topic itself, which we will cover in a future article, but the basic premise here is to include some potential questions you would ask during the interview. I usually tell people to form interview questions using the TEDW method, as they tend to open up conversation in an unbiased way. Below is the TEDW question method:

• “Tell me more…”
• “Explain…”
• “Describe what you mean by…”
• “Walk me through…”

UX Metrics

• Evaluation and decrease in the amount of bugs found in the product
• UX improvements for pain points, including potential prototypes which should be user tested
• JTBD statements and desired outcomes
• Personas and customer journey maps (even if wireframes)

Timeline
Approximately 4–6 weeks for recruitment, qualitative research, analysis, surveying and quantitative analysis

Create your own

Not only do I write research plans at the beginning of each project, which keeps the team really on track, but they are also a great tool to share and teach to colleagues. At my most recent position, I have been encouraging product owners to structure a draft of their own research plans before they come to me, which really prompts them to think about the questions they want to be answered, and the expected (or desired outcome). It has been a wonderful way to evangelize research throughout the company, and get people really thinking about user research.

If you are interested in writing your own, feel free to use my template (updated!), and let me know what you think, including any areas that seem unclear or unnecessary!

---

Interested in more user research? I teach an Introduction to User Research Course and am available for one-on-one mentoring. Check out the User Research Academy.

Please join the User Research Academy Slack Community for more updates, postings, and Q&A sessions

Asking about the future in user research

And why this method never works out well.

It is a tried and true saying: “never ask participants what they want” or “never ask about future behavior.” Why is this?

Let’s deviate for a second: 51 million Americans waste $1.8 billion dollars on unused gym memberships (Source). About 67% of people who sign up for the gym don’t go, yet they still signed up. They had every intention of going, but they didn’t. They failed to correctly predict their future behavior. This is why, as user researchers, we never ask about the future. Yes, gyms are still making money, but the fitness industry is worth about 80 billion US dollars (source), and is a very well-marketed field, which the same can’t be said about most companies trying to deliver paid-for features/memberships.

The only thing that can predict the future is the past.

The problem we face

User researchers constantly face this conundrum: we are expected to foretell the future of our users, to predict whether a user will pay for something, or even simply use something. All the time, I field questions from colleagues: “can you ask how much they would pay for this feature?”, “can you figure out how often they would use this feature?”, “can we gauge how much interest they would have in this new feature we are thinking about?”

As the voice of the user, and my desire to improve the world for, both, users and businesses, I walk a thin line between trying to ensure products increase revenue while making the user’s life more delightful. I don’t want users to create and pay for something they aren’t using or getting value out of. To me, that is a bit too much on the side of dark UX. I would rather be a part of something that makes the business money, but also provides tangible value to a user. I want the users, as well as the business, to have cake and eat it too.

Where asking about the future fails

Regardless of good intention, sometimes it can be really easy to fall into the future-based question trap. You want to get answers for your team, you want to give the guidance and direction, or else user research might be deemed as “useless” (it has happened). Not only are future-based questions completely unreliable and hypothetical, they can be quite uncomfortable for users to answer, and may lead to some social desirability bias.

Everybody has an aspirational self, and oftentimes, they lie to themselves and others because of this.

I have seen a wonderful example, which I have fictionalized:

Researcher: Okay, cool, so would you like an app that combines booking you a plane, a hotel as well as securing you a person who could dog-sit while you are away, or maybe just take your dog on walks…or, if you don’t have a dog, someone who house-sits and waters your plants?”

Participant: Um, yeah, that would be really nice.

Researcher: Awesome, so you would use that kind of app

Participant: Yeah, I think I would. It would help me out

Researcher: Cool, and how would that work or benefit you?

Participant: Well, I’ve never really thought about this…so just let me think…

There you go. “I’ve never really thought about this.” That is the golden nugget no one really wants to hear, but is extremely important. Yes, the participant said they would use an app that would, seemingly, make their lives easier, but they. have. never. thought. about. it.

Yes, there is a universe in which this particular service could be a hit, but there are a lot of risks associated with approaching solutions in this way. I’m in no ways trying to dissuade people from taking risks, sometimes it is necessary, but I would like teams to gather more reliable data before taking the leap.

it is hard, but possible. Researchers have to work a bit of magic.

How user researchers become psychics

As mentioned above, the past predicts the future. Instead of asking users how often they might use a feature, or how much they might pay for something, we have to reference their past. I have an exercise I commonly recommend to students, where we take future-based questions, and turn them into question about the past.

First, what are future-based questions?

• Would you use [feature/service/product/app]?
• How often would you use [feature/service/product/app]?
• Would you buy [feature/service/product/app]?
• On a scale of 1–10, how likely are you to buy [feature/service/product/app]?
• How much would you pay for [feature/service/product/app]?
• Would you pay X for [feature/service/product/app]?

I know I said these questions are bad, but a famous quote from Howard Schultz, former two times CEO of Starbucks should help highlight this concept:

If I went to a group of consumers and asked them if I should sell a $4 cup of coffee, what would they have told me?

Cool, great point, and I obviously agree. So how do we turn the future into the past. Just a note here, people may still lie when you ask them about the past, but it is still a much better indicator than asking them to predict their own future behavior.

How to predict the future from the past

Let’s use an example here. One of the easiest examples is a membership or subscription. I’ve always want to start a subscription box with stationary, based on user’s upcoming celebrations (so, I need 3 birthday cards next month and 2 congratulatory cards, etc). When I initially thought of this idea, I had a line of questions. I will put them in future-based format and then turn them into the past:

The future

• Would you use a subscription box that allowed you to have curated stationary for all the events you need delivered right to your door?
• How often would you want the subscription box to come?
• Would order a curated stationary subscription box?
• On a scale of 1–10, how likely are you to order a curated stationary subscription box?
• How much would you pay for a curated stationary subscription box?
• Would you pay $15 a month for a curated stationary subscription box?
• What else would you like to receive in your curated stationary subscription box? Tea, cookies, cats?

Quite honestly, if I do say so myself, this curated stationary box sounds pretty wonderful. In all fairness, I would have answered yes to these questions, and paid the $15 a month. However, it is quite funny because, there are stationary subscription boxes out there already, and, I subscribe to none of them. On top of that, once this was out, I can’t say for certainty that I would subscribe. Hell, I can’t even say for sure what will happen tomorrow, so how can I predict something like this? (If you answered yes to any of the above questions, shoot me a line. Kidding.)

The past

How do we turn these into more helpful questions?

First, more general questions:

• Have you ever ordered a subscription box in the past? Why or why not?
• What type of box?
• How often did it arrive? What was your experience with that?
• How did you feel about the experience?
• What value did this subscription box bring?
• If you can remember, how much did you pay for the subscription box? What did you think of that price?

Second, dive more into the specifics:

• Have you ever ordered a stationary subscription box in the past? Why or why not?
• How was your experience with the box?
• How often did it arrive? What was your experience with that?
• Walk me through the last stationary subscription box you received and the contents in it.
• How much did you pay for the stationary subscription box? What did you think of the price?
• How much would you pay for a curated stationary subscription box?
• What was missing from the stationary subscription box?

So, that is how I would switch from the future to the past. This is, potentially, assuming someone has ordered these in the past and, you might be sitting there thinking “what if my participant hasn’t ordered subscription boxes in the past?” There are three things you can do:

1. Screen your participants to ensure they have ordered them in the past
2. Take the time to figure out why they haven’t ordered subscription boxes in the past, as that can be very useful information when deciding either to move forward or not, as well as how to move forward
3. In this scenario, you could also ask about subscriptions in general, to understand what they subscribe to (such as Netflix, Amazon prime, etc), to better understand what might be missing from a subscription box

In addition to this, I have another simple method to help teams. Once the researcher conducts theses interviews, where the past is discussed and analyzed, don’t just jump straight into development and roll-out. Instead:

1. Prototype the concept — put this idea into a prototype form, and see how people react. I have done this before and heard people say, “wow, I would never pay for that” or “oh, yeah, that makes sense as a premium feature.”
2. Thereafter, A/B test the concept. Don’t just roll it out to all of your customers — test to see what happens: Is there a drop in retention? Or in conversion rate? Is no one interacting with the feature? Or the exact opposite?

With all these methods, there is really no reason to be asking future-based questions, and basing features or products solely around this shaky foundation. We want to build better products that improve people’s lives, so let’s use the best predictor to make sure we can accomplish that in the future!