Asking about the future in user research

And why this method never works out well.

It is a tried and true saying: “never ask participants what they want” or “never ask about future behavior.” Why is this?

Let’s deviate for a second: 51 million Americans waste $1.8 billion dollars on unused gym memberships (Source). About 67% of people who sign up for the gym don’t go, yet they still signed up. They had every intention of going, but they didn’t. They failed to correctly predict their future behavior. This is why, as user researchers, we never ask about the future. Yes, gyms are still making money, but the fitness industry is worth about 80 billion US dollars (source), and is a very well-marketed field, which the same can’t be said about most companies trying to deliver paid-for features/memberships.

The only thing that can predict the future is the past.

The problem we face

User researchers constantly face this conundrum: we are expected to foretell the future of our users, to predict whether a user will pay for something, or even simply use something. All the time, I field questions from colleagues: “can you ask how much they would pay for this feature?”, “can you figure out how often they would use this feature?”, “can we gauge how much interest they would have in this new feature we are thinking about?”

As the voice of the user, and my desire to improve the world for, both, users and businesses, I walk a thin line between trying to ensure products increase revenue while making the user’s life more delightful. I don’t want users to create and pay for something they aren’t using or getting value out of. To me, that is a bit too much on the side of dark UX. I would rather be a part of something that makes the business money, but also provides tangible value to a user. I want the users, as well as the business, to have cake and eat it too.

Where asking about the future fails

Regardless of good intention, sometimes it can be really easy to fall into the future-based question trap. You want to get answers for your team, you want to give the guidance and direction, or else user research might be deemed as “useless” (it has happened). Not only are future-based questions completely unreliable and hypothetical, they can be quite uncomfortable for users to answer, and may lead to some social desirability bias.

Everybody has an aspirational self, and oftentimes, they lie to themselves and others because of this.

I have seen a wonderful example, which I have fictionalized:

Researcher: Okay, cool, so would you like an app that combines booking you a plane, a hotel as well as securing you a person who could dog-sit while you are away, or maybe just take your dog on walks…or, if you don’t have a dog, someone who house-sits and waters your plants?”

Participant: Um, yeah, that would be really nice.

Researcher: Awesome, so you would use that kind of app

Participant: Yeah, I think I would. It would help me out

Researcher: Cool, and how would that work or benefit you?

Participant: Well, I’ve never really thought about this…so just let me think…

There you go. “I’ve never really thought about this.” That is the golden nugget no one really wants to hear, but is extremely important. Yes, the participant said they would use an app that would, seemingly, make their lives easier, but they. have. never. thought. about. it.

Yes, there is a universe in which this particular service could be a hit, but there are a lot of risks associated with approaching solutions in this way. I’m in no ways trying to dissuade people from taking risks, sometimes it is necessary, but I would like teams to gather more reliable data before taking the leap.

it is hard, but possible. Researchers have to work a bit of magic.

How user researchers become psychics

As mentioned above, the past predicts the future. Instead of asking users how often they might use a feature, or how much they might pay for something, we have to reference their past. I have an exercise I commonly recommend to students, where we take future-based questions, and turn them into question about the past.

First, what are future-based questions?

• Would you use [feature/service/product/app]?
• How often would you use [feature/service/product/app]?
• Would you buy [feature/service/product/app]?
• On a scale of 1–10, how likely are you to buy [feature/service/product/app]?
• How much would you pay for [feature/service/product/app]?
• Would you pay X for [feature/service/product/app]?

I know I said these questions are bad, but a famous quote from Howard Schultz, former two times CEO of Starbucks should help highlight this concept:

If I went to a group of consumers and asked them if I should sell a $4 cup of coffee, what would they have told me?

Cool, great point, and I obviously agree. So how do we turn the future into the past. Just a note here, people may still lie when you ask them about the past, but it is still a much better indicator than asking them to predict their own future behavior.

How to predict the future from the past

Let’s use an example here. One of the easiest examples is a membership or subscription. I’ve always want to start a subscription box with stationary, based on user’s upcoming celebrations (so, I need 3 birthday cards next month and 2 congratulatory cards, etc). When I initially thought of this idea, I had a line of questions. I will put them in future-based format and then turn them into the past:

The future

• Would you use a subscription box that allowed you to have curated stationary for all the events you need delivered right to your door?
• How often would you want the subscription box to come?
• Would order a curated stationary subscription box?
• On a scale of 1–10, how likely are you to order a curated stationary subscription box?
• How much would you pay for a curated stationary subscription box?
• Would you pay $15 a month for a curated stationary subscription box?
• What else would you like to receive in your curated stationary subscription box? Tea, cookies, cats?

Quite honestly, if I do say so myself, this curated stationary box sounds pretty wonderful. In all fairness, I would have answered yes to these questions, and paid the $15 a month. However, it is quite funny because, there are stationary subscription boxes out there already, and, I subscribe to none of them. On top of that, once this was out, I can’t say for certainty that I would subscribe. Hell, I can’t even say for sure what will happen tomorrow, so how can I predict something like this? (If you answered yes to any of the above questions, shoot me a line. Kidding.)

The past

How do we turn these into more helpful questions?

First, more general questions:

• Have you ever ordered a subscription box in the past? Why or why not?
• What type of box?
• How often did it arrive? What was your experience with that?
• How did you feel about the experience?
• What value did this subscription box bring?
• If you can remember, how much did you pay for the subscription box? What did you think of that price?

Second, dive more into the specifics:

• Have you ever ordered a stationary subscription box in the past? Why or why not?
• How was your experience with the box?
• How often did it arrive? What was your experience with that?
• Walk me through the last stationary subscription box you received and the contents in it.
• How much did you pay for the stationary subscription box? What did you think of the price?
• How much would you pay for a curated stationary subscription box?
• What was missing from the stationary subscription box?

So, that is how I would switch from the future to the past. This is, potentially, assuming someone has ordered these in the past and, you might be sitting there thinking “what if my participant hasn’t ordered subscription boxes in the past?” There are three things you can do:

1. Screen your participants to ensure they have ordered them in the past
2. Take the time to figure out why they haven’t ordered subscription boxes in the past, as that can be very useful information when deciding either to move forward or not, as well as how to move forward
3. In this scenario, you could also ask about subscriptions in general, to understand what they subscribe to (such as Netflix, Amazon prime, etc), to better understand what might be missing from a subscription box

In addition to this, I have another simple method to help teams. Once the researcher conducts theses interviews, where the past is discussed and analyzed, don’t just jump straight into development and roll-out. Instead:

1. Prototype the concept — put this idea into a prototype form, and see how people react. I have done this before and heard people say, “wow, I would never pay for that” or “oh, yeah, that makes sense as a premium feature.”
2. Thereafter, A/B test the concept. Don’t just roll it out to all of your customers — test to see what happens: Is there a drop in retention? Or in conversion rate? Is no one interacting with the feature? Or the exact opposite?

With all these methods, there is really no reason to be asking future-based questions, and basing features or products solely around this shaky foundation. We want to build better products that improve people’s lives, so let’s use the best predictor to make sure we can accomplish that in the future!