The civil union between User Research & KPIs
The Well-Known Usability KPIs
When I first started my journey into user experience research, KPIs didn’t even cross my mind. To me, they simply weren’t applicable to user research, especially qualitative research. I gave myself the excuse to shy away from KPIs, due to their irrelevance to my field and expertise, and would argue that the link between these two spaces simply wasn’t there. Yes, research could impact KPIs, but what did it matter?
As I learned more about usability testing, the idea of measuring KPIs became more tangible and I started warming up to it. However, the KPIs I was looking at were purely behavioral or attitudinal, such as:
- Binary task Success
- Time on task
- Number of errors
- Single Ease Question (SEQ)
- System Usability Scale (SUS)
It felt nice quantifying data for a change. If someone commented on the inability to draw conclusions from qualitative data (another post coming soon), I could turn around and show them these metrics. A whole new world opened to me in the marriage of qualitative and quantitative data — I could show statistics, benchmark, talk about confidence intervals, compare products/services to competitors — the world was my UX oyster. As I’ve grown in my career, I highly value these metrics (though they aren’t always used) and encourage UXers to find ways to bring qualitative and quantitative data together as a holistic view of an experience.
I know, duh, this isn’t news to anyone. Ideally, a holistic experience is key for truly understanding the problems users face. Yet, after learning about these KPIs, I ran into new ones…
Business KPIs, OH MY!
I was able to wrap my brain around usability metrics and those translating into usability KPIs. I could also bridge the gap between user research and NPS (albeit, quite difficult for me because of my thoughts on the amount of confounding variables that go into NPS). It started to make sense. Then I was introduced to the world of business KPIs through pitching research project proposals to stakeholders. I quickly realized that most stakeholders, especially higher up, don’t particularly care about time on task or the SEQ. They care about revenue. It makes sense — my job is to advocate for the user, their job is to advocate for keeping a business running.
I wracked my brain for ways to better pitch research to this audience, who were primarily interested in financials. The typical financial metrics are: revenue, acquisition and retention and I had to relate user experience research back to these KPIs I knew little about.
Where did I start? Research. I started by uncovering what these metrics actually meant. Revenue was understandable, but felt abstract, so I focused in on acquisition and retention, both of which impact revenue. How do we get more users and, subsequently, how do we keep them?
To me, the lightbulb was immediate: you acquire new users through having a seamless sign-up, whether it is a form, information on your website, a checkout flow, a trial or a download (among many more). You keep users by having a satisfying experience within your product. How does one know if you are both seamless and satisfying? You guessed it, user experience research.
I started formatting my proposals to reflect this point-of-view by relating the research objectives to the potential impact the business. For example, an evaluative approach: exploring user’s current frustrations with a checkout flow, and testing new ideas that alleviate said annoyances, could positively impact both acquisition and retention; or, a generative approach: understanding user’s overarching pain points and needs within a platform could increase retention and customer lifetime value, while reducing churn rate. Of course, I also put in numbers from previous data, for instance, showing the drop off rates at certain points of the journey. Stakeholders listened! Huzzah!
Learnings
While this may not be innovative to all, it was definitely a great learning experience for me in expanding my view from solely making the customer happy, to also incorporate more high-level strategy and players. I recommend playing around with these KPIs, and others, when thinking about your research objectives — it opens more doors to creativity and innovation. I have listed some qualitative methods I have found helpful when thinking about these business KPIs. Please reach out or comment if you have any ideas, I would love to hear how others approach this!
- Acquisition: Card sorting, mental models, usability testing, competitive analysis/benchmarking, ideal journey mapping
- Retention: Generative research sessions, usability testing, tree testing, participatory design, contextual inquiry, projective exercises, journey mapping, dairy studies
Even though my title is user experience researcher, I can’t only focus on the end-user, but have to encompass everyone involved, from stakeholders to other departments, in my understanding and empathy.
[embed]https://upscri.be/50d69a/[/embed]