Navigating the Data Maze: A User Researcher's Guide to Using Data Analytics
👋🏻Hi, this is Nikki with a 🔒subscriber-only 🔒 article from User Research Academy. In every article, I cover in-depth topics on how to conduct user research, grow in your career, and fall in love with the craft of user research again.
As a qualitative user researcher, I often felt overwhelmed by quantitative data and was unsure how to fit it into my user research process. I did my best to circle around the problem as much as possible, sometimes even ignoring the need my teams had for quantitative data to back up the qualitative. And for some time, I even said that quantitative data wasn’t necessary and all you really needed was some qualitative research because it told a better story.
Eeek. 🙈
I did this not because I didn’t find quantitative data useful, but because I had no idea how to use it in user research. I was so unfamiliar with product analytics and using data this way that I tried to avoid it at all costs.
There is no other way to put this: avoiding quantitative data negatively impacted my career. Data can help in many ways, including how we track our impact and show value as user researchers.
Finally, there came a point in my career where I couldn’t avoid using quantitative data in my work anymore and I had to learn how to incorporate this critical approach into my day-to-day (well, really, it didn’t start as day-to-day, but rather the occasional experiment). I made using quantitative data my top professional development goal so I had plenty of motivation to make positive progress (I was also terrified at the same time).
Once I learned to work better with data, I didn’t look back. It changed how I approached user research. As a disclaimer, I am not a quantitative user researcher and have some mixed methods experience, but I would still call myself a qualitative user researcher. So, this is a guide for those in that space looking to get more comfortable incorporating quantitative data into their process.
Importance of Quantitative Data
There are three main types of research design:
Exploratory sequential design involves initial qualitative research to explore a concept, followed by quantitative research to test these explorations broadly. For example, interviews to understand user needs are followed by surveys of a larger audience to quantify needs.
Explanatory sequential design reverses this order, beginning with quantitative research to identify trends and then using qualitative methods to delve deeper into those findings, like surveying for top pain points and conducting interviews to get detailed insights on why/how those pain points occur.
Parallel convergent design occurs when qualitative and quantitative research are conducted independently and simultaneously, and then results are merged to reach a conclusion. This approach allows for comprehensive insights by combining the depth of qualitative data with the breadth of quantitative data, enhancing the understanding of research findings. An example is simultaneously conducting interviews to explore users’ needs and goals while deploying surveys to measure user satisfaction levels and then integrating these insights to guide product development decisions.
For the majority of my career, I focused on exploratory design. I sometimes threw in a survey at the end, but mostly as a way to quell stakeholders’ needs for quantifying data or getting numbers rather than as a way to gather more data.
However, there were many times when I could have benefitted from explanatory sequential design, starting with data analytics.
Data analytics can give you an extremely clear picture of what is happening. It can show user behavior, trends, and potential problems and greatly enhance qualitative data. Additionally, data can help you with narrowing the scope of your research project.
There were many projects I started that were way too broad and could have used narrowing down. One of the best ways to do this is using quantitative data to better understand the landscape before diving into a broader topic or problem space. I remember once having to tackle the question of churn. At that point, I decided to try to tackle churn through interviews.
It was rough. I barely spoke to anyone because, well, churned users don’t care about helping the company they no longer use, and the interviews felt all over the place. There was little structure, and I would have benefitted from incorporating quantitative data into the project.
By making the shift toward using quantitative data, you can:
Use more objective decision-making based on data as it will show you what is happening, such as the concrete user behavior.
Enhance deliverables, such as journey maps and personas, by including concrete behavior data and quantifying qualitative information, such as needs, goals, and pain points.
Narrow the scope of your research projects to be more effective and impactful so you focus on a tangible scope.
Through product analytics, such as bounce rates, drop-offs, session durations, and other metrics, you can see potential pain points or friction in your product/service.
Quantitative data can help in various ways, and learning these skills is very important to move your career forward as a user researcher. This doesn’t mean you have to become a quantitative or mixed-methods user researcher, but knowing how to utilize quantitative data can up-level your skills and help you advance in your career. It also makes you feel more confident in the studies you run!
So, how do we go about bringing quantitative data into the fold?
Defining Research Goals
The first and most important thing to do as a user researcher starting a new project, especially when starting a project type different from the one you are used to, is to write out the project's goals. This stage was critical when incorporating more quantitative data into my research process. It helped me better understand and define what I was trying to accomplish through the project and by using quantitative data.
Research goals help you give focus to your study and enable you to understand what approaches to take and methods to use to get you the information you need most effectively and efficiently. When I first started using quantitative data in research and experimented with explanatory sequential design, my goals looked like this:
Look at data to understand what is happening
Explore data analytics to understand behavior
While it was great that I looked at this data, I still had no clear goals in mind, so I got lost in the product analytics, uncertain of what I was looking for. This approach wasted a lot of time and, ultimately, ended up with me feeling frustrated and defeated in the process.
I wouldn’t have written qualitative research goals like this, so I set out to revise how I approached quantitative research goals.
Writing Quantitative Research Goals
Research goals directly relate to your research statement because they are the more in-depth areas you want to explore in your research statement that will help you answer what you are trying to learn. Your research goals should address what you want to learn and how you will study the research statement.
You want to gather information about these goals by the end of the study. They aren’t posed as questions, but you want to be able to “answer” them by getting enough data to feel comfortable making decisions.
To define these goals, I ask myself the following questions:
What do we want to learn about [research topic]?
What type of experiences do we want to learn about?
What information do we want at the end of the study?
What decisions are we trying to make by the end of the study, and what can help us make those decisions more confidently?
When it came to qualitative research, this felt easy to me because I was familiar with writing goals about deeply understanding peoples’ processes and mental models. However, writing goals for quantitative research projects confused me. I wasn’t exactly sure how to write goals about understanding what was happening or about behavior since I had avoided these types of goals in my qualitative studies.
So, I went back to basics and asked myself the above questions, but with a quantitative lens. When it came to updating my research project goals, they looked more like this:
Understand the trends of behavior patterns within [x flow]/[y topic]
Identify potential pain points or friction within [x flow]
Measure user satisfaction when it comes to [x product]/[y topic]
Evaluate the top behaviors within [x product]/[y flow]
With these types of goals, it became clearer what I wanted to achieve when looking at the quantitative data rather than simply “exploring data.”
When writing quantitative research goals, we are looking at what is happening, so try to align your goals to identify behaviors and patterns within the quantitative data you are looking at.
Tying into Business Objectives
Conducting impactful user research means engaging in research aligned with business objectives. By understanding what the business is trying to achieve, we can utilize that information to help give us clarity and direction on what we need to understand better and ensure we are researching the most important information for the business.
I struggled a lot with integrating research into the business, and for a while, I thought it was simply impossible. I believed research lived just outside the scope of business and that I couldn’t bring the two together. However, over time, I was able to better understand business needs and see how research could help a business achieve its objectives.
Regardless if you are conducting qualitative or quantitative research (or both!), it is essential for us to align our research with greater business objectives. I’ve found this easier to do regarding quantitative user research.
For example, imagine you work at a company that streams TV shows and movies, and your business goal is to enhance user engagement; you can use that to guide your research study. Once you define user engagement (let’s say for this we can say viewing time and subscription renewals), you can dive right in with quantitative data to better understand what is happening, such as:
What is the current average viewing time, and how does this differ from our ideal viewing time?
What is the current subscription renewal rate, and how does this differ from our ideal?
Where are the potential frictions/pain points in the current viewing experience?
What are people searching for that currently might not be on our platform?
Straight off the bat, you can dig into analytics to understand what people are interacting with and start to form some impactful research goals like these:
Understand what users are currently interacting with and where there are gaps in our platform
Identify potential pain points in the viewing experience
Evaluate any trends behind when people decide not to resubscribe (e.g., after a certain period of time)
With these goals in mind, you can use the following approaches and methods to gather the information to answer these goals:
Keep reading with a 7-day free trial
Subscribe to User Research Academy to keep reading this post and get 7 days of free access to the full post archives.