Have you ever learned something amazing from user research, but then had trouble acting on it? Getting everyone excited about a significant change can be difficult, especially when you’ve only done a few user interviews. Sometimes, finding the right users to talk with is even tough.
Enter product analytics.
Before joining Heap, I had little experience with product analytics. Learning more about quantitative data and the best ways to use the different tools has leveled up my work as a designer.
There’s a lot I could cover, but I’ll stick to three ways I’ve found product analytics helpful.
1. Using it to find the right people to talk with
It’s super simple, but it turns out that product analytics is an amazing tool for targeting invites for surveys or interviews. Why? Because it can show you exactly which users are engaging with the features you want to get their feedback on.
The easiest way to do this is with a simple chart focused on a feature of interest. I’ll use Heap’s engagement chart as an example. (Note: you don’t have to do this in Heap! I just happen to work there.)
Start by making a new usage over time chart, and then pick an event that captures users engaging with the feature you’re interested in.
Group your chart by the property you use to identify users in Heap (usually the email or identity property). This will show results per user instead of overall numbers.
View results by “entire range,” then display them in a table for a clear list of all users who performed your event during the selected date range. Above, I found a list of users who were top users of our product detail page.
From here, you can export the list to CSV and use it to send survey or interview invites.
💡Too many users on your list? Narrow down to the most engaged with a funnel chart.
Funnel charts allow you to see a list of users who complete chosen events. In the above example, I’ve created a funnel to identify users who have viewed the same type of chart three times in a single session. (i.e. users who really engage with the chart!)
This creates a much smaller list of users to work with. They’re also more likely to have better feedback on the feature because they use it frequently.
2. Understanding the ‘why’ behind user behavior
We’ve looked at ways to make user research easier or more compelling, but what if there isn’t time for in-depth qualitative research? Product analytics can help you better understand user behavior through exploratory hypothesis building. I’ll show you what I mean!
We’re releasing a minor UI update to our chart navigation soon, and we were curious which actions in the header were used most often.
As you can see, there are a lot of choices. To explore what users do here, we created an engagement analysis—a new type of chart available in Heap.
It surprised us that the “Duplicate chart” has some of the highest usage even though it’s towards the bottom of the menu. Why is that?
To explore these questions more, we used session replay. This allowed us to see how and in what situations folks used this feature. We realized that, in many cases, folks were duplicating their charts to riff on something existing without starting from scratch.
Not only did this data help us better understand how header actions are used, but it also gave us future research ideas where we might look into these editing workflows in more detail.
3. Backing up qualitative research with hard numbers
Using product analytics to support, guide, and back up qualitative research is arguably the most powerful way to win over cross-functional partners. More importantly, it will make you feel more confident in your next steps. How you do this depends on the situation, but some scenarios come up often.
Use data to help confirm research findings
Let’s say you’ve just wrapped up a study where several new users said it was hard to get started in your app. This might feel like anecdotal evidence. You can use product analytics to see if this is widespread.
In this case, you could look at your team's existing metrics for onboarding workflows and filter them down to new users only. This process is called “segmentation” and is one of the most valuable tools to find insights in your data.
Use existing KPIs as a starting point for new research studies
You can also flip the process around. Let’s say a key metric, like checkout conversion, has been decreasing for the past few weeks. By grouping the data to see different user segments or different related events, you can see how conversion rates vary for almost any data. Some of the simplest ones to start with include landing page, country, and device type (i.e., mobile/desktop).
Quantify the scope of a problem
Finally, suppose users mention a particular error they’ve hit, or a workflow they find especially painful. In either case, you can use product analytics to see how their journey through your product compares to other users. This can help you quantify the impact of a particular issue or behavior. My favorite way to do this in Heap is with our journeys chart.
This journey shows how many users view a page (pageview event) and encounter an error while in a trial. It then shows how that affects the percentage of trial users who purchase compared to trial users who don’t experience an error.
To turn the journey into a direct comparison, select “Include users who skipped this step.” This makes it easy to see the impact of doing—or not doing—an event on the completion rate of the journey.
💡If an event has a higher conversion rate, be cautious. It’s tempting to think you need more users to do this event, but it’s almost always the case that doing any event on a site can be a sign of intent—even if the event is an error!
However, if an event has a lower conversion rate for users who do it, it’s a good sign you’ve found something important.
Product analytics = another tool in the design process
I’ve learned while working at Heap that the design process isn’t a recipe you must follow step by step. Instead, it’s more like a toolbox where you reach for a different set of tools depending on the project.
These three techniques in Heap have made product analytics one of the tools I use on almost every project, and I hope you’ll find them just as helpful in your design process!