Improving our Mobile Event Visualizers with Product Data

This post details how we used insights we found in our behavioral data to drive investment decisions and make rapid product improvements in an efficient way. We were able to increase usage of a key new feature from 8.25% before the new experience to 38.7%.

Earlier this year, we set out to improve the experience for Product Managers using Heap on mobile apps. We focused on our investments on making improvements to the event creation workflow for mobile apps. This included heavy investment in improving our Event Visualizers on mobile: a way to visually click and tag events in your product.

Some background on how Heap works: because of Heap’s Autocapture technology, the process of creating new events in Heap is called “defining” new events. In Heap the event data has already been captured, but no name exists for it yet, so Heap does not surface it to you for analysis. Once you tell Heap what a particular event is (basically applying a name to it), that event is then surfaced within Heap and you’re able to analyze it.

Our Android Event Visualizer allows users to pair a device or emulator with Heap and then see the interactions you trigger in the app, as well as some useful screenshots that make it easier to confidently tag the events you care about.

When we first introduced our Android SDK, we didn’t initially provide an Event Visualizer experience. This meant that our users had to jump through some hoops to create new events on Android apps. The most common approach was to use the Live View within Heap, and create a filter for your IP address. That allowed our users to only see the events they were generating, and tag events based on the raw data.

There were a number of limitations in this workflow. For example, you can’t have both the event definition dialog and see all of the properties associated with the event on the same screen. This meant that in order to define an Android event, our users had to open multiple tabs and switch between them — not an ideal experience!

Using Product Analytics: Heap On Heap

As a product manager at an analytics company, I love using product data to back my decisions. (One of our mottos here at Heap is to power business decisions with truth.) In this case, it didn’t require looking at any data to know that this experience needed to be improved. There was some clear low-hanging fruit here, and we knew that we wanted to create an experience where it was possible to do all of the normal event definition activities from a single screen. We also knew that it can be a challenge to identify an event just looking at raw event data, so wanted to make sure we could provide a useful screenshot (and an indicator showing where the event occurred in the case of touches), which would make it much easier to contextualize the data. The goal here was to improve the event creation experience for all users no matter the level of technical sophistication.

Another core goal of building the Android Visualizer was to make it simple to pair an emulator or phone with Heap. We started by providing an API to initiate pairing from the app, and a developer could use stick a button in an app debug menu to enable PMs to activate the pairing. We planned on adding a gesture later, but decided to release our first beta version to early customers with just the API.

This produced little fruit, however. We determined it was because it was a lot of effort and time-consuming to have developers wire up a new button. So after a little waiting, we decided to build in a default pairing gesture. This made it a lot easier for users to get started with the Visualizer, and enabled us to drive initial usage. Unfortunately after about a month of availability – while there was some use of the Event Visualizer – it still lagged significantly behind the other methods for defining events, such as using the Live View.

Usage in the month before the guide was put in place

This was disappointing, so we turned to another tool at our disposal: in-product walk throughs. Based on a hypothesis that the Event Visualizer was simply not easily discoverable for existing users, we added an in-product walk through to the app. Any time someone came to the Live View or the Events screen to define an Android event, we’d gently nudge them in the direction of our new Event Visualizer.

After about two weeks of the guide being live, we saw dramatic results:

With the guide in place, we saw an increase in Event Visualizer usage from 8.25% of all new events to 38.7% after the guide was added.

Scaling These Learnings to Other Features

With the new Android Event Visualizer experience in place, we started taking a look at our iOS Event Visualizer experience, with the plan of giving it a big facelift to match the Android Visualizer. We’ve supported an iOS Event Visualizer experience for quite a few years, although the iOS Visualizer experience has a bit less polish than the Android Visualizer:

There’s a lot less flexibility in terms of which properties you can filter on, and the screenshots that come back are a lot less informative, since they’re cropped down to just the UI component that was tapped. We also knew that the pairing process for iOS was a lot less ideal than what we’d built for Android. On iOS, when you go to the Event Visualizer page in the Heap app, it triggers an alert in the mobile app asking if you want to pair with the device. This pairing experience produced unintended results sometimes, and a number of users were complaining that they would accidentally pair with a device belonging to someone else on their team who was doing testing at the same time.

Overhauling the entire iOS Event Visualizer was looking like it might be a large amount of engineering work, so we went to our product analytics data to try to understand the problem more:

What we found was unexpected compared to our Android SDK users. Within Heap, we learned that the overwhelming majority of users creating new events for iOS were already using the iOS Event Visualizer, which suggested that there were no major issues with creating events with it. The workflow is not the prettiest thing in Heap’s product, but seems to be doing the trick for most people.

Pairing with the iOS Event Visualizer, however, was a known pain point for our users. The iOS Event Visualizer used a server-side pairing mechanism, which triggered when a user opened the Event Visualizer in their browser. This would trigger an alert on a device asking to pair. It was intended only to be used in debug builds by employees of the app maker, but we’d had an incident where the Event Visualizer was accidentally enabled in a customer’s production build. This resulted in end users seeing unexpected pairing notifications. This presented risks and concerns for our customers, and was the most common complaint with our iOS Event Visualizer.

The behavioral data we were able to see in Heap allowed us to make a strategic decision to in our product investments in this area. We decided to scope down the project significantly, and we went from a planning full refresh of the Event Visualizer to just revamping the pairing mechanism. Our shifted focus to provide a consistent experience with the Android Visualizer – and critically – to change it to a client-side trigger.

The product usage data enabled us to focus on what mattered most and to build with agility. We were able to significantly reduce the project risk and engineering effort, while also delivering a high-value, customer-delighting change.

Leveraging Heap to analyze product analytics data and how customers use our product makes it possible for us to make quicker, more effective decisions and prioritize the things that will make the biggest impact for our customers. We’re continuing to invest heavily in the product experience for our mobile users, and this includes our mobile Event Visualizers. We have significant improvements and new capabilities planned for the remainder of this year.

If you’ve been using Heap to analyze mobile app data, hopefully you’ve had a chance to see the improvements we’ve been making. If you haven’t seen them or don’t use Heap for mobile app product analytics, you can try Heap for free.