NEW: Heap for mobile. Track every interaction, on every platform.

Learn more
skip to content
Loading...
    • The Digital Insights Platform Transform your digital experience
    • How Heap Works A video guide
    • How Heap Compares Heap vs. competitors
    • The Future of Insights A comic book guide
  • Data Insights

    • Session Replay Complete context with a single click
    • Illuminate Data science that pinpoints unknown friction
    • Journeys Visual maps of all user flows

    Data Analysis

    • Segments User cohorts for actionable insights
    • Dashboards Share insights on critical metrics
    • Charts Analyze everything about your users
    • Playbooks Plug-and-play templates and analyses

    Data Foundation

    • Capture Automatic event tracking and apis
    • Mobile Track and analyze your users across devices
    • Enrichment Add context to your data
    • Integrations Connect bi-directionally to other tools

    Data Management

    • Governance Keep data clean and trusted
    • Security & Privacy Security and compliance made simple
    • Infrastructure How we build for scale
    • Heap Connect Send Heap data directly to your warehouse
  • Solutions

    • Funnel Optimization Improve conversion in user flows
    • Product Adoption Maximize adoption across your site
    • User Behavior Understand what your users do
    • Product Led Growth Manage PLG with data

    Industries

    • SaaS Easily improve acquisition, retention, and expansion
    • eCommerce Increase purchases and order value
    • Financial Services Raise share of wallet and LTV

    Heap For Teams

    • Product Teams Optimize product activation, conversion and retention
    • Marketing Teams Optimize acquisition performance and costs
    • Data Teams Optimize behavioral data without code
  • Pricing
  • Support

    • Heap University Video Tutorials
    • Help Center How to use Heap
    • Heap Plays Tactical how-to guides
    • Heap Updates
    • Professional Services

    Resources

    • Blog A community for digital builders
    • Content Library Ebooks, whitepapers, videos, guides
    • Press News from and about Heap
    • Webinars & Events Virtual and live events
    • Careers Join us

    Ecosystem

    • Customer Community Join the conversation
    • Partners Technology and Solutions Partners
    • Developers
    • Customers Over 8,000 successful companies
  • Free TrialRequest Demo
  • Log In
  • Free Trial
  • Request Demo
  • Log In

All Blogs

Data Stories

The Single Most Useful Analytics Strategy for Early-Stage Startups

Ravi Parikh
May 25, 20144 min read
  • Facebook
  • Twitter
  • LinkedIn
Heap

In the early days of Heap, we pushed a change to our dashboard that caused engagement to skyrocket. The day after the change went live, people on average ran over twice as many queries in the Heap dashboard as the day before.

The way I phrased that is a bit misleading. While mostly true, the increased engagement had nothing to do with the change. It was entirely attributable to our tiny sample size. At the time we had fewer than 20 beta users, so there was large fluctuation in usage from day to day.

This illustrates a problem all early-stage products face: there isn’t enough data to do traditional analytics. It’s impossible to get anything approaching statistical significance when running A/B tests or measuring engagement with only a few beta users. It’s especially frustrating because it seems at odds with the widespread “Lean Startup” mentality. How are you supposed to measure and iterate quickly without real data?

A large part of early stage development is talking to customers, following your product vision, and a lot of guesswork. But there’s still room for data and analytics. The single most important tool at your disposal in an early stage product is individual user analysis: the process of monitoring user activity at a granular, individualized level. In this article we’ll take a look at how it works, sample use cases, and things to look out for.

Individual user analysis

Record what your users are doing at a much more granular level and you’ll be able to see patterns that don’t require statistical significance. Here are a few examples:

  • A user is spending a lot of time using your product, but isn’t using the features you think are valuable. At Heap, we noticed that sometimes people would only run queries that were much better suited to Google Analytics. This meant that we weren’t doing a good enough job communicating our value proposition or onboarding users. Even mature product have this problem, as this post about Evernote demonstrates.

  • You notice that every time someone tries to use “Feature X” for the first time, they visit the documentation page or send you a support ticket. You only need to see this happen a few times to realize that the UI for Feature X doesn’t do a good job communicating its function.

  • A customer decides to cancel their account. You can look back and see exactly what they did prior to cancellation. Maybe they got frustrated trying to figure out a certain feature or never went through the full setup process.

What this isn’t: You might think “this seems strictly worse than speaking with your users.” Individual user analysis is more than just a lo-fi version of real user feedback, for many reasons:

  • Things may not seem significant enough to mention from the user’s perspective, but might be from yours. For example, maybe users are ignoring a button in your UI. They don’t think to mention this, but it’s an important feature that you’d like to measure.

  • A user’s memory is incomplete. They may remember the high-level actions they took, but not the exact paths they took or why they did what they did.

  • Also, memory is malleable and flawed. A user may give a neat post-hoc explanation for how they interacted with your product, but the reality may be different.

  • And of course, your users may not always want to have in-depth conversations with you.

The right solution is to get as complete a picture of your users as possible. Combine your conversations, tracking data, and any known user metadata into a holistic picture of how someone is using your product.

Implementation

Tools like Google Analytics don’t help. GA is built for aggregate reports and doesn’t allow individual user analysis at all. Other event-based analytics tools will get you closer, but require manual event instrumentation to track events of interest. This is the opposite of the approach you want in the early stages: it’s time consuming, expensive, and difficult to track usage at a high level of granularity.There are other solutions better geared towards individual user analysis:

  • Session recording tools like ClickTale allow you to play back what users are doing, getting the next-best thing to looking over a user’s shoulder as they navigate your website.

  • Our own product Heap takes a different approach. Heap automatically tracks pageviews, clicks, form submission events, and more without any custom code. This allows you to see at a glance the discrete actions a user took on your site. Because these actions are discrete rather than a whole session recording, Heap also allows for aggregate analytics (conversion funnels, segmentation, etc) once your product grows.

Caveats

Individual user analysis is a powerful tool, but there are a few things to watch out for.

  • Be mindful of more than just the raw data. Data can help inform product decisions and where you should invest your time, but it’s not gospel. Individual user analysis can’t tell you how to pivot to reach product-market fit, or whether there’s demand for a feature that doesn’t yet exist.

  • You have a non-representative sample. Early users of startups are often quite different from the eventual userbase. They’re more likely to be tolerant of bugs. They may even know you personally and be invested in the success of your product. While you can gain useful insights from monitoring these users, don’t overgeneralize.

  • It doesn’t scale. Your product may reach a stage where you need a different analytics process. Even with just a few hundred active users, you have enough data to start relying on aggregate data. High-level engagement numbers and conversion funnels will begin to matter more. Regardless, individual user analysis is still useful to massive applications. It’s important to have a qualitative, detailed view of what users are doing.

Ravi Parikh

Was this helpful?
PreviousNext

Related Stories

See All

  • Google Analytics 4

    Product Insights

    Google Analytics 4: What it promises, and what that really means

    April 28, 2022

  • Heap.io

    How to

    The 3 key first steps to improving CRO

    March 29, 2023

  • Heap.io

    Data Stories

    Celebrating H&R Block as the inaugural winner of the Digital Innovator Award

    March 22, 2023

Subscribe

Sign up to stay on top of the latest posts.

Better insights. Faster.

Request Demo
  • Platform
  • Capture
  • Enrichment
  • Integrations
  • Governance
  • Security & Privacy
  • Infrastructure
  • Illuminate
  • Segments
  • Charts
  • Dashboards
  • Playbooks
  • Use Cases
  • Funnel Optimization
  • Product Adoption
  • User Behavior
  • Product Led Growth
  • Customer 360
  • SaaS
  • eCommerce
  • Financial Services
  • Why Heap
  • The Digital Insights Platform
  • How Heap Works
  • How Heap Compares
  • The Future of Insights
  • Resources
  • Blog
  • Content Library
  • Events
  • Topics
  • Heap University
  • Community
  • Professional Services
  • Company
  • About
  • Partners
  • Press
  • Careers
  • Customers
  • Support
  • Request Demo
  • Help Center
  • Contact Us
  • Pricing
  • Social
  • Twitter
  • Facebook
  • LinkedIn
  • YouTube

© 2023 Heap Inc. All Rights Reserved.

  • Legal
  • Privacy Policy
  • Status
  • Trust