Unlock 2025 Benchmark data → Access insights to stay ahead in the digital experience race.

Get the Report
skip to content
Loading...
    • Why Product Analytics And what can it do for you?
    • How Heap Works A video guide
    • How Heap Compares Heap vs. competitors
    • Product Analytics + Digital Experience Analytics A deeper dive
    • The Future of Insights A comic book guide
    Watch a Demo
  • Data Insights

    • Journeys Visual maps of all user flows
    • Sense AI Analytics for everyone
    • Web Analytics Integrate key web metrics
    • Session Replay Complete context with a single click
    • Heatmaps Visualize user behavior instantly
    • Heap Illuminate Data science that pinpoints unknown friction

    Data Analysis

    • Segments User cohorts for actionable insights
    • Dashboards Share insights on critical metrics
    • Charts Analyze everything about your users
    • Playbooks Plug-and-play templates and analyses

    Data Foundation

    • Capture Automatic event tracking and apis
    • Mobile Track and analyze your users across devices
    • Enrichment Add context to your data
    • Integrations Connect bi-directionally to other tools

    Data Management

    • Governance Keep data clean and trusted
    • Security & Privacy Security and compliance made simple
    • Infrastructure How we build for scale
    • Heap Connect Send Heap data directly to your warehouse
  • Solutions

    • Funnel Optimization Improve conversion in user flows
    • Product Adoption Maximize adoption across your site
    • User Behavior Understand what your users do
    • Product Led Growth Manage PLG with data

    Industries

    • SaaS Easily improve acquisition, retention, and expansion
    • Retail and eComm Increase purchases and order value
    • Healthcare Build better digital patient experiences
    • Financial Services Raise share of wallet and LTV

    Heap For Teams

    • Product Teams Optimize product activation, conversion and retention
    • Marketing Teams Optimize acquisition performance and costs
    • Data Teams Optimize behavioral data without code
  • Pricing
  • Support

    • Heap University Video Tutorials
    • Help Center How to use Heap
    • Heap Plays Tactical how-to guides
    • Professional Services

    Resources

    • Down the Funnel Our complete blog and content library
    • Webinars & Events Events and webinar recordings
    • Press News from and about Heap
    • Careers Join us

    Ecosystem

    • Customer Community Join the conversation
    • Partners Technology and Solutions Partners
    • Developers
    • Customers Stories from over 9,000 successful companies
  • Free TrialRequest Demo
  • Log In
  • Free Trial
  • Request Demo
  • Log In

All Blogs

Data-Informed

Quantitative vs Qualitative Tools. When should you use each?

John Cook
July 20, 20235 min read
  • Facebook
  • Twitter
  • LinkedIn

Being a product manager means having a diverse toolkit. That’s a good thing: different situations call for different types of analysis. Often at the same time.

On the other hand, having too many tools can lead to a kind of paralysis. Which tool to use when? Which approach best fits the scenario you’re in?

Knowing which analytics strategy to adopt is a kind of art. But it’s an art that can definitely be learned. We’re here to help!

So which should you use - quantitative tools (like product analytics), or qualitative tools (like session replay or heatmaps)? Let’s find out.

First things first: focus on Product-Market-Fit

A key outcome for every product team is Product-Market Fit (PMF). As you’re likely aware, PMF can be a fickle, moving target. The point where a product satisfies real customer needs and gets them willing to pay can change as the product evolves. The sweet spot is constantly shifting.

In general, teams have two research methods that help find PMF: the qualitative and the quantitative. 

Qualitative product research involves talking with customers and listening to their stories and experiences. It focuses on the needs of a few users, which it takes as representing the whole of the user base. Qualitative research helps us understand what customers really want in a deep and meaningful way.

Quantitative product research, on the other hand, is all about using data to measure and analyze customer behaviors and preferences. It’s great for analyzing trends and seeing what different groups of users do. Unlike qualitative research, quantitative tools give us a more objective picture of how well a product is doing.

Qualitative and Quantitative Research Frameworks

The challenge is knowing when to use each method and how to combine their insights to find Product-Market Fit. In the past, this was tough, but now we have plenty of thought leadership and tools to help product managers. 

We recently interviewed Dan Olsen, a product management expert, to learn more about these approaches.

Check out the full interview between Dan Olsen and Heap CEO, Ken Fine.

Watch On-Demand

According to Dan, “There are two fundamental ways to do research to minimize that uncertainty and learn.” Dan describes qualitative tools as the 'Oprah' approach, and quantitative tools as the 'Spock' approach.

Oprah Spock Quant Qual Image

Image courtesy of Dan Olsen

Qualitative - The Oprah Approach

In Dan’s words:

“I want to describe at a high level what each of these is really all about. So qualitative research or learning is when you actually care about what each person has to say. You want to sit down one-on-one with them and talk to them and have an interactive conversation with them to understand the nuances of what their problems are, what they care about, and what's important to them.

You're not going to hit statistical significance when you're doing qualitative research, even if you talk to 10 or 20 or 30 people. 

If you think about someone from television who's really good at sitting down with someone one-on-one and staring into their eyes and interviewing them and getting them to open up, it's Oprah Winfrey. There are people that have interviewed with her and cried. There are people who say it’s like she was staring into their souls. And they cried in a good way because she got them to open up.

So that's kind of the qualitative approach, personified by Oprah. Each user is a snowflake! Of course, we want a pattern match across multiple users. But with qualitative research, you want that rich nuance of data.”

Quantitative - The Spock Approach

Dan continues: 

“In contrast, for quantitative research, you don't care about any one user. You want thousands of data points, or hundreds of thousands, or millions! The point isn’t to be crass - it’s that with quantitative research your goal is to reach statistical significance.

It's are very different mindset. In this case, the fictional character from tv to think about is the master of logic and analysis: Spock. Spock wants data. He wants a lot of data! Spock’s goal is to prove things. He’ll use math and analytics to get to the answer.”

The key for Product Managers is to know when to use each approach and how to combine the insights to drive key aspects of the product life cycle. 

Choosing the right mix, for the right product phase

Using both qualitative and quantitative research methods throughout the product life cycle has big benefits for finding Product-Market fit. Before hitting the market, qualitative research digs deep into customer pain points and desires through interviews and surveys. This helps shape the product to match what customers really want. 

After launching, quantitative research takes over, crunching numbers to track user engagement, retention, and conversions. This data-driven approach guides product managers in making smart decisions to keep the product successful and competitive.

Combining these two methods gives a complete picture and leads to better products and long-term success.

Choosing the right platform for your research needs

As we circle back to the key question a product manager has to answer on their journey to Product-Market fit, we turn to Dan again.

“If we broaden our perspective, Christen Rohrer, a UX leader, presents an excellent framework that combines quantitative and qualitative aspects with the additional dimensions of attitudinal and behavioral factors.

The difference between the two can be demonstrated in this example. If I ask you how often you go to the gym you might say “once a week” (attitudinal), but if I observe your behavior I’ll learn it’s more like “once a month” (behavioral). 

image8

Low left corner: This is strictly qualitative, one on one interviews, Voice of the Customer (VOC) feedback - sharing their opinion directly. 

Upper left: This is still qualitative but it is behavioral. Here you are observing people one at a time, and listening to what they say. This is where pure usability testing and session replay would live. The reality is any time you’re going to do usability testing - you’re going to get both attitudinal and behavioral elements.

Lower right: We switch to the quantitative realm. Now we’re not talking to each user one on one like Oprah - we are in the “Attitudinal” space. This means employing a way of getting many customers' opinions at scale. This is where surveys live.

For example, NPS scores are a good example for this. There’s a reason it’s so popular because there’s one question - with a scale.  “How likely are you to recommend this product to a friend?” and then you can pick values between 1-10. I’ve seen how surveys can be misused in the past, but NPS score helps clear the noise. But then you want to get the WHY - why did you give us a 10 or 5?

Upper right: Still quantitative, this time behavioral. Here you are collecting data at scale and this is where analytics, A/B testing, and experimentation live. You’re looking at real product usage data, at scale, and derive statistically significant insights.”

This is an effective way to break down the different tools your team needs to carry out the different types of research, across different product phases. In a phase, you would need to lean in on different methodologies and techniques to find PMF and validate your hypothesis.

Recap

As you explore the various research methodologies needed for each phase of your product development, remember that the real power lies in combining both quantitative and qualitative data.

When seeking tools and platforms to support your team's agile research, opt for those that seamlessly integrate both approaches. By doing so, you'll gain a more comprehensive understanding of what's happening and why, allowing you to uncover valuable insights that pave the way for faster and better product development.

John Cook, Sr. Product Marketing Manager

Was this helpful?
PreviousNext

Related Stories

See All

  • Creative visualization of AI CoPilot capability
    article

    Heap announces new generative AI CoPilot

    Heap, the leader in product analytics, unveils AI CoPilot’s open beta today.

  • Heap.io
    article

    What’s Next in Experience Analytics?

    What does the future of analytics hold, and what does it mean for you?

  • Heap.io
    article

    Building a Retention Strategy, Part 2: Connecting Activities to Revenue with a Metrics Tree

    If you read one post from this series, it should be this one.

Better insights. Faster.

Request Demo
  • Platform
  • Capture
  • Enrichment
  • Integrations
  • Governance
  • Security & Privacy
  • Infrastructure
  • Heap Illuminate
  • Segments
  • Charts
  • Dashboards
  • Playbooks
  • Use Cases
  • Funnel Optimization
  • Product Adoption
  • User Behavior
  • Product Led Growth
  • Customer 360
  • SaaS
  • Retail and eComm
  • Financial Services
  • Why Heap
  • Why Product Analytics
  • How Heap Works
  • How Heap Compares
  • ROI Calculator
  • The Future of Insights
  • Resources
  • Blog
  • Content Library
  • Events
  • Topics
  • Heap University
  • Community
  • Professional Services
  • Company
  • About
  • Partners
  • Press
  • Careers
  • Customers
  • DEI
  • Support
  • Request Demo
  • Help Center
  • Contact Us
  • Pricing
  • Social
    • Twitter
    • Facebook
    • LinkedIn
    • YouTube

© 2025 Heap Inc. All Rights Reserved.

  • Legal
  • Privacy Policy
  • Status
  • Trust