Using Quantitative and Qualitative Research in UX Design

UX designers often get stuck focusing on the wrong things.

They focus on details like the latest Figma techniques without understanding the bigger picture.

This poses a challenge to businesses and stakeholders. Because at the end of the day, they have targets to achieve.

They need team members (designers included) who know this and deliver output to make this happen.

And that starts with a UX research strategy.

An important step is uncovering the insights from two types of research: quantitative (which tells us "what" people do) and qualitative (which explains "why" they do it).

This guide will explain each kind of research and how they are used in UX research.

On its own, however, it won't get anywhere. Its value lies in knowing how to apply it in the real world.

In our UX research course, learn the exact processes and tactics that Michael Wong (Mizko) used to help build a UX agency that generated over $6 million in revenue.

‍

What is quantitative research in UX design

Quantitative research focuses on gathering and evaluating numerical data. The goal is to identify trends, calculate averages, or find connections between data points.

For example, Google Analytics uses quantitative research to monitor the total number of site visits, inquiry counts, and bounce rates. With this data, you get insights into how users behave on your site and how it's performing.

‍

Quantitative research in UX: Knowing when and why use it

Quantitative research can be super helpful at different stages, mainly because it gives you solid numbers to work with.

Here's why it's important in your UX research:

‍

1. You'll spot trends or patterns

Before diving into the nitty-gritty details of your design, you might want to get a general sense of user behavior or preferences.

Example: For example, you could use surveys to determine what features users want the most or analyze website traffic to see where users spend their time.
‍

2. Test your concepts and assumptions

Once you have some design ideas or changes in mind, quantitative research can test if those ideas resonate with a wider audience.

Example:  This is where A/B testing comes in handy. You might create two versions of a page and see which one performs better in terms of clicks, engagement, or conversions.
‍

3. Validate or refine your designs

After implementing your designs, you’ll want to know if they match your desired results. It also helps pinpoint usability issues.

Example: Use Google Analytics or other analytics tools to track conversions and engagements to assess if your product meets your objectives.

‍

Types of quantitative research methods in UX design

Let's explore the common quantitative research methods in UX:

1. Usability testing (Benchmarking)

• What is it: This method measures the usability of a product by quantifying user performance on specific tasks. Sample metrics include task success rate, time to complete a task, and error rate.
• Best used for: A starting point for usability and tracking progress over time. It's useful for objectively evaluating how design changes affect usability.
• Tools: Zoom, Maze.co, and Optimal Workshop.
  ‍

2. Web or app analytics

• What is it: Software that tracks and analyzes how users interact with a website or app. You can see what features people use the most, where they spend their time, and where they drop off.
• Best used for: Understanding user behavior on a large scale. It's great for spotting trends and patterns in how users navigate and interact with your product.
• Tools: Google Analytics, PostHog, MixPanel, Amplitude, FullStory, and HotJar.
  ‍

3. Visual heatmaps

• What is it: Mouse heatmaps show where users click, move their mouse, or scroll on a page. Radar-like color coding is used to indicate areas of high and low activity.
• Best used for: Visualizing user engagement and interest on specific webpage or app screen sections. It helps identify which areas attract the most attention and which are ignored.
• Tools: FullStory and Hotjar.
  ‍

4. Funnel analysis

• What is it: This method tracks users' steps towards a specific goal within your app or website. This includes making a purchase or signing up. It shows where users drop off in the process.
• Best used for: Identifying stages in the user journey where people are leaving or getting stuck. This is crucial for optimizing conversion rates.
• Tools: Google Analysis, Mixpanel, and Amplitude.
  ‍

5. Cohort analysis

• What is it: Cohort analysis involves grouping users based on shared characteristics or behaviors, such as sign-up date and tracking their actions over time.
• Best used for: Understanding how different groups of users engage with your product over their lifecycle. It's useful for spotting long-term trends and the impact of changes or features.
• When to use: Measuring user retention, engagement, and the effectiveness of updates or new features.
• Considerations: You'll need to decide how to segment your users into cohorts and what metrics to track for each group.
• Tools: Google Analytics.
  ‍

4. A/B testing

• What is it: A type of testing that shows two variants of a web page, app screen, or feature to different segments of users. This is done to determine which one performs better against a predefined goal.
• Best used for: Making data-driven decisions on design changes, new features, or content strategies. It helps determine what works best for your users.
• Tools: Mixpanel and Amplitude.
  ‍

5. Fixed-question surveys or questionnaires

• What is it: Research methods that collect feedback directly from users through answers to specific questions.
• Best used for: Gathering data that can be analyzed statistically for patterns or trends.  For instance, respondents may be asked to rate satisfaction on a scale of 1 to 5 or gather demographic details to see broader trends.
• Tools: Typeform and Google Forms.

‍

Challenges of quantitative research in UX design

Here are the problems UX designers might face:

• Doesn't dig deep into complicated ideas: Quantitative research may not fully capture feelings and opinions. Numerical data alone can't express these complexities in detail.
• Overlooks the "why": It might overlook important reasons behind what users want and how they decide because it only looks at what they do, not why they do it.
• Limits on what people can say: Quantitative research has predetermined answer choices, so it might not capture every user's viewpoint. This can lead to participants choosing options that don't fully represent their thoughts.

‍

What is qualitative research in UX design

Qualitative research gathers non-numerical data to understand concepts, thoughts, or experiences. It provides depth and context to user behaviors, motivations, and emotions.

‍

Qualitative research in UX: When and why to use it

Qualitative UX research is all about understanding the stories, feelings, and thoughts behind user actions. You talk to users, watch how they interact with your design, or see their reactions to understand their experiences.

Here are the reasons why qualitative research is important in UX design:

1. To get inspired

Before you even start designing, talking to potential users can give you inspiration for what to build by understanding what they need or what problems they have. This can spark ideas for what to build.

Example: Before starting a new app for home gardeners, you talk to potential users and discover common pain points. They struggle with remembering watering schedules and managing pests organically. This feedback inspires the idea of an app that sends watering reminders and offers natural pest control tips.
‍

2. ​It gets to the heart of the problems

Hearing directly from users and understanding their experiences makes creating solutions that appeal to them easier. This includes understanding user needs, behaviors, and the context of their problems.

Example: Imagine you're developing a new fitness app. At this stage, you might interview users to understand their fitness routines, exercise motivations, and frustrations with current fitness apps. This can help you identify features your app should have, like personalized workout plans or motivation tracking.
‍

3. Improve something specific

Even after your design is out in the world, you’ll want to keep improving it. Talking to users can help you understand what’s working, what’s not, and why. It also builds industry insights and bridges knowledge gaps.

Example: When redesigning a public transportation app, holding focus groups could reveal that users want real-time updates about delays.
‍

Types of qualitative research methods in UX

1. User interviews

• What is it: One-on-one conversations with users to explore their experiences, needs, and challenges. Unlike structured surveys, these interviews allow for open-ended responses and follow-up questions.
• Best used for: Gaining deep insights into user behaviors, motivations, and attitudes. They can help you understand the nuances of user needs and the context of their interactions with a product.
• Tools: Calendly, Zoom, Google Sheets, and ScreenApp.
  ‍

2. User observations

• What is it: This method involves watching users interact with a product in their natural environment to see how they use it without interference.
• Best used for: Identifying usability issues and understanding user behavior in a real-world context. Observations can reveal how intuitive a product is and how users navigate it.
• Tools: FigJam and Miro.
  ‍

3. Usability testing

• What is it: A method where users are asked to complete tasks using the product while observers or UX resarchers note where they encounter problems or confusion.
• Best used for: Directly assessing the functionality and user-friendliness of a product. It helps pinpoint specific areas where users struggle, informing targeted improvements.
• Tools: Zoom, Maze.co, and Optimal Workshop.
  ‍

4. Open-ended surveys

• What is it: Unlike fixed-choice surveys, these surveys primarily use open-ended questions to allow respondents to express their thoughts and feelings in detail.
• Best used for: Gathering qualitative data on user opinions, experiences, and suggestions. They offer rich, narrative data that can uncover new insights or deepen understanding of known issues.
• Tools: Typeform and Google Docs.
  ‍

5. Focus groups

• What is it: A moderated discussion with users about their experiences, perceptions, and opinions regarding a product or service.
• Best used for: Exploring diverse viewpoints on a topic, generating new ideas, and understanding user attitudes.
• Tools: Calendly, Zoom, and FigJam.
  ‍

6. Diary studies

• What is it: A research method where users record their experiences, thoughts, and frustrations while using a product.
• Best used for: Understanding long-term user behavior, experiences, and patterns that emerge over time. Diary studies capture how feelings and usage may change in different contexts.
• Tools: Journey, Daylio, and Google Sheets.
  ‍

7. Card sorting

• What is it: An interactive method where users organize topics into categories that make sense to them.
• Best used for: Structuring or reorganizing content on a website or app. It reveals how users conceptualize different information groups.
• Tools: Dovetail and OptimalSort.
  ‍

Challenges of qualitative research in UX design

While qualitative research in UX design offers deep insights, it comes with its own set of challenges:

• Limited scope of participants: While qualitative research goes deep, it does so with fewer participants. This means you hear from a relatively small group while you get detailed insights from each person. This limits how much you can extrapolate findings to represent a larger population.
• Requires a lot of time, effort, and resources to analyze: Breaking qualitative data into actionable insights can take much time and effort. Unlike quantitative data, it's not as straightforward to visualize through charts or graphs. This makes it harder to grasp or present findings quickly.
• Risk of bias from researchers: The subjective nature of qualitative research means it's more open to bias from the researcher or participants. Reproducing the exact results can be tricky since every interview or focus group may unfold differently.
• Generalization issues: Because qualitative studies typically include small group participants, applying the findings to larger trends or behaviors among users is hard.
• Privacy concerns: People might not always feel comfortable sharing their thoughts in settings where their identity is known, especially on sensitive topics. This might make people hold back or change their answers, making the data less genuine and helpful.

‍

How quantitative and qualitative research work together in UX

Doing both qualitative and quantitative research methods in UX design is like having the best of both worlds.

Qualitative research digs into the "why" behind user actions, while quantitative research offers solid stats to back up your findings and spot trends.

Pairing these two methods gives you a full view of what's happening and why. Start with qualitative methods, like interviews, to get raw, in-depth feedback.

Next, conduct surveys or similar quantitative techniques to determine if those feelings or issues are common among your users.

Both research methods can sometimes uncover deeper insights that using one method alone might miss.

Let's take a look at a real-life case study from Spotify.

‍

Case study: How Spotify used the mixed-method approach

Problem: Qualitative data revealed a discrepancy compared to the quantitative numbers from an A/B test on using a new feature that lets you skip ads.

They noticed users were engaging with the feature in different ways. However, when they talked to users directly, they realized there was confusion about how it worked, which the numbers alone didn't explain.

What Spotify did: Spotify's Product Insights team, which consisted of a mix of User Researchers and Data Scientists, adopted a mixed-methods research strategy known as "simultaneous triangulation."

This approach involved:

• Clearly defining the research questions to focus the study effectively.
• Combining qualitative methods (user interviews and diary studies) with quantitative methods (A/B tests and data tracking) to gather insights.
• Using these methods together on the same group of users helps get a complete picture of how users behave and what they think.

How it helped: This integrated approach allowed Spotify to understand why there was a difference between user feedback and the data.

For example, they found that users they thought were "power skippers" were just confused about the rules for skipping ads. Only talking to users could reveal this, showing a full view of how people interacted with the feature.

Outcome: Using both user feedback and data, Spotify could fix misunderstandings.

They sent messages to users confused about whether there was a limit to skipping ads. They explained clearly that they could skip as many ads as they wanted. This clear communication messaging greatly improved how well the feature worked and doubled its feature success metrics.

By combining both research methods, Spotify solved the puzzle of mismatched insights and made better decisions for their product.

‍

Boost your skills in 10+ hours and impress your stakeholders

To gain the confidence and trust of stakeholders, designers often focus on making their designs look prettier and on-trend.

That's the old way of thinking. Companies nowadays seek designers who go beyond that and bring more results.

And it starts with understanding their users.

To truly meet their needs and solve their problems, you need to dive deep into their world.

Understanding how your work fits the larger business objectives is crucial to stand out.

Learn how to master UX research in just 10+ hours with Mizko's Practical UX Research & Strategy Course.

This course is your bridge to knowing your users and truly understanding them. By the end of the course, you'll be able to know what type of research methods to use in your projects to uncover crucial insights.

Here’s what makes it stand out:

• Hands-on learning: Apply what you learn through real-world challenges. Knowing UX research is one thing, knowing how to apply it.
• Efficiency: Forget about sifting through countless pages or websites. Get straight to the heart of effective UX research methods.
• Expert guidance: Learn from Mizko, an experienced designer who successfully transitioned to an agency owner and startup advisor. Get exclusive access using the resources and templates that helped him succeed.
• Flexible learning: Study at your own pace without the stress of a fixed schedule.

You’ll also learn how to:

• Draw valuable insights to inform your designs.
• Choose the right participants for meaningful feedback.
• Visualize research findings for easy understanding.
• Prioritize insights into actionable design tasks.
• Clearly communicate your findings and recommendations to stakeholders.

This course has already helped over 6,800 designers from top tech companies like Google, Meta, and Squarespace to integrate UX research into their daily work.

Here's what some of them have to say:

"I'm 100% more confident when talking to stakeholders about User Research & Strategy and the importance of why it needs to be included in the process. I also have gained such a beautiful new understanding of my users that greatly influences my designs." - UX/UI Designer Alyssa Durante

"The process in this course connects the dots and its easy to lead the clients through this process... + I can adapt the process based on the resources and the needs of the client. Life is now way easier to be honest because I have clear path to show to the clients even before we start the project because I can build expectations from the start." - UX/UI Designer Milosh Jakjimovski

"As I was struggling to find a high-level research framework for my day-to-day design work, I was able to fill many of the knowledge gaps because I took this course. It provides a well-structured research process that cuts out the "guesswork" that I'm adopting in my projects. Mizko covers his reasoning for every decision he took from start to end. Real-life examples and practical solutions throughout the course were incredibly helpful. Right now, I'm more confident than ever with my design decisions, presenting meaningful briefs to stakeholders, and so on." -  UX Designer Al Razi Siam

Go beyond learning Figma skills and deliver designs that drive business results.

‍