Beyond The Numbers: Why Qualitative Research Has a Place In CRO 

Introduction 

Since its inception, Conversion Rate Optimisation (CRO) has been synonymous with number crunching. We’ve all been meticulously deciphering metrics like click-through rates, bounce rates, and conversion rates to provide us with digital clues to make informed business decisions. The data doesn’t lie, right?  Whilst these analyses provide crucial insight into what customers are doing, we could be missing a vital piece of this puzzle, the why.  This is where qualitative research comes alive.

By understanding users’ motivations, pain points and the emotions behind their behaviours, CRO teams can craft better-informed strategies and truly optimise the holistic user experience.  In this article, we will explore how qualitative insights can turbo-boost your CRO programmes and how to successfully integrate them into your optimisation ambitions. 

Why the data isn’t always enough 

Qualitative research shines where quantitative analyses struggle.  Decoding the Why that is driving the What allows CRO teams to better optimise the customer experience. In essence, it is better to hear a consumer’s voice, rather than look at their footprints. 

An example to bring this to life might be found when analysis shows checkout abandonment rates:   

• The What: a large proportion of sessions are abandoning checkout on the payment page 

• The Why: it could be assumed that customers are expecting more payment options, and as such may introduce alternative payment providers.  However, customer support transcripts show that people are actually concerned about the security of their credit card information 

The numbers can’t tell you everything

Quantitative data is undeniably powerful.  It provides concrete, actionable and measurable insights into customers’ actions.  Understanding the patterns and trends highlights areas that require attention.  However, relying solely on these findings can provide a skewed view of reality – like using a map without a compass.  Using data, we know how many people abandon their carts, but we can’t explain why. Did they face technical issues, did they not trust the payment system, or did someone simply knock on their door?  Without a qualitative lens, we’re somewhat left guessing.   

Missing the nuances that matter 

A narrow focus on key metrics can leave us blind to unexpected opportunities or conceal underlying issues that research would readily unearth. Product page heatmaps can show us where people are clicking, but they won’t tell us directly if the layout is confusing, or if visitors were engaged with the content.  Interviews, however, let us dive deeply into a customer’s experience, focusing on what they really care about 

Uncovering the hidden opportunities 

Sometimes the most valuable learnings come from the unexpected.  We’ve heard product teams complaining that users are avoiding their latest feature but speaking with visitors we learn that they’re unclear on how it benefits them, or they might not even know it exists – these are all findings that numbers alone would struggle to reveal. 

The qualitative research toolkit 

Various methods can be integrated into your CRO strategy.  Here are some of the most effective methods for gathering CRO insights. 

User interviews 

One-on-one interviews are one of the most powerful ways to understand behaviour.  An in-depth conversation facilitated by an expert qual researcher allows for a comprehensive exploration of a user’s experiences, motivations, and pain points.  Guiding participants with non-leading, open-ended questions provides rich and nuanced insights that can inform optimisation decisions. 

On a slight tangent, an interesting study by the Wall Street Journal examines leading questions and their impact on responses.  One example that best highlights this: 

• “Do you think the United States should allow public speeches against democracy?” 21% agree 

• “Do you think the United States should forbid public speeches against democracy?” 39% agree 

Usability testing 

Observing customer in their natural habitat provides a unique and unbiased view of how they interact with your prototype or site. It allows us to see how people are actually using the website without skewing it by guiding people on how it was intended to be used.  Are they struggling to find what field is presenting an error preventing them from continuing, or is certain information causing them to hesitate?  Using this method can help to design more intuitive flows that align with users’ expectations and not the expectations of the business. 

Card sorting 

When it comes to information architecture, card sorting is an invaluable tool.  It helps us to understand how people instinctively categorise and organise content. By asking users to group similar items together and label the categories, we can design more intuitive design systems that align with customers’ mental models.  

Surveys 

Surveys give customers a voice and can act as a bridge between qualitative and quantitative insights.  Combining questions where users rate responses on a scale with open-ended enquiries can provide a wealth of information.  They also have the added benefit of being able to target a much larger sample size and can be comparatively more cost-effective than other methods discussed. 

Integrating Qualitative Research in a CRO Programme 

The most effective CRO strategies are those which are built on the foundation of both qualitative and quantitative research.  But you might be wondering how to integrate both in your processes, bringing the best of these worlds together. 

Identifying opportunities and formulating hypotheses 

We have explored how these methods can more effectively identify hidden opportunities on the website.  Baking these findings into the ideation process provides a more holistic view of the pain points. They also allow for the development of more target, more targeted hypotheses.  For example, if customer interviews reveal that the biggest blocker to adding a product to their bag is, in fact, an overly complex pricing structure the obvious AB test would be to explore simpler or more transparent pricing models. 

Validating findings 

Validating findings works both ways. Analytics can help size opportunities unearthed in interviews, ensuring we don’t fall into creating solutions for problems faced by only one person. 

On the flip side, qualitative research can help us understand the context behind the findings. After releasing a new landing page, we saw a drop in sales, but was it the messaging or design? 

Follow-up research post-testing 

Sometimes experiments produce unexpected results.  A series of tests tackling a well-established issue have so far failed to fix the problem. This is exactly where we need to dig deeper.  Maybe the assumed reason behind a bizarre behaviour is completely misguided.  A deep understanding of the nuances behind behaviours will allow us to make better-informed conclusions and recommendations. 

Some practical tips for conducting qual research 

• Formulate a clear plan defining objectives, target audience and methods 

• Recruit a diverse and representative sample of participants from a variety of backgrounds and demographics.  Lean on personas to get a well-rounded understanding of behaviour by segment 

• Create a comfortable, unbiased environment where participants can express their honest opinions, free from judgement 

• Ask open questions and listen! Probing deeper will help uncover underlying motivations or emotions 

• Analyse all findings systematically and identify key themes.  Looking for reoccurring themes and patterns will help identify insights and support data-driven decisions. 

So, what is better: Qualitative Research or Quant Research? 

In short if you can only do one, do quantitative – it’s the best truth you have. But CRO is advancing beyond the times when qualitative data is a nice-to-have and is becoming an indispensable component of a high-performing optimisation strategy.  By understanding the why behind a behaviour, we can make better-informed decisions and recommendations, and ultimately craft superior user experiences.  Integrating research into your CRO strategy is not just the smart move; it’s the key to unlocking a deeper level of understanding which will drive greater success. 

If you are finding it hard to progress with just quantitative, want to transform experiences or just want to leverage every tool available for success; using qualitative and quantitative together will power your ideation, create better tests that are more likely to win and ultimately turn your CRO team into a Customer CRO team who truly understand what the customer is trying to do and how to help them do that more efficiently and effectively 

If you would like to find out more, please get in touch with Station10 for a consultation….