🎙 On the go? Listen to the article through our podcast, Cave Bits!
Listen to more episodes of Cave Bits
For every conversion rate optimization (CRO) professional, there’s a moment of triumph when the organization they’re working with recognizes the value of experimentation and decides that they need a proper experimentation program of their own. Yay!
If you’ve just celebrated this moment, congrats!
Yet, this is only the beginning. Now it’s up to you as a CRO professional to deliver valuable learnings and exceptional results – and prove that investing in experimentation (and you) was the right decision for the organization to make.
There are so many things for you to test! So, where do you start to maximize the impact? To answer that, you need a CRO roadmap – and a good one.
What is a CRO Roadmap
A CRO roadmap is a document that lists experiments that you plan to run within a certain timeframe, with priorities, hypotheses, details, and expected outcomes defined for each experiment.
Here’s what makes a good CRO testing roadmap:
- It’s agreed upon by all stakeholders and is aligned with the business goals.
- The experiments described in it rely on data-informed hypotheses and are well documented.
- It prioritizes the experiments consistently, based on evidence.
Why Do You Need a CRO Roadmap
-
Test Prioritization and Strategy
There’s an infinite number of things you could test right now. To make sure that at each moment you’re testing that one thing that is likely to have the most impact, you need to prioritize.
And after that test, you also need to not jump on a random testing idea that you got from it or from stakeholders, but strategically move to the next most promising test on the list.
Every CRO expert says that CRO is a long strategic game – there is no place for spontaneous decisions in it. Having (and sticking to) a conversion rate optimization roadmap helps adhere to the chosen strategy.
This is one of the tests that Derek Rose, a premium nightwear brand, ran. Before they ran it, they collected and analyzed a lot of customer data using Mouseflow session replay. They prioritized it because they had a strong hypothesis backed with data.
Read about the other changes Derek Rose made that increased conversions in the eCommerece store by 37%.
-
Focus and Consistency
It’s not uncommon for the very same stakeholders who supported having the experimentation program to occasionally undermine your efforts by introducing their own ideas, disrupting your planned process.
And that’s another reason why you need a CRO roadmap. If there’s a document that all parties have agreed on and that is aligned with the business goals, it’s much harder for the management to hijack your plans with this brilliant idea that they’ve just had and that they want you to test as soon as possible.
-
Planning and Resource Allocation
It would’ve been much simpler if it was only you who’s involved in running CRO experiments, but it’s never like that.
CRO always involves many parties such as analytics, design, and development. More often than not, there are also marketing people involved. In a way, they all are a part of the CRO team – and need to be on the same page about who’s doing what and when. And of course, there are other stakeholders that agreed to having an experimentation program because they have expectations. That can include people from sales, product, and even the CEO himself.
To make sure all these people dedicate their time to executing their part of the experiment when it’s time to do so, you need to plan together with them in advance. Having a document to which all parties involved in experimentation can refer when planning their time helps a lot.
How to Create an Experimentation Roadmap in 10 Steps
1. Know the Business Goals
Before coming up with any ideas, make sure you know the business goals that you’re trying to achieve. If a conversion rate optimization roadmap is not aligned with the business goals, it’s doomed to fail.
If you’re the business owner – likely, you have the goals in mind. If you’re an employee or a consultant helping the business, you absolutely have to be 100% aligned with the people running the business on what they’re trying to achieve.
Have the business goals in writing, on the first page of your CRO roadmap.
2. Jot Down the Strategy
Once the business goals are clear, it’s time for you to figure out the strategic goals: what would you be trying to implement to achieve the business goals?
For example, if the business goal is increasing average order value (note: it’s not about higher conversions!), there are different ways to approach it:
- Upsell the website visitors
- Highlight more expensive goods
- Adjust pricing
Or maybe you could use a combination of these. That’s your CRO strategy that should go next to the business goals to make sure your roadmap follows it. It doesn’t need to be detailed at this stage – but it should outline the areas you’ll be focusing on. You’re doing it to align the CRO goals with the business goals.
3. Set Up the Tooling
To run experiments, you’ll need a toolset, and it goes beyond simply having an A/B testing tool (even though it’s also a must for effective CRO).
You’ll also need tools that collect the customer data that helps you come up with hypotheses and understand if your experiments are actually successful or not. There are different data sources, and ideally you’ll need a tool for each:
- Quantitative analytics: Use tools like Google Analytics or Adobe Analytics to analyze standard metrics such as conversion rates, bounce rates, click-through rates, and so on.
- Behavior analytics: To add depth to this data, use a heatmap tool and session replay tool like Mouseflow. These tools provide visual insights into how users interact with your website, helping you identify areas for improvement.
- User feedback: Asking for your users’ opinions directly via surveys and feedback forms and reading their reviews on 3rd party websites helps to gain a deeper understanding of their needs and pain points. For that, you’ll need a user feedback tool – and, well, some time to analyze it and go through the reviews.
- User testing: You can take a step further and conduct interviews and usability tests to better understand how users navigate your site. That helps discover areas where they struggle and evaluate the effectiveness of potential changes.
Then, depending on what and where you’re going to test, you may need other tools like landing page builders, email automation tools, etc. We have a curated list of those in our article about the best CRO tools.
4. Run a CRO Audit
Running a CRO audit of your website and campaigns helps you achieve two very important things:
- It gives you a baseline – the starting points to which you can compare your CRO results when you run tests.
- It gives you ideas and hypotheses. As you analyze different parts of your website, the metrics, the user behavior, and so on, you’ll see a lot of opportunities to improve something.
We’ve prepared a step-by-step guide for doing a website CRO audit for you.
5. Write Down Hypotheses You Get From the Audit
More often than not, A/B testing hypotheses stem from thin air. Sometimes they emerge from your experience and your perceptions of best practices. Other times, they may arise from directives handed down by your boss or your boss’s boss.
But that’s why you’ve been running a CRO audit – to get data-driven hypotheses that are much more likely to win than random ones.
Here’s the formula for the perfect conversion rate optimization hypothesis compiled by one of the industry’s best experts:
If we [change this thing], it will [have this impact on the metric] because [your theory as to why that would work].
To put it simply, to formulate a strong A/B testing hypothesis, you need to focus on the “why” behind the potential changes.
Check out this post on creating data-driven CRO hypotheses to get familiar with the framework.
6. Support Them with Additional Data
Usually, one source of data is enough to come up with an idea. But to make it into a strong hypothesis that is more likely to win, you need additional evidence.
Your most valuable tools in this case are heatmaps and session recordings, or user feedback, or user testing — ideally, all of them. But also, don’t forget other sources of evidence, such as customer support tickets, customer reviews, etc.
“You can take your hypotheses to another level by being far more descriptive about why you believe the change would have that impact.”
If you want an example, check out this case study about Cotopaxi, a travel gear and apparel brand, that builds their data-informed website experimentation roadmap using Mouseflow’s feedback, heatmaps, and session replay.
7. Get Stakeholder Buy-in
Somewhere around this stage, it’s also important to make sure you have stakeholder buy-in to run the tests that you want to run.
Stakeholders usually have their own reasons and opinions (CRO experts even have a name for some of those – HiPPO, which stands for “Highest Paid Person’s Opinion”). They may think that some pages are too sensitive to run experiments, or maybe there’s an update coming soon that will completely change something – and you just didn’t know about it. Or they may want you to test something completely different, based on their ideas.
Getting approvals can be tedious, and people often try to minimize the amount of them by only getting the final approvals. That’s a mistake though. If they don’t get them, all the work they’ve done would be in vain. So, even though it may seem counterintuitive, it’s better to seek stakeholder approval as early as possible.
Before jumping from theory to action, make sure you have support from all stakeholders and all the necessary resources such as time from developers, designers, marketers, and other people who are going to be involved in this one way or the other.
Solid data, especially when presented visually through tools like heatmaps or session replays, holds persuasive power with stakeholders. So, it’s important to collect it before trying to get their approval.
“If we have data, let’s look at data. If all we have are opinions, let’s go with mine.”
The HiPPO Problem
HiPPO is an acronym for “Highest Paid Person’s Opinion”. The term was coined because there’s an actual problem behind it that a lot of CRO professionals face: there’s a tendency in organizations to lean heavily on the viewpoints of highest-ranking individuals.
This prioritizes subjective judgments over objective analysis or valuable insights derived from data. And the problem goes deeper than running experiments that are less likely to win. HiPPO-driven decision-making leaves lower-ranking employees frustrated, disempowered, and demotivated.
DALL-E’s take on how HiPPOs look
In the absence of sufficient data, the HiPPO wins, every time. The most effective approach to address HiPPO behavior is to prepare the necessary data beforehand. It’s hard to go against hard data. So, data – especially if it comes from multiple sources – is the best defense against HiPPOs.
And that is yet another reason to have a data-informed CRO experimentation roadmap.
8. Document the Experiments
Prior to this stage, you’ve been investing your time in research. Now, you’re moving on to acting on it.
Every hypothesis that you came up with is just a hypothesis. To become an experiment, it needs much more detail:
- what exactly are you going to change for the experiment,
- what metrics will you track,
- where the experiment is going to take place,
- what audience segment are you planning on running it,
- what are the expected outcomes,
- how long should it take and how much traffic it requires for statistical significance,
- what is the minimum sample size,
- what are the necessary resources,
- who are the stakeholders,
- and more.
We describe the process of documenting experiments in our post about how to run CRO tests the right way.
9. Prioritize the Experiments
Now that you have everything documented, you are one step away from having a ready and actionable CRO roadmap.
To get from a list of experiments to a testing roadmap, you need a system to prioritize the tests. How do you do it? Obviously, you want to prioritize the ones with the highest impact. But also, you likely would want to take into account the amount of resources that goes into each experiment, the confidence you have in it, and the amount of people the experiment influences – its reach.
There are a bunch of different models used for scoring – ICE score (impact, confidence, ease), RICE score (reach, impact, confidence, ease), and others. We cover some of them in our conversion rate optimization guide.
10. Plan Your Tests and Build an Experimentation Calendar
At this point, you’re almost good to go. The finishing touches are scheduling your experiments and building an experimentation calendar so that it’s easy for you and everyone else to understand what experiment is running now and what’s next in the queue.
The golden rule here is to run one experiment at a time, even if you have resources to build more at once. Following this rule is important because that’s the only way you’ll be able to clearly see the impact each experiment will have. If you’re running two tests on the same page at the same time, you don’t know which one contributed to the success or failure.
Done with that? Congrats, your experimentation roadmap is ready! Use it to guide your experimentation efforts and to shield from random assignments and experiment ideas people may want you to try.
Start to Execute and Analyze
Each experiment will bring 10x the value if you properly analyze the results.
Analyzing the results is much more than saying “this experiment is a winner, let’s implement the changes.” There’s a million and one actionable insights waiting for you if you spend time digging into the results, segmenting them in various ways, and looking for patterns.
Post-test-segmentation helps you understand things like
- how different variations performed across various devices,
- which user segments were positively impacted by the change,
- how the CRO test affected behavior of different user segments,
- why did the winning variation win,
- etc.
This understanding is a perfect source for new hypotheses and also for pre-test division – choosing which audience segment to run the next experiments on.
Using post-test segmentation to drive pre-test division was the ultimate weapon that allowed Matt Scaysbrook, founder and ex-director of optimization at WeTeachCRO, to turn a 10k win into a 250k win.
As usual, when analyzing the results, focus on the “why.” The very same behavior analytics tools – heatmaps and session replay – can help you understand not only if your hypothesis was right or wrong, but also why it was right or wrong.
Conclusion
Having a CRO roadmap solves multiple problems – from knowing what to focus on at each moment to protecting your CRO program from unexpected requests that always occur. It also helps maintain a steady test velocity, so that you are always prepared and don’t have huge pauses between the tests.
Backing up each hypothesis in the testing roadmap with data and properly documenting every test makes the roadmap much stronger and better at both driving revenue and maintaining focus.
