How Do Data Privacy Laws Impact A/B Testing for eCommerce Brands

A/B testing is the backbone of optimizing any eCommerce business – and data privacy regulations can feel like something standing in the way, especially for global eCommerce brands operating across regions.

Fail to comply? There’s a risk of getting huge fines, negative publicity, and consumer distrust. However, challenges such as privacy laws and directives can help businesses rethink how they interact with customers and create experiences that people genuinely want.

To help you understand and seize these opportunities, we’ll cover:

• How data regulations impact A/B testing and personalization.
• An overview of the main data privacy laws you should be aware of with links to resources.
• A cheat sheet to audit your experimentation practices for compliance.

Data regulations aim to protect the privacy and personal data of consumers, which is undeniably good. They can also make it more difficult for businesses to gather and use data for A/B testing and personalization. However, they don’t have to.

Here’s what you need to know about how data regulations are impacting A/B testing and personalization for eCommerce brands.

1. Most A/B tests do not require Personally Identifiable Information (PII)

While you’ll need to check with your specific A/B testing tool, most do not collect PII, as defined by GDPR, in their standard setup. And while you can choose to use PII data in personalization activities, it tends to feel unsettling for users.

Instead, personalization can use non-PII data such as behavioral, contextual, and trend information to improve experiences – how does the user navigate the website, where do they stop, what do they click. This form of ‘hot’ data allows you to customize experiences based on real-time website behavior while protecting user anonymity. For example, Kameleoon uses hot data and AI to predict the conversion intent of new visitors within 15 seconds of their arrival on a website.

There are different aspects of the experimentation process, and for some PII might be necessary, while for others it’s not. If you’re conducting user research, you’ll likely need contact information to arrange in-person moderated research but might not need to collect any for an exit poll.

You can also use a session recording tool like Mouseflow to conduct much deeper user research, and still avoid collecting personally identifiable data. Mouseflow allows you to comply with all the data privacy laws by removing PII from session recordings thanks to the Visual Privacy Tool so that nobody accidentally sees something personal while watching the recording.

2. Regulations are impacting test results and analysis.

From the standpoint of a person who’s running an A/B test, the problem with different data privacy laws is that they impact the ability to trust A/B test results. A range of cookie-related policies enacted by browser giants, Safari, Google, and others means that some third-party cookies can no longer be used or can only be used for a short time.

Returning users may be bucketed incorrectly into different test variations, polluting your test data. However, there are ways to overcome this problem, such as cross-domain local storage solutions which allow you to use cookies over an extended period, or use a server-side snippet, eliminating the need for cookies entirely. Just keep that in mind when planning out your A/B tests.

3. Build a culture that encourages experimentation AND the responsible use of data

Building a culture that encourages experimentation and the responsible use of data is vital. The danger for more traditional eCommerce players is too much caution, which stifles growth. To innovate and out-pace the competition, however, experimentation is essential – and it must be done in a way that respects the privacy and personal data of customers.

To achieve this, businesses need to prioritize transparency, gain explicit consent, ensure data security, and guarantee that data is used only for intended purposes.

By fostering a culture that values responsible experimentation and data use, businesses can create a space where employees feel encouraged to take calculated risks, collaborate, and challenge the status quo. This can lead to breakthroughs in innovation and a more engaged workforce that is committed to upholding ethical principles in data-driven decision-making.

Below is a jumping-off point to understand what data regulations and laws exist and where to find more information.

1. General Data Protection Regulation (GDPR)

General Data Protection Regulation (GDPR) governs the collection, use, storage, and sharing of personal data of individuals in the EU. The regulation applies to all organizations, regardless of their location, that process or store personal data of individuals residing in the EU, making it one of the most comprehensive data protection laws in the world.

If you process data or offer goods or services to EU citizens or residents, then GDPR applies to your business regardless of where you are.

Read our FAQ about GDPR.

2. California Privacy Rights Act (CPRA)

CPRA is an update to the former CCPA regulation. It applies to businesses that:

1. Sell products or services in California with over $25 million annual revenue.
2. Buy, sell, or share personal information of 100,000+ California residents or households.
3. Or derive 50% or more of their annual revenue from selling or sharing California residents’ personal information.

Read our FAQ about CCPA.

Similar criteria exist for businesses operating in other states under laws such as the Connecticut Data Privacy Act (CTDPA), Colorado Privacy Act (CPA), and Virginia Consumer Data Protection Act (CDPA). However, there are differences in these laws, this is not an exhaustive list, and more states are following suit. So consult a legal representative to understand what rules apply to your business.

3. Consumer Privacy Protection Act (CPPA) Canada

CPPA is set to replace the Personal Information Protection and Electronic Documents Act (PIPEDA), a federal privacy law that governs the collection, use, and disclosure of personal information by private sector organizations. However, it is still being debated by lawmakers.

Find out more about CPPA.

4. Personal Information Protection Law (PIPL) China

PIPL has an extraterritorial scope. It applies to businesses that process personal information within China or are located outside of China but process the personal information of Chinese residents.

Find out more about PIPL.

5. Brazilian General Data Protection Law (LGPD)

LGPD has an extraterritorial scope and applies to businesses that process data within Brazil, process data about individuals from Brazil, or process data collected in Brazil.

Find out more about LGPD.

Of course, there are more, such as South Africa’s POPIA, and other countries’ data privacy laws – and the scope of those you need to comply with depends on where your business is operating.

6. Cookie regulations

The ePrivacy Directive covers the usage of cookies, email marketing, and other aspects of data privacy.

Meanwhile, private companies have implemented their own policies on cookies. Apple’s browser Safari introduced ITP in 2017, limiting website tracking to either seven days or 24 hours, depending on the cookie type. Mozilla Firefox implemented “Total Cookie Protection” in 2022, and Google Chrome plans to implement a similar cookie policy in 2024.

It’s time to evaluate your level of compliance and identify areas that need to change. Here’s a cheat sheet for how to audit your experimentation process:

1. Identify the laws and regulations which apply to your eCommerce business. Depending on your business’s location and where it operates, you may be subject to various laws and regulations governing data privacy, consumer protection, online marketing, taxation, and intellectual property. Seek external legal expertise if you do not have the resources to identify this in-house.
2. Create a list of tools you use. This includes A/B testing, analytics, product recommendation engines, and user research tools. Document how they collect data, what is collected, and where it’s stored. Most vendors will have a dedicated page on their website covering privacy and security that you can use for reference (for example, here’s Kameleoon’s privacy and security page and here’s Mouseflow’s Legal Hub).
3. Review internal processes or standard operating procedures for experimentation and identify areas where data is used. By reviewing these processes, you can identify areas where data may be at risk of being mishandled or misused and take corrective action to mitigate these risks. This may involve improving data security, implementing data anonymization practices, ensuring that consent is obtained for data collection and use, and providing training for employees involved in data handling.
4. Draw up a plan of the actions you need to take to ensure you are compliant with all applicable laws. The plan should outline the specific actions that need to be taken, such as implementing new policies and procedures, providing employee training, and improving data security measures. It should also identify responsible parties for each action and set timelines for completion. Professional data security audits such as SOC 2 or international ISO 27001 certification could also be considered.

Remember, this isn’t a one-and-done process. As the regulatory landscape constantly evolves, you must keep up to date with the changing laws and re-audit your processes and tools periodically.

With the increased focus on data privacy and protection, eCommerce brands must comply with various laws and regulations governing data collection, storage, and use. Failure to comply can result in hefty fines and damage to your brand reputation.

To ensure compliance, eCommerce brands must audit their A/B testing processes, review internal processes and standard operating procedures, and draw up a plan of action to ensure that they are compliant with all applicable laws.

By doing so, eCommerce brands can not only avoid legal and financial consequences but also build trust with their customers, demonstrating a commitment to protecting their privacy and data.