Creating an A/B Test

How to Create and Track an A/B Test?

A/B tests for websites involve splitting the audience and directing them to two different versions of the site to see which one generates more engagement.

Let's explore each step:

Creating an experiment
Traffic segmentation
Editing your variant
Creating events and collecting data
Funnel and results

Creating an Experiment

In the sidebar, click on "Experiments"

Experiments screen

This screen lists your experiments. Click on "Create new Experiment"

Some information will be required:

Name
Description
Traffic (percentage that will go to the new version)

Traffic Segmentation

You have just chosen the traffic percentage for your test, but you can customize this even further.

The creation of the Experiment has generated a new Segment, which can be checked in the sidebar, under "Segments":

Segments screen

Thus, you can also combine segments to conduct tests for specific audiences:

Select the created test and make the desired changes.

Examples of segmentation:

50% of Mobile traffic
30% of Facebook Campaign traffic
10% of traffic from Rio de Janeiro

Editing Your Variant

Now with the segment created, you can choose whether to test a complete page or a specific section.

Create a variant: Create variants

Select the segment: Select a segment

Make the desired changes: Variants

Creating Events and Collecting Data

Templates start with some events, but you can create new ones.

See how to create them through code or through the panel:

Example in code:

import { sendEvent } from "./sdk/analytics.tsx";

<button
    onClick={() => sendEvent({ name: "add_to_cart"}, params)}
>
    Add to Cart
</button>

Funnel and Results

Additionally, it is possible to check the results of the A/B Test

On the screen that lists the Experiments, select the desired Test.

Check out the features:

Sample Size

Indication of the minimum sample size for the test to be considered statistically significant.

Probability

See the probability of your variation being the winner relative to the other.

Goal Setting

Select the goals you want to compare, there is no limit.

Filter by Period

Filter by period to understand if there were any outliers during the test.

Funnel

See the conversion rate of each of your metrics to understand where you are losing your user.

Real-Time

Data is transferred in real-time, no more waiting a day or two to analyze them.