FAQ - A/B Test

Find answers to frequently asked questions and situations related to the setup and application of Guru's Neural A/B Test and analysis of related metrics.

On this page:

• Understanding the A/B Test

• Setup and Application

• Metrics Analysis

• Errors or Malfunctions

Understanding the A/B Test

In this section, you will find the main advantages of using Guru's Neural A/B Test and other important concepts to understand the best performance of the Test.

- What is the main advantage of using an A/B test?

The A/B Test analyzes real samples and measures objective impact. By allocating more traffic to winning variants, traffic ROI grows rapidly - faster than manual approaches, for example, which take time to identify what works.

Automated tools standardize data collection and interpretation analyses and produce ready-made reports. Furthermore, statistical controls enabled by A/B Tests help avoid false conclusions that informal tracking could generate.

- What is the advantage of using the Neural A/B Test instead of a common A/B Test?

Neural tests automatically reallocate traffic to better-performing variants, reducing the time users are exposed to weak variants and accelerating conversion gains. This is especially valuable in checkouts, where each visit has high value.

- Can the Neural A/B Test harm my traffic if I have few visits per day?

No, on the contrary. The Neural A/B Test is especially suitable for low traffic volume. The algorithm accelerates learning and quickly diverts traffic to the winning version, minimizing wasted visits on the losing version.

- What does “conversion score” mean?

It is a performance score calculated by the algorithm used by Guru. The score indicates which variant of the Test (A or B, for example) is generating more profit, more valid sales or leads, and based on this conversion it determines in real-time which one should receive the most traffic.

Setup and Application

In this section, you have access to some tips for better configuration and application of Guru's Neural A/B Test.

- How to decide whether to apply the Neural A/B Test to checkouts, leads, or sales?

To decide where to apply the A/B Test, you should evaluate your business's main need, identifying any problems.

• If you identify a high abandonment rate or financial impact, choose Checkout - optimizes conversions by directing traffic to the checkout version that demonstrates a higher sales conversion rate, maximizing profit and minimizing losses in the payment funnel;

• If the issue is contact capture, choose Leads - optimizes lead capture, directing traffic to the pages that perform best;

• If the issue is the offer, choose Sales - tests the effectiveness of a specific product and optimizes sales conversions, directing traffic to the offer pages with the best performance.

- Is there any advantage in creating a Neural A/B Test for checkouts, leads, and sales at the same time?

No, on the contrary. Technically, you can try to track all objectives (checkouts, leads, and sales) simultaneously, but this is not recommended as it negates the great advantage of the Neural A/B Test.

Guru offers separate test types for the neural A/B test (Checkout, Leads, Sales) because each is optimized for a specific metric. By mixing objectives, the algorithm loses the ability to effectively optimize traffic for the most critical metric at the moment.

You can choose which test ensures that Guru's algorithm focuses the power of neural optimization on the most important metric for you at that moment - for example, if your focus is the payment rate, choose the A/B checkout test.

- Can I change a checkout page already registered in a test? Does this affect the results?

It is not recommended to make changes to a checkout page already active in a test, as this action invalidates the experiment results.

The essential principle of A/B Testing is one variable to determine its influence on conversion. By modifying a checkout element during execution, the Neural A/B Test will lose accuracy, as the system will be trying to learn and optimize based on data from two variables tested simultaneously, making it impossible to identify whether any conversion gain or loss is a result of the original alteration tested or the new modification.

- Can I use the Neural A/B Test together with RPPCs?

You can use the A/B test together with an P.P.C.T., but it is important to know how to use them together. If your goal is to test a lead capture page, for example, you can create the A/B Leads Test and configure the other RPPCs, such as the Checkout P.P.C.T..

That is, if you are going to use an A/B Test, you do not need to create an P.P.C.T. for the same page structure, as the A/B Test is a type of tracking that will also split between several pages or checkouts.

Metrics Analysis

In this section, you will find tips for analyzing the metrics provided by Guru's Neural A/B Test, aiming for the best application for your business.

- How to analyze the statistics provided by Guru's Neural A/B Test?

The analysis of the statistics provided by Guru's Neural A/B Test is a strategic process that minimizes the need for manual calculations, as the statistical work is performed by the system, which intelligently directs traffic.

The secret to analysis lies in understanding the test score and using detailed metrics to validate your decisions. For this, you can analyze:

• The conversion score: the score calculated by Guru's system, which evaluates the performance of each variant in real-time. The variant with the highest score is considered the "winner" and automatically receives most of the traffic - your first analysis step is to confirm the dominance of this variant;

• Other calculation parameters: the score is based on the results of the period defined in the conversion window - this ensures that Guru prioritizes the current page performance, preventing old results from keeping an ineffective variant as the winner;

• Financial validation in the Capture Funnel: metrics: to confirm that the winning version is the most profitable, consider valid sales and revenue, and indicators such as Costs, ROI, and CPV;

• The detailed performance metrics , such as daily sales, leads, and clicks: they help measure the abandonment rate and identify any problems in the payment flow or in any checkout element.

Errors or Malfunctions

In this section, you will find solutions for any errors or malfunctions when accessing Guru's Neural A/B Test.

- I created an A/B Test, but when testing it, it always opens the same page (variant) in my browser. What could be wrong?

There is nothing wrong. Once the user has accessed the A/B Test link, the same variant will always be displayed to them. This serves to ensure the consistency of the displayed offer - useful in cases where different prices are to be tested. Therefore, it is absolutely normal for you to always see the same page during the test. Try opening the A/B Test link on other devices.

More Resources