A/B Tests

Premium feature.

Whether a banner is accepted depends heavily on layout and wording. Instead of guessing, run two variants against each other.

Creating a test

Field	Meaning
Test name	Internal label.
Status	Draft, Running or Completed. Only running tests are served.
Split ratio	Percentage of visitors who see variant A. 50 means an even split.
Complete automatically after X views	Once reached, the test ends and the winner is marked. Leave empty to end it manually.
Sales channel	Leave empty to test in all channels.

Configuring the variants

Each variant is a design configuration in the same JSON format as the design editor.

A practical route: set up the design you want in the design editor, save, copy the JSON value from the Banner design configuration field of the plugin configuration and paste it here. Then restore your original design.

Invalid JSON is rejected on save.

Assignment

Visitors are assigned to a variant on their first page view. The assignment is stored for 30 days in the cookie prems_cc_ab_variant so that nobody bounces between variants. The assigned variant overrides the stored banner design completely — including layout and overlay.

Live statistics

For each variant you see views, accepted and the resulting acceptance rate. Any decision that goes beyond the required groups counts as accepted.

Let a test run long enough. At a few hundred views a difference of two or three percentage points is noise. As a rule of thumb: at least a few thousand views per variant and a full week, so weekday effects even out.

Adopting the winner

Once a test is completed, Adopt winner as default copies the design configuration of the better variant into the regular banner design. The test then stops running.