A/B testing is one of the most powerful tools in Alia—it lets you experiment with different versions of your campaigns to see which one performs best. Whether you're testing pop-up design, delay timing, or educational content, this guide walks you through the full process of setting up an A/B test in just a few steps.
By utilizing A/B testing, you can:-
Test the presence of a floating button (teaser) versus no floating button to understand its impact on user engagement.
Experiment with Smart Trigger functionality to dynamically display popups based on user behavior.
Compare different visual designs and call-to-action copies.
Optimize timing by testing different display delays.
Align your final version with specific engagement metrics to ensure maximum ROI.
Pro Tip: Only change one variable at a time (like the delay or the headline) so you can clearly identify what caused the change in performance.
Step-by-Step Setup Guide
Follow these steps to launch your first experiment within the Alia dashboard:
Step 1: Choose Your Campaign
Navigate to the Campaigns section in your Alia dashboard.
Select the campaign you want to test. For this example, we’ll use a draft popup named “Control” in the campaign “All users”.
Step 2: Add a New Variation
Locate the dropdown menu next to the Preview button.
Select Duplicate as draft. This creates an identical second version of your popup to test against the original.
You can also click on "Add test variant" and that will open a duplicated version of the control popup. You can then edit that version to test it against the original.
Additionally, you can also click on the dropdown button next to the add test variant and build a popup from scratch to test it against the control version.
Step 3: Customize the Variation
Click on the newly created draft version to modify the specific element you wish to test. Common tests include:- Configuring a teaser-only variant where only the floating button is displayed, and the full popup appears upon clicking the teaser.
Testing the presence versus absence of a floating button to evaluate its effect on engagement.
Click on the duplicated version in draft and choose what you want to test (e.g. pop-up design, delay time, etc.). In our example, we’re testing a 5-second delay on the first version and a 10-second delay on the second version.
We recommend renaming your popups (e.g., "5s Delay" vs "10s Delay") to keep your dashboard organized.
Step 4: Launch the A/B Test
Once your variation is ready, select Start New A/B Test.
Give your test a descriptive title, such as "Time Delay Test - 5s vs 10s".
Select Create and Publish to begin the test immediately.
Monitoring Your Results
Once your test is live, Alia automatically distributes traffic between your variations. To see which version is winning, navigate to your campaign dashboard and click the View Test Results button.
You can analyze metrics such as signup rates to determine which variation should become your permanent "Control" version.
Advanced Testing: Floating Buttons and Smart Trigger
Testing Floating Buttons (Teasers)
Create an A/B test with one variant configured to display the floating button and another without it.
Ensure the full popup opens only when a visitor clicks the teaser for the floating button variant.
Keep Smart Trigger turned off during this test to isolate the impact of the floating button.
Using Smart Trigger in A/B Tests
Smart Trigger dynamically displays popups based on user behavior.
After completing the floating button test, run a separate A/B test to compare Smart Trigger on versus off.
Configuring Teaser-Only Variants
Set up a variant to display only the floating button instead of the full popup.
The full popup will appear only when a visitor clicks the teaser, allowing you to test the teaser's effectiveness in driving engagement.