Alia’s A/B testing analytics dashboard helps you confidently identify winning popup variations by measuring signup performance, conversion impact, and downstream revenue.
With advanced filtering capabilities, you can analyze results across specific shopper segments — including UTMs, device type, country, survey responses, browsing behavior, and more — allowing you to uncover insights that go far beyond surface-level metrics.
Step 1: Access A/B Test Results
Navigate to your campaign dashboard in Alia.
Click on "View A/B Test Results" to open the test results page.
You’ll land on the analytics dashboard where all experiment data is automatically aggregated.
Step 2: Review Test Timeline
The dashboard displays the full test duration, including:
Start date
End date (or current progress if still running)
If your test is active, metrics update in near real-time so you can monitor performance trends as data accumulates.
💡 Best practice: Avoid ending tests too early — stronger data leads to more reliable winners.
Step 3: Compare Control vs. Test Group
Each experiment includes:
Control: Your original popup
Variants: The versions being tested
You can have an unlimited number of variants
Review performance side-by-side to understand which experience drives stronger results.
👉 The percentage lift displayed between variants helps quickly identify directional winners.
Step 4: Segment Your Results with Advanced Filters
One of the most powerful features of Alia’s new analytics experience is the ability to filter performance across virtually any shopper dimension.
You can analyze results by:
Traffic & Marketing
UTM source, medium, and campaign
Initial path
Domain
Shopper Attributes
Country, region, and city
Shopify market and locale
Device type, browser, and OS
Behavior & Actions
Visitors matching targeting
Email submissions
Reward variants
Purchased products or collections
Zero-Party Data
Survey responses
Mini-quiz answers
Custom user properties
💡 Why this matters:
A test may appear neutral at a high level but reveal a clear winner within a high-value segment — such as paid traffic or mobile shoppers.
This is where the biggest optimization opportunities often live.
💡 The percentage difference between the control and test groups will help determine which performs better.
Step 5: Evaluate Key Performance Metrics
Your dashboard includes all primary analytics, plus sitewide impact metrics.
Core Metrics to Watch
Email & SMS submit rates
Opt-in rates
Attributed conversion rate
Attributed orders and sales
Average attributed order value
Time to purchase
Sitewide Impact Metrics
Sitewide conversion rate
Sitewide sales
Bounce rate
These help ensure your popup improves signup performance without harming the overall shopping experience.
💡 If statistical significance isn’t reached, allow the test to run longer for more reliable results.
Step 6: Check Statistical Significance
Each metric includes a significance indicator:
✅ Check mark — Results are statistically significant
⏳ Clock icon — More data is needed
💡 Pro Tip: If significance hasn’t been reached, allow the test to run longer before making a decision. Ending tests prematurely can lead to false winners.
Step 7: Identify and Publish the Winning Variation
Once a clear winner emerges:
Select the top-performing variant
Publish it directly from the results page
Alia will automatically:
✅ Keep the winning popup live
✅ Move the losing variation to draft
✅ Preserve test data for future analysis
What’s Next?
Winning a test is just the beginning.
Use your insights to continuously improve performance:
🚀 Test new messaging or offers
🎨 Experiment with design changes
🎯 Refine targeting
📊 Optimize by traffic source
🧠 Personalize experiences using zero-party data
The highest-performing brands treat A/B testing as an ongoing growth engine — not a one-time exercise.