Overview:
A/B testing is a powerful technique that you can use to compare and evaluate two versions of subject lines, preview text, sender email address, and email content/template to determine which performs better.
By splitting and testing, you get insights into the variant's performance based on measuring metrics like Opens, Clicks, Total orders, and Total revenue.
In order to schedule a campaign, you should keep the following ready:
Contact list
Validated company email address
Email design
How To A/B Test A Campaign On BayEngage
Recommended Read: Here's a detailed step-by-step guide for sending a regular campaign on BayEngage.
Navigate to the email campaign option on the left-hand side menu bar and click on Create a campaign
2. Give your campaign a descriptive and easily identifiable name. Use the tags for future identification
Note:
You can use these tags to analyze reports and create segments using tags.
3. Select A/B testing under campaign type and click on next.
4. You can test unlimited A/B variants of from address, preview text, subject lines, and email contents. For ease of understanding, let’s take the example of 2 variants - Variant A and Variant B.
5. To create Variant B, you can either Clone and edit Variant A or create a new one with different email subject lines or content.
6. Setting/Configuring A/B test:
Begin by editing the size of the testing group.
The default testing size is 30%, i.e., 30% of the sample group will be equally divided and assigned to the variants.
For instance, if the total recipients = 100, 30% of the total recipients is 30, which is the sample size. This will be split into two variants. So, 15 users will receive Variant A, and the other 15 users will receive Variant B.
Here are a few things that you need to keep in mind while A/B testing your campaign:
Decide on the testing period
Select a method for choosing the winner of your A/B test (whether it’s going to be based on Open Rate, Click Rate, No. of Orders, or Total Revenue)
The variant (A or B) that has the highest percentage for the chosen winning metric will be considered as the winner and will be sent to the rest of the recipients, i.e., the remaining 70 recipients.
Note:
If the results of the variant get tied, then Variant A will be considered the winner and will be sent to the rest of the emails.
The variant with the highest percentage of the set winning metric - Open Rate/ Click Rate/ No. of orders/ Total Revenue after the ‘set time’ will be considered as the winning variant. That variant will be sent to the rest of the group.
Pro-Tip:
Experiment with subject lines and preview text if you wish to improve the open rates, and try different variants in the email content if you want to improve the click-through rate or conversion rate.