Creative testing, also known as split testing, is a method used to compare the performance of two or more variations of a creative to determine which one is more effective in achieving specific objectives.
Developing a creative testing strategy helps eliminate guesswork and drives the ability to make evidence-based decisions. Testing can also be seen as an ongoing process in your overall creative strategy, as it's important to keep refining your creatives to adapt to changing audience preferences and market conditions.
In this article, we'll review three creative testing options within the MNTN platform:
First, let's compare the options and determine a strategy best suited for you.
Determining the Testing Option Best Suited for You
Before selecting a testing option, think about how you want to implement the insights from your creative test after it's complete.
It's important to consider that once your campaign is set live with testing (A/B or Multivariate) enabled, you cannot edit its creative groups. This means:
❌ You will not be able to remove creative groups
❌ You will not be able to adjust the creative group's weighting
❌ You will not be able to adjust or remove the creative testing type
If you're looking to run a short test or you're looking to make creative adjustments within the same campaign shell, we'd recommend launching with the None option.
Below, you can compare the options side by side to determine a testing method best suited to your needs.
| No Testing | A/B Testing | Multivariate Testing |
How many creative groups can I add to the campaign? | Unlimited | 2 | 3+ |
Can I add and remove creative groups on the campaign? | Yes | No | No |
Can I adjust the weight of the creatives post-launch? | Yes | No | No |
How is my audience split? | At launch, each audience user or household will only see one creative group. If the weighting is updated, the audience split will be reset. | You set the creative weighting before the campaign launch. Each audience user or household will only see one creative group throughout the campaign's lifetime. | You set the creative weighting before the campaign launch. Each audience user or household will only see one creative group throughout the campaign's lifetime. |
None
When adding multiple creative groups on a new campaign, the creative testing toggle will automatically default to None. The None option allows you to add unlimited creative groups with custom weighting to your campaigns.
When using the None testing option, your audience will be split based on the weighting of your creative groups, with each audience user or household seeing only one creative group.
We recommend leaving the creative testing set to None to make adjustments to creatives after initial learnings.
If you want to run a creative test with the None functionality, do not change your creative groups post-launch, as this will reset your audience associated with each creative.
💡Note: Each time the weighting on your creative groups is updated, the audience split will be reset.
A/B Testing
With A/B Testing, you can run two creative groups against each other to test the impact of both on campaign performance.
Your test will be hard coded at a 50/50 split, and each user or household will see one creative group throughout the lifetime of your campaign.
It's important to note that if you've enabled A/B Testing and want to make creative adjustments post-launch, you'll need to create an entirely new campaign. This new campaign will require a new learning period, which is aligned with your typical sales cycle + attribution window.
Multivariate Testing
With Multivariate Testing, you can run 3+ creative groups against each other to test the impact of each on campaign performance.
The creative weight is equally split based on how many creative groups are being tested. Each audience user or household will see one creative group throughout the lifetime of your campaign.
It's important to note that if you've enabled Multivariate Testing and want to make creative adjustments post-launch, you'll need to create an entirely new campaign. This new campaign will require a new learning period, which is aligned with your typical sales cycle + attribution window.
WHERE TO GO NEXT: