Cell Configuration
The platform offers 5 data cleaning options:
Experiment Name: The name displayed on the platform on the incrementality experiments page.
Number of Experiments: The number of cells to design for in parallel. For less than 20 regions, we recommend a single cell at most. For 50 regions, a 2-cell experiment can be feasible. For more than 50 regions, we can run a 3-cell experiment. Only run Multicell experiments when its a business requirement, otherwise single cell experiments are more robust since that have access to all of the data at once
Channel name: The name of the channel we’re targeting.
Channel CPIC or CPA or IROAS: Input the target efficiency metric for your channel. CPIC (Cost Per Incremental Conversion) measures cost for each additional conversion, CPA (Cost Per Acquisition) represents total cost per customer acquisition, and IROAS (Incremental Return On Ad Spend) shows additional revenue generated per dollar spent.
Always give conservative values for these metrics. When designing the experiment, the initial estimate we input for iROAS or CPIC sets the lower bound of what we expect to detect.
For example, if we input an iROAS of 10 and the experiment requires a minimum detectable lift of $10,000 to be statistically significant, the required investment would be $1,000. This $1,000 would theoretically return 10× its value, providing the minimum detectable lift needed. However, if we overestimated the iROAS and the actual value is only 2, our spend wouldn't generate enough lift to reach the detection threshold, causing the experiment to fail.
The iROAS is usually reported through ad platforms, or though the MMM. A conservative value is around 1.5. as for CPIC or CPA this is more difficult to determine with absolutely no reference.
Objective: Defines if we are interested in a lift test to prove scalability, holdout test to measure incrementality, or a launch test to validate launch a new channel. Algorithmically, a lift or a launch test are identical
Budget:
In the case of a lift test: We ask for the maximum budget a client is willing to spend to run an experiment. If a client’s budget is 10k max, we will try to not suggest spending more than 10k in locations X, Y, Z…
In the case of a holdout test: We ask for the budget a client spends in all geos on that channel over 30 days. Consider this to be 100k. The test we designed targets 20% of our market over 15 days. we then calculate the spend in that portion of the market over 15 days to be $10000015/300.2=10000$. Knowing this, we will try to suggest experiments that target require to remove a maximum of 10000 in budget.