It would be ideal if we could treat each ASIN as a stand-alone experiment, but if we change an ASIN's price we don't truly know what would have happened had the price not been changed. The only way to do that is through a kind of A/B testing, but to our best knowledge this is not possible at the moment. The alternative therefore is to have the scientific approach of experiment vs control. I agree that ASINs are all different, but if the control is large enough, each of the ASINs differences will balance each other out such that on average we can get a good comparison between that and the average of the ASINs we are repricing. We have various statistical tests for this.

We also have another performance measure that we consider, where essentially we use our AI to predict what would have happened had prices not been changed. If the control group data is not sufficient, we will go with this approach only.

We also have constructed profit measures that are adjusted for ads, since ad spend is often a variable that affects experimentation.