In Synthace, given a specific set of data, significant model terms will be identified automatically. However, there are a couple of limitations with the automatic approach used by Synthace to identify significant effects:
The significance of an effect is determined independently from any other significant effects that may have already been found or are yet to be found. This means that what might be found to be significant alone may not be significant given all other effects.
Assuming a 5% significance cut off, if you run 100 significance tests you would expect that 5 of those tests would result in false positives. This is known as the multiple testing problem.
Therefore, it is important that you do not just accept the p-values, but instead scrutinise them with your biological intuition and prior knowledge of the system under study.
This is especially true for effects that sit right on the border of the 5% significance cut off; these should be assessed using the significant effect plots on the Find Effects tab (click here to learn how to use these) and your biological intuition.
Given your assessment you may want to do one of two things:
Remove an effect that was determined to be significant.
Add an effect that wasn’t significant but you know to be something you want to include in your model.
Why might you want to remove an effect?
Having looked at the significant effects plots you might decide that the effect size appears very small and that it looks like a potential signal within the noise of your data points.
Why might you want to add an effect?
Whilst the statistics might tell you an effect is not significant you may know unequivocally from your own knowledge or prior experiments that the effect is important to the system.
It might not appear significant in this iteration of your DOE. Maybe, you tested that factor at an unsuitable range and missed the active range, but is an effect you want to model as you know you are going to continue to study this effect in further iterations.
Adding, removing and restoring significant effects
Click on the check box next to an effect that you want to remove from the selected set.
Click on the check box next to an effect that you want to add to the selected set.
Click on Select significant to restore the list of effects to just those that were identified as significant.
Well done for making it to the end of this tutorial.
To learn how to use the significant effects plots, click here.