Design of Experiments is a methodology used to support the planning, execution and analysis of statistically robust experiments. It has been used for many years across many fields of industry, from chemical and aeronautical engineering to bioprocessing and drug discovery.

Before diving into the details about what DOE is, let’s start with what it isn’t.

Originally, developed in the 1920s, DOE is not new. The principles of DOE were first laid out by Ronald Fischer whilst at Rothamsted Research. The story goes, that it all started in a very British manner - undermining someone else's opinion on how to make a good cup of tea. Wondering how to debunk what he saw as a dubious claim about tea-making, Fischer immediately saw that there was a much bigger question to answer: how do we make decisions and improve something? He found himself asking questions about:

Dissatisfied with the experimental design tools at his disposal to answer these questions, he set out to establish the founding principles of modern experimental design:

These principles were published as The Design of Experiments and have formed the basis of statistical experimentation ever since.

The traditional approach to science we’re all familiar with teaches that whenever we investigate several things in a system we should only ever change one at a time, holding all the rest constant. DOE practitioners call this approach “One Factor at a Time”, or OFAT. In this way, we test the impact of the selected variable (or Factor in DOE speak) in each variation.

The OFAT method works well in some cases but has two big limitations:

The influence of one DOE factor on the impact of another is known as an interaction. Interactions are the relationships between factors, where the value set for one factor impacts the effect of another factor on the signal you are measuring (or Response in DOE speak).

Lets work through this concept using the example below. We run an experiment consisting of the 4 runs (Table 1), measure our response, and plot the data (Figure 1).

Table 1. The design matrix for investigating all the combinations of two factors at low or high levels requires 4 runs.

Figure 1. Factor interaction plots. Left panel, Regardless of the setting for Factor 2, we can see that change in the response is the same as we increase the setpoint of Factor 1 from low to high. This tells us that there is no significant interaction between Factor 1 and Factor 2. Right panel, However, if we observe that when Factor 2 is “Low”, increasing Factor 1 leads to a decrease in the response, but when Factor 2 is High, increasing Factor 1 leads to a higher response signal then we can say that there is a factor interaction between these two factors.

By plotting the responses we measure, we can easily assess whether interactions are present within our data. Interactions will be indicated by diverging, converging, or intersecting slopes in our analysis. For more information on interpreting factor interaction plots, click here.

Interactions between factors are very common, particularly in the highly complex and emergent systems found in biology. Unstructured experiments, such as those used in OFAT, are simply not designed to enable you to investigate interactions between the variables (or factors) in your experiments. This causes a problem, by using OFAT, you are limiting the knowledge you can gain from your experimental work. This ultimately limits the questions you can answer of your data and the insight you can get from that data.

Taking an OFAT approach forces you to ignore many variables and doesn’t look at the interactions between these variables, it can often lead to us identifying the wrong system state as the optimum. In other words, we might have all the data we need to think we have the optimum but, in reality, we’re missing it entirely.

In contrast to OFAT experimentation, DOE allows you to investigate the interactions between variable factors in your experiment. By carefully selecting where our experimental runs are placed in our design space, we can better understand how our factors interact with each other to influence the response of interest.

The key principles of DOE allow you to gain faster, deeper insight into how experimentation impacts the process you are investigating. By taking a statistical, systematic and structured approach to how run conditions in an experiment are “located”, we gain more power to map, and ultimately improve our experimental process and understanding.

The techniques for designing and analyzing experiments that we use in DOE provide a practical framework for unpicking complex systems or processes. This is useful for biologists who face the unique challenge of working with enormously complex systems, often without the benefit of well-developed theoretical frameworks guiding their exploration. With DOE, scientists can take a more holistic approach in their methods when formulating and testing hypotheses.

The key difference between DOE and an OFAT approach is simple: put your experiments where they will give you the most possible information about an experimental space.

As scientists, this is really not drastically different from how we learn to experiment - change a variable and measure the effect to better understand the role of the variable. However, rather than changing one variable at a time we can actually change multiple variables at once. All DOE does is apply a systematic structure to these changes.

Another fundamental value of DOE is “more information from less effort”. The systematic structure of DOE experiments has more benefits than just mapping interactions. By utilising different “flavours” of design structure, we can select smaller and smaller numbers of experiments, and still get the critical understanding of our system.

There is often a misconception that DOE requires you to run very large and complex experiments. Whilst the combination of factors, and randomisation can make DOEs complex, it is possible to use highly efficient design types to keep the design size (or run number) to a minimum. Rather than performing one large experiment to answer every question you can think of it is best to take a more iterative approach - learning more information as you go.

DOE Campaigns can be structured in various ways, usually using some or all of the following steps (in roughly this order):

You can read more about this in our introduction to DOE Experiments and Campaigns.

One of the most powerful features of DOE is that it lets you do more than just see how your system behaves in the exact places you observed it. The aim behind designed experiments is to facilitate building a model which can predict behaviour anywhere in the region you have investigated.

DOE aims to make this process as efficient as possible. By filtering out unimportant factors and interactions in a set of iterative experiments, you can focus your modelling efforts on the factors which truly impact your process.

You can read more about this in our introduction to DOE and Modelling.

DOE is ultimately an approach to designing your experimental work which employs systematic experimental designs to augment the science you already do. It allows you to dive deeper into the complexities of biological systems and ultimately guides you through critical decisions to get the most impact out of your research efforts more quickly and at lower costs.

Biological research is hard, stop making it harder: use DOE!

To learn more about DOE experiments and campaigns click here.

To learn more about DOE and modelling click here.