The heart of DOE is how it allows you to divide-and-conquer even very daunting and complex experimental and technical questions. Here we will briefly outline the main tools DOE has to help you do this.

The first and most important question for your DOE investigation is what your ultimate goal is. It’s essential to decide on this at the beginning and keep it in mind as it defines how to make many of the choices that come up.

DOE breaks down into two main kinds of investigation:

Of course, these are not mutually exclusive: indeed, it’s not possible to optimize the system without understanding it to some degree.

How you will get to whichever goal you choose is then a question of where you are starting from. Common scenarios are

In each case you will commonly run a series of designed experiments, which we call a campaign. How long this series is and which steps are included varies depending on both the goal and the amount of knowledge, as well as what results you find as you go along. Critically, each of these experiments (usually called iterations in DOE terms) depends on the results of the ones before it, and in some cases reuses some or all of the specific experiments to save time and money (Figure 1).

Figure 1. DOE campaigns comprise a choice of different stages run iteratively to allow you to navigate through your experimental space honing in on the system optimum.

The stages below would nearly always be done in the order presented, however you may not do all of them. The following sections should help to identify when a stage is likely to be needed based on your goal and level of knowledge.

Goals:

Often you don’t know much about the system you’re investigating. This means there’s a risk that any given experimental run will fail to give any results, meaning you don’t really learn anything.

To get past this you need to do some early experiments to get to know your experimental system. DOE broadly terms these sorts of activities scoping. There are several ways to do this, depending on how many runs you can do and how little you actually know.

In cases where you can do lots of runs (96 or more) you could use a space-filling design (or spacefill). This is a good choice when you don’t know anything much about your system. If you’re hoping mostly to characterize the system you would think about using this sort of stage to explore a very broad range of behaviour.

If you can’t afford as many runs you could instead use a full factorial design for small numbers of factors, or a main-effects only optimal design.

If you already know how to avoid ‘dead’ regions of your space and have a good starting point then you can bypass this stage entirely and start with the next type of stage.

Goals:

If you know enough to have already defined a good place to start exploring without too much risk of lots of your experimental runs failing, but you don’t know which of a set of possible effects actually matter to the system you can start by screening.

Screening is where you investigate lots of possible factors at once, but sacrifice looking in detail by exploring only a small number of different settings (levels) of each factor. The goal of screening is to identify a small set of factors to investigate in more detail.

Depending on your goal you may make different choices here: when characterizing a system you may already know which factors you’re interested in and may not want to reduce the set much. When optimizing you only really care about the factors which make a big difference, so you’re likely to reduce the set more.

If you are already sure your set of factors is right (and sufficiently small), and you have a good starting point you can move to a more detailed investigation.

Goals:

Exploration is about just that: finding out about areas of your system’s behaviour that you don’t already know about and where there may be something interesting.

If this seems obscure it may help to think about your campaign as exploring a mountain range and trying to find peaks. If you’re characterizing the range the emphasis is on finding as many peaks as you can, particularly if they are different or unexpected. If you’re trying to optimize you just care about finding the highest one.

In either case you can often find yourself having explored a steep slope. That’s a good suggestion that there’s a peak somewhere nearby but to really know where it is and what it’s like you need to do more experiments. It’s also likely there are other areas which may contain peaks that you haven’t explored yet.

The exploration stage is about moving the region you’re exploring around by changing the factor levels, without necessarily changing the number of factors.

Goals

The name of this stage is a little misleading: you can use it regardless of whether or not you are trying to optimize. Really it’s about creating a design which allows you to estimate a more detailed, reliable model of your system.

Designs aimed at optimization are a lot more run-intensive than those used for screening and can cope with fewer factors - aiming to explore them in more depth to create the more detailed model.

Unlike designs made for screening, optimization designs explore the system at more than 2 levels, aiming to fit peaks rather than just flat planes.

If the optimization produces interesting results and the model seems good enough you would then often move to the final stage and check your predictions and see how sensitive the system is to small changes around those points.

Goals:

The final stage usually involves a small experiment aimed at confirming your findings at the optimization stage by running several replicates at your predicted optimum. This is most applicable when optimizing although it still serves as a good test of models made in characterization (although in that case there’s more than likely several predictions to test).

Additionally it’s usually important to understand the sensitivity of the system to small changes in the factors nearby the points of interest. For this you would typically explore around the point with a small design to confirm that any anticipated changes within the normal range the system would experience (e.g. to temperature) don’t drastically change the response.

As mentioned above, the number and types of stages you might choose to run really depends on the level of knowledge about your system that you are starting with, what you are trying to achieve, and potentially any cost or time constraints you have around the experiments you can run.

Even when your goal is to optimise your system, you might not need to run multiple iterations to find the optimal conditions if all you need is for your system to be good enough - above a threshold. In the latter scenario you might be able to find robust conditions that meet this criteria within a couple of iterations.

There are no firm rules here, and that is the beauty of DOE: it is flexible to fit the needs of your goals with each iteration providing you with more knowledge of your system, it is up to you when you feel you know enough to stop iterating.

To learn about DOE and modelling, click here.