Brereton Chemometrics

EXPERIMENTAL DESIGN					89
	Table 2.36 A {5, 2} simplex centroid design.
1		0	0	0	0
0		1	0	0	0
0		0	1	0	0
0		0	0	1	0
0		0	0	0	1
1/2		1/2	0	0	0
1/2		0	1/2	0	0
1/2		0	0	1/2	0
1/2		0	0	0	1/2
0		1/2	1/2	0	0
0		1/2	0	1/2
0		1/2	0	0	1/2
0		0	1/2	1/2	0
0		0	1/2	0	1/2
0		0	0	1/2	1/2

A {k, m} simplex lattice design consists of all possible combinations of 0, 1/m, 2/m, . . . , m/m or a total of

N = (k + m − 1)!/[(k − 1)!m!]

experiments where there are k factors. A {3, 3} simplex lattice design can be set up analogous to the {3, 3} simplex centroid design given in Table 2.34. There are

•three single factor experiments,

•six experiments where one factor is at 2/3 and the other at 1/3,

•and one experiment where all factors are at 1/3,

resulting in 5!/(2!3!) = 10 experiments in total, as illustrated in Table 2.37 and Figure 2.34. Note that there are now more experiments than are required for a full Sheffe´ model, so some information about the significance of each parameter could be obtained; however, no replicates are measured. Generally, though, chemists mainly use mixture models for the purpose of optimisation or graphical presentation of results. Table 2.38 lists how many experiments are required for a variety of {k, m} simplex centroid designs.

Table 2.37 Three factor simplex lattice design.

Experiment		Factor 1	Factor 2	Factor 3
1		1	0	0
3		0	0	1
2	Single factor	0	1	0
4		2/3	1/3	0
6		2/3	0	1/3
5		1/3	2/3	0
7		1/3	0	2/3
		0	2/3	1/3
8
	Binary	0	1/3	2/3
9
10 Ternary		1/3	1/3	1/3

90			CHEMOMETRICS
		Factor 1
		1
	4	6
	5	10	7
2	8	9	3
Factor 2			Factor 3

Figure 2.34

Three factor simplex lattice design

Table 2.38 Number of experiments required for various simplex lattice designs, with different numbers of factors and interactions.

Factors (k)			Interactions (m)
	2	3	4	5	6
2	3
3	6	10
4	10	20	35
5	15	35	70	126
6	21	56	126	252	462

2.5.4 Constraints

In chemistry, there are frequently constraints on the proportions of each factor. For example, it might be of interest to study the effect of changing the proportion of ingredients in a cake. Sugar will be one ingredient, but there is no point baking a cake using 100 % sugar and 0 % of each other ingredient. A more sensible approach is to put a constraint on the amount of sugar, perhaps between 2 and 5 %, and look for solutions in this reduced mixture space. A good design will only test blends within the specified regions.

Constrained mixture designs are often difficult to set up, but there are four fundamental situations, exemplified in Figure 2.35, each of which requires a different strategy.

1.Only a lower bound for each factor is specified in advance.

•The first step is to determine whether the proposed lower bounds are feasible. The sum of the lower bounds must be less than one. For three factors, lower bounds of 0.5, 0.1 and 0.2 are satisfactory, whereas lower bounds of 0.3, 0.4 and 0.5 are not.

•The next step is to determine new upper bounds. For each factor these are 1 minus the sum of the lower bounds for all other factors. If the lower bounds for three factors are 0.5, 0.1 and 0.2, then the upper bound for the first factor is

EXPERIMENTAL DESIGN

91

100 % Factor 1

100 % Factor 1

Lower

bound

Allowed

Upper

Lower

bound

mixture

Allowed

bound

mixture

Upper

Upper

Lower

bound

bound

bound

100 % Factor 2

100 % Factor 3

100 % Factor 2

100 % Factor 3

(a) Lower bounds defined

(b) Upper bounds defined

Allowed

100 % Factor 1

2

mixture

Factor

Factor 3

100 % Factor 2

100 % Factor 3

Factor 1

(c) Upper and lower bounds defined, fourth factor as filler

(d) Upper and lower bounds defined

Figure 2.35

Four situations encountered in constrained mixture designs

1 − 0.1 − 0.2 = 0.7, so the upper bound of one factor plus the lower bounds of the other two must equal one.

•The third step is to take a standard design and the recalculate the conditions, as follows:

xnew ,f = xold ,f (U,f − Lf ) + Lf

where Lf and Uf are the lower and upper bounds for factor f . This is illustrated in Table 2.39.

The experiments fall in exactly the same pattern as the original mixture space. Some authors call the vertices of the mixture space ‘pseudo-components’, so the first pseudo-component consists of 70 % of pure component 1, 10 % of pure component 2 and 20 % of pure component 3. Any standard design can now be employed. It is also possible to perform all the modelling on the pseudo-components and convert back to the true proportions at the end.

2.An upper bound is placed on each factor in advance. The constrained mixture space often becomes somewhat more complex dependent on the nature of the upper bounds. The trick is to find the extreme corners of a polygon in mixture space,

92	CHEMOMETRICS

Table 2.39 Constrained mixture design with three lower bounds.

	Simple centroid design				Constrained design
	Factor 1	Factor 2	Factor 3		Factor 1	Factor 2	Factor 3
	1.000	0.000	0.000		0.700	0.100	0.200
	0.000	1.000	0.000		0.500	0.300	0.200
	0.000	0.000	1.000		0.500	0.100	0.400
	0.500	0.500	0.000		0.600	0.200	0.200
	0.500	0.000	0.500		0.600	0.100	0.300
	0.000	0.500	0.500		0.500	0.200	0.300
	0.333	0.333	0.333		0.567	0.167	0.267
Lower	0.5	0.1	0.2
Upper	0.7	0.3	0.4

Table 2.40 Constrained mixture designs with upper bounds established in advance.

(a) Upper	0.3	0.4	0.5
bounds
1	0.3	0.4	0.3	Factors 1 and 2 high
2	0.3	0.2	0.5	Factors 1 and 3 high
3	0.1	0.4	0.5	Factors 2 and 3 high
4	0.3	0.3	0.4	Average of experiments 1 and 2
5	0.2	0.4	0.4	Average of experiments 1 and 3
6	0.2	0.3	0.5	Average of experiments 2 and 3
7	0.233	0.333	0.433	Average of experiments 1, 2 and 3
(b) Upper	0.7	0.5	0.2
bounds
1	0.7	0.1	0.2	Factors 1 and 3 high
2	0.3	0.5	0.2	Factors 2 and 3 high
3	0.7	0.3	0.0	Factor 1 high, factor 2 as high as
				possible
4	0.5	0.5	0.0	Factor 2 high, factor 1 as high as
				possible
5	0.7	0.2	0.1	Average of experiments 1 and 3
6	0.4	0.5	0.1	Average of experiments 2 and 4
7	0.5	0.3	0.2	Average of experiments 1 and 2
8	0.6	0.4	0.0	Average of experiments 3 and 4

perform experiments at these corners, midway along the edges and, if desired, in the centre of the design. There are no hard and fast rules as the theory behind these designs is complex. Recommended guidance is provided below for two situations. The methods are illustrated in Table 2.40 for a three factor design.

•If the sum of all (k − 1) upper bounds is ≤1, then do as follows:

(a)set up k experiments where all but one factor is its upper bound [the first three in Table 2.40(a)]; these are the extreme vertices of the constrained mixture space;

(b)then set up binary intermediate experiments, simply the average of two of the k extremes;

(c)if desired, set up ternary experiments, and so on.

EXPERIMENTAL DESIGN	93

•If this condition is not met, the constrained mixture space will resemble an irregular polygon as in Figure 2.35(b). An example is illustrated in Table 2.40(b).

(a)Find the extreme vertices for those combinations of (k − 1) factors that are less than one, of which there are two in this example.

(b)Each missing vertex (one in this case) increases the number of new vertices by one. If, for example, it is impossible to simultaneously reach maxima for

factors 1 and 2, create one new vertex with factor 1 at its highest level (U1), factor 3 at 0 and factor 2 at (1 − U1), with another vertex for factor 2 at U2, factor 3 at 0 and factor 1 at (1 − U2).

(c)If there are v vertices, calculate extra experimental points between the vertices. Since the figure formed by the vertices in (b) has four sides, there will be four extra experiments, making eight in total. This is equivalent to performing one experiment on each corner of the mixture space in Figure 2.35(b), and one experiment on each edge.

(d)Occasionally, one or more experiments are performed in the middle of the

new mixture space, which is the average of the v vertices.

Note that in some circumstances, a three factor constrained mixture space may be described by a hexagon, resulting in 12 experiments on the corners and edges. Provided that there are no more than four factors, the constrained mixture space is often best visualised graphically, and an even distribution of experimental points can be determined by geometric means.

3.Each factor has an upper and lower bound and a (k + 1)th factor is added (the fourth in this example), so that the total comes to 100 %; this additional factor is called a filler. An example might be where the fourth factor is water, the others being solvents, buffer solutions, etc. This is common in chromatography, for example, if the main solvent is aqueous. Standard designs such as factorial designs can be

employed for the three factors in Figure 2.35(c), with the proportion of the final factor computed from the remainder, given by (1 − x1 − x2 − x3). Of course, such designs will only be available if the upper bounds are low enough that their sum is no more than (often much less than) one. However, in some applications it is common to have some background filler, for example flour in baking of a cake, and active ingredients that are present in small amounts.

4.Upper and lower bounds defined in advance. In order to reach this condition, the sum of the upper bound for each factor plus the lower bounds for the remaining factors must not be greater than one, i.e. for three factors

U1 + L2 + L3 ≤ 1

and so on for factors 2 and 3. Note that the sum of all the upper bounds together must be at least equal to one. Another condition for three factors is that

L1 + U2 + U3 ≥ 1

otherwise the lower bound for factor 1 can never be achieved, similar conditions applying to the other factors. These equations can be extended to designs with more factors. Two examples are illustrated in Table 2.41, one feasible and the other not.

The rules for setting up the mixture design are, in fact, straightforward for three factors, provided that the conditions are met.

94					CHEMOMETRICS
	Table 2.41 Example of simultaneous
	constraints in mixture designs.
	Impossible conditions
	Lower	0.1	0.5	0.4
	Upper	0.6	0.7	0.8
	Possible conditions
	Lower	0.1	0.0	0.2
	Upper	0.4	0.6	0.7

1.Determine how many vertices; the maximum will be six for three factors. If the sum of the upper bound for one factor and the lower bounds for the remaining factors equal one, then the number of vertices is reduced by one. The number of vertices also reduces if the sum of the lower bound of one factor and the upper bounds of the remaining factors equals one. Call this number ν. Normally one will not obtain conditions for three factors for which there are less than three vertices, if any less, the limits are too restrictive to show much variation.

2.Each vertex corresponds to the upper bound for one factor, the lower bound for another factor and the final factor is the remainder, after subtracting from 1.

3.Order the vertices so that the level of one factor remains constant between vertices.

4.Double the number of experiments, by taking the average between each successive vertex (and also the average between the first and last), to provide 2ν experiments. These correspond to experiments on the edges of the mixture space.

5.Finally it is usual to perform an experiment in the centre, which is simply the average of all the vertices.

Table 2.42 illustrates two constrained mixture designs, one with six and the other with five vertices. The logic can be extended to several factors but can be complicated.

Table 2.42 Constrained mixture design where both upper and lower limits are known in advance.

(a) Six vertices
Lower	0.1	0.2	0.3
Upper	0.4	0.5	0.6

Step 1

•0.4 + 0.2 + 0.3 = 0.9

•0.1 + 0.5 + 0.3 = 0.9

•0.1 + 0.2 + 0.6 = 0.9 so ν = 6

Steps 2 and 3 Vertices
A	0.4	0.2	0.4
B	0.4	0.3	0.3
C	0.1	0.5	0.4
D	0.2	0.5	0.3
E	0.1	0.3	0.6
F	0.2	0.2	0.6

EXPERIMENTAL DESIGN					95
	Table 2.42 (continued )
	Steps 4 and 5 Design
1		A	0.4	0.2	0.4
2		Average A & B	0.4	0.25	0.35
3		B	0.4	0.3	0.3
4		Average B & C	0.25	0.4	0.35
5		C	0.1	0.5	0.4
6		Average C & D	0.15	0.5	0.35
7		D	0.2	0.5	0.3
8		Average D & E	0.15	0.4	0.45
9		E	0.1	0.3	0.6
10		Average E & F	0.15	0.25	0.6
11		F	0.2	0.2	0.6
12		Average F & A	0.3	0.2	0.5
13		Centre	0.2333	0.3333	0.4333

(b) Five vertices
Lower	0.1	0.3	0
Upper	0.7	0.6	0.4

Step 1

•0.7 + 0.3 + 0.0 = 1.0

•0.1 + 0.6 + 0.0 = 0.7

•0.1 + 0.3 + 0.4 = 0.8 so ν = 5

Steps 2 and 3 Vertices
A	0.7	0.3	0.0
B	0.4	0.6	0.0
C	0.1	0.6	0.3
D	0.1	0.5	0.4
E	0.3	0.3	0.4

Steps 4 and 5 Design
1	A	0.7	0.3	0.0
2	Average A & B	0.55	0.45	0.0
3	B	0.4	0.6	0.0
4	Average B & C	0.25	0.6	0.15
5	C	0.1	0.6	0.3
6	Average C & D	0.1	0.55	0.35
7	D	0.1	0.5	0.4
8	Average D & E	0.2	0.4	0.4
9	E	0.3	0.3	0.4
10	Average E & A	0.5	0.3	0.2
11	Centre	0.32	0.46	0.22

If one is using a very large number of factors all with constraints, as can sometimes be the case, for example in fuel or food chemistry where there may be a lot of ingredients that influence the quality of the product, it is probably best to look at the original literature as designs for multifactor constrained mixtures are very complex: there is insufficient space in this introductory text to describe all the possibilities in detail. Sometimes constraints might be placed on one or two factors, or one factor could have

96	CHEMOMETRICS

an upper limit, another a lower limit, and so on. There are no hard and fast rules, but when the number of factors is sufficiently small it is important to try to visualise the design. The trick is to try to obtain a fairly even distribution of experimental points over the mixture space. Some techniques, which will indeed have feasible design points, do not have this property.

2.5.5 Process Variables

Finally, it is useful to mention briefly designs for which there are two types of variable, conventional (often called process) variables, such as pH and temperature, and mixture variables, such as solvents. A typical experimental design is represented in Figure 2.36, in the case of two process variables and three mixture variables consisting of 28 experiments. Such designs are relatively straightforward to set up, using the principles in this and earlier chapters, but care should be taken when calculating a model, which can become very complex. The interested reader is strongly advised to check the detailed literature as it is easy to become very confused when analysing such types of design, although it is important not to be put off; as many problems in chemistry involve both types of variables and since there are often interactions between mixture and process variables (a simple example is that the pH dependence of a reaction depends on solvent composition), such situations can be fairly common.

Figure 2.36

Mixture design with process variables

EXPERIMENTAL DESIGN	97

2.6 Simplex Optimisation

Experimental designs can be employed for a large variety of purposes, one of the most successful being optimisation. Traditional statistical approaches normally involve forming a mathematical model of a process, and then, either computationally or algebraically, optimising this model to determine the best conditions. There are many applications, however, in which a mathematical relationship between the response and the factors that influence it is not of primary interest. Is it necessary to model precisely how pH and temperature influence the yield of a reaction? When shimming an NMR machine, is it really important to know the precise relationship between field homogeneity and resolution? In engineering, especially, methods for optimisation have been developed which do not require a mathematical model of the system. The philosophy is to perform a series of experiments, changing the values of the control parameters, until a desired response is obtained. Statisticians may not like this approach as it is not normally possible to calculate confidence in the model and the methods may fail when experiments are highly irreproducible, but in practice sequential optimisation has been very successfully applied throughout chemistry.

One of the most popular approaches is called simplex optimisation. A simplex is the simplest possible object in N -dimensional space, e.g. a line in one dimension and a triangle in two dimensions, as introduced previously (Figure 2.32). Simplex optimisation implies that a series of experiments are performed on the corners of such a figure. Most simple descriptions are of two factor designs, where the simplex is a triangle, but, of course, there is no restriction on the number of factors.

2.6.1 Fixed Sized Simplex

The most common, and easiest to understand, method of simplex optimisation is called the fixed sized simplex. It is best described as a series of rules.

The main steps are as follows, exemplified by a two factor experiment.

1.Define how many factors are of interest, which we will call k.

2.Perform k + 1(=3) experiments on the vertices of a simplex (or triangle for two factors) in factor space. The conditions for these experiments depend on the step size. This defines the final ‘resolution’ of the optimum. The smaller the step size, the better the optimum can be defined, but the more the experiments are necessary. A typical initial simplex using the step size above might consist of the three experiments, for example

(a)pH 3, temperature 30 ◦C;

(b)pH 3.01, temperature 31 ◦C;

(c)pH 3.02, temperature 30 ◦C.

Such a triangle is illustrated in Figure 2.37. It is important to establish sensible initial conditions, especially the spacing between the experiments; in this example one is searching very narrow pH and temperature ranges, and if the optimum is far from these conditions, the optimisation will take a long time.

3.Rank the response (e.g. the yield of rate of the reaction) from 1 (worst) to k + 1 (best) over each of the initial conditions. Note that the response does not need to be quantitative, it could be qualitative, e.g. which food tastes best. In vector form the conditions for the nth response are given by xn, where the higher the value of

98					CHEMOMETRICS
		3	centroid of b and c
31		b		new
temperature
30
	a		c
1			2
3.00		3.01	3.02	3.03
			pH

Figure 2.37

Initial experiments (a, b and c) on the edge of a simplex: two factors, and the new conditions if experiment a results in the worst response

n the better is the response, e.g. x3 = (3.01 31) implies that the best response was at pH 3.01 and 31 ◦C.

4. Establish new conditions for the next experiment as follows: xnew = c + c − x1

where c is the centroid of the responses 2 to k + 1 (excluding the worst response), defined by the average of these responses represented in vector form, an alternative expression for the new conditions is xnew = x2 + x3 − x1 when there are two factors. In the example above

• if the worst response is at x1 = (3.00 30),

•the centroid of the remaining responses is c = [(3.01 + 3.02)/2 (30 + 31)/2] = (3.015 30.5),

• so the new response is xnew = (3.015 30.5) + (30.015 30.5) − (3.00 30) = (30.03 31).

This is illustrated in Figure 2.37, with the centroid indicated. The new experimental conditions are often represented by reflection of the worst conditions in the centroid of the remaining conditions. Keep the points xnew and the kth (=2) best responses from the previous simplex, resulting in k + 1 new responses. The worst response from the previous simplex is rejected.

5.Continue as in steps 3 and 4 unless the new conditions result in a response that is worst than the remaining k(=2) conditions, i.e. ynew < y2, where y is the corresponding response and the aim is maximisation. In this case return to the previous conditions, and calculate

xnew = c + c − x2

where c is the centroid of the responses 1 and 3 to k + 1 (excluding the second worst response) and can also be expressed by xnew = x1 + x3 − x2, for two factors.