1 Experimental design

So far in this Workbook we have looked at some of the statistical methods used in the analysis and interpretation of experimental results. There are occasions when the planning of an experiment is not in the control of the statistician responsible for analysing the results. It is always preferable to have some idea of the likely variability of the data so that any experimental design can take this into account. For this reason, the design of experiments is of crucial importance if weight is to be given to the results obtained. Usually, the experimenter will have to take into account:

1. The definition of the problem to be investigated. This would usually include the selection of the response variable to be measured and the factors or treatments influencing the response. Remember that the factors may be quantitative (such as temperature, pressure or force), qualitative (such as days of the week, machine operators or machines themselves) and decisions must be taken as to whether these factors are fixed or random and at what levels they are to be used.
2. The sample size. Clearly the experimenter should determine the number of observations to be taken and the random order in which the experiments are to occur in order that the effects of uncontrollable or unforeseen variables are minimized.
3. Data collection. Decisions need to be taken as to how the data are to be collected and tabulated. The calculation of the test statistics needs to be taken into account as does the level of acceptance or rejection of any hypotheses used.

We have already looked at some introductory ANOVA situations and we now turn our attention to so-called block designs used in the conduct of experiments.

Block design

Block design, or more specifically, randomized block design enables an unbiased estimate of error to be obtained and ensures that the error obtained is a minimum.

As an illustration, imagine that we wish to compare four extrusion processes and measure their effect on the brittleness of copper wire produced. Assume further that the copper from which the wire is made is delivered in quantities to allow only four tests per batch. We will refer to such a batch as a block . If we replicate each treatment four times, we could organize the four members of block 1 all to receive treatment $A$ , all four members of block 2 all to receive treatment $B$ and so on. This situation could be represented as shown below.

Giving every member of a given block the same treatment is not a very sensible thing to do. The reason for this is that any observed differences might well be due to differences between the blocks and not differences between the treatments. Remember that each block consists of a batch of material and so as engineers we would expect some variation in batches of materials delivered for processing. We can avoid this pitfall by ensuring that the blocks are distributed among the treatments. Ensuring that each block experiences every treatment could lead to the situation represented below.

On the face of it, we appear to have a sensible way of organizing the treatments between blocks. However it is entirely possible that other variables might come into play which might change with time which would bias the observations made. For example, if the treatments are always applied in a particular order, say $A,B,C$ and then $D$ it could be that the state of the extrusion machines might change with time which would bias the results. For example, if treatment $A$ is always applied first, it could be that the extrusion machine is not fully ‘warmed up’ and so non-typical results might occur.

In order to remove the bias referred to above, we could randomize the order in which the treatments are applied. This might result in the randomized block design represented below in which comparisons are made between sets of treatments applied to fairly homogeneous material.

Note that this design may be taken to consist of two independent variables, namely blocks and treatments. The total sum of the squares (previously referred to as $S{S}_T$ ) may be partitioned as

$S{S}_T=S{S}_{\text{Blocks}}+S{S}_{\text{Treatments}}+S{S}_{\text{Error}}$

We illustrate this in Example 2.

Example 2

The compressive strength of concrete to be used in the construction of a dam may depend on the proportion of a particular component of the mix. In order to investigate this, the compressive strength of concrete containing a variety of percentages of the component, 2%, 4%,.....10% was measured. The concrete was made using four batches of cement. One part of each batch was used with each of the percentages of the component of interest. The resulting concrete samples were then subjected to tests to determine their compressive strength. The data obtained are given below.

What conclusions may be drawn about the effect of the mix component on the compressive strength of the resulting concrete? Is there evidence of differences between the batches of cement. Use the 5% level of significance.

Solution

Assuming that no interaction takes place between the blocks and the parts, we can calculate the appropriate sums of squares as done previously. The grand total of the observations is represented by $T$ .

The total sum of squares is given by

$S{S}_T=201^2+\dots +182^2+\dots +260^2-\frac{4604^2}{5\times 4}=1082234-1059840.8=22393.2$

The sum of squares for the treatments is given by

$S{S}_{Tr}=\frac{1190^2+\dots +1113^2}{5}-\frac{4604^2}{5\times 4}=1060736.4-1059840.8=895.6$

The sum of squares for the blocks is given by

$S{S}_{Bl}=\frac{747^2+\dots +1092^2}{4}-\frac{4604^2}{5\times 4}=107950.5-1059840.8=19661.7$

The sum of squares for the errors is given by

$S{S}_E=S{S}_T-S{S}_{Tr}-S{S}_{Bl}=22393.2-895.6-19661.7=1835.9$

These calculations give rise to the following ANOVA table

Conclusions

1. Blocks

   From the $F$ -tables, $F_{3,12}=3.49$ and since $1.95<3.49$ we have no evidence to reject the null hypothesis that there are no differences between the batches of cement.

2. Treatments

   From the $F$ -tables, $F_{4,12}=3.26$ and since $32.13>3.26$ we have sufficient evidence to reject the hull hypothesis that the addition of the mix component has no effect on the compressive strength of the resulting concrete. Hence our conclusion (at the 5% level of significance) is that the addition of the mix component does affect the compressive strength of the resulting concrete.

Task!

The tensile strength of aluminium alloy tubing to be used in the construction of aircraft is known to depend on the extrusion process by which the tubing is produced. In order to investigate the tensile strength of the alloy made by four different extrusion processes, four large samples were made using each extrusion process. One sample from each process was sent to each of four laboratories for measurement of its tensile strength. The data obtained are given below.

What conclusions may be drawn about the effect of the extrusion processes on the tensile strength of the resulting tubing? Is there sufficient evidence to suggest that there are systematic differences between measurements from the different laboratories? Use the 5% level of significance.

Answer

1.1 Latin Squares

We have previously looked at the elimination of a source of variation via the use of randomized blocks. If, in a given situation, there are two sources of variation to be controlled, a Latin Square design may provide the best possible analysis. Essentially this design groups the treatments involved into different blocks simultaneously. As an example, consider the problem of checking the quality of the output of four machines over four 6-hour shifts manned by the same four operators on each shift. A Latin Square design allocates the sixteen combinations to be used. The machines are labelled $A,B,C$ and $D$ .

A design is as follows.

Notice that each machine appears in each row and each column exactly once. Notice also that the number of observations must be equal to the square of the number of treatments (here $4^2=16$ ), or a multiple of this. In situations where a large number of treatments are used, a very substantial testing effort is implied. In return, a high reduction in errors is achieved since every row and every column is a complete replication. Experiments using Latin Squares should be designed so that the differences in rows and columns represent the major sources of variation to be considered.

Example 3

In an experiment designed to compare the tensile strengths of plastic tubes manufactured by different methods there are four different methods, $A,B,C,D.$ It is also believed that there may be effects due to differences between batches of the plastic pellets which are used as raw material and between manufacturing plants. These factors are arranged in a Latin Square with four different manufacturing plants as the row factor and the pellet batch as the column factor.

Analyse the results (in coded units of tensile strength), given below, to determine whether or not there are significant differences between the methods. Use the 5% level of significance.

Solution

The treatment totals are as follows:

The row totals are as follows:

The column totals are as follows:

The grand total is 281.0. The total sum of squares is

$16.6^2+\cdots +19.2^2-\frac{281.0^2}{16}=13.2375.$

The treatments sum of squares is $\frac{1}{4}\left(67.4^2+\cdots +68.9^2\right)-\frac{281.0^2}{16}=5.8925.$

The batches sum of squares is $\frac{1}{4}\left(69.7^2+\cdots +72.4^2\right)-\frac{281.0^2}{16}=2.4625.$

The plants sum of squares is $\frac{1}{4}\left(68.3^2+\cdots +72.6^2\right)-\frac{281.0^2}{16}=2.4125.$

The analysis of variance table is as follows.

The upper 5% point of the $F_{3,6}$ distribution is 4.76. Thus we can draw the following conclusions.

• Treatments (manufacturing methods)

  The variance ratio is significant at the 5% level so we conclude that there is sufficient evidence to reject the null hypothesis that the treatments give equal mean tensile strengths.

• Batches

  The variance ratio is not significant at the 5% level so we conclude that there is not sufficient evidence to reject the null hypothesis that the batches have no effect on mean tensile strengths.

• Plants

  The variance ratio is not significant at the 5% level so we conclude that there is not sufficient evidence to reject the null hypothesis that the plants have no effect on mean tensile strengths.

Task!

The yields of a chemical process when using three different catalysts, A, B, C, are to be compared. It is also believed that there may be effects due to the reaction vessel used and the operator. A Latin Square design is used with three operators and three vessels. A batch is produced using each combination of catalyst, vessel and operator. The results (%) are as follows.

Analyse the results to test for the effects of the factors. Use the 5% level of significance.

Answer

The row totals are as follows:

The column totals are as follows:

The grand total is 584.3. The total sum of squares is

$81.4^2+\cdots +72.5^2-\frac{584.3^2}{9}=722.3556.$

The treatments sum of squares is

$\frac{1}{3}\left(220.1^2+197.7^2+165.5^2\right)-\frac{584.3^2}{9}=483.1289.$

The operators sum of squares is

$\frac{1}{3}\left(204.9^2+178.4^2+201.0^2\right)-\frac{584.3^2}{9}=136.4689.$

The reaction vessels sum of squares is

$\frac{1}{3}\left(201.0^2+182.7^2+200.6^2\right)-\frac{584.3^2}{9}=72.8289.$

The analysis of variance table is as follows.

The upper 5% point of the $F_{2,2}$ distribution is 19.00.

Thus we can draw the following conclusions.

1. Treatments (catalysts)

   The variance ratio is not significant at the 5% level so we conclude that there is not sufficient evidence to reject the null hypothesis that the treatments give equal mean yields.

2. Operators

   The variance ratio is not significant at the 5% level so we conclude that there is not sufficient evidence to reject the null hypothesis that the operators have no effect on mean yields.

3. Reaction vessels

   The variance ratio is not significant at the 5% level so we conclude that there is not sufficient evidence to reject the null hypothesis that the reaction vessels have no effect on mean yields.

Note that, with only two degrees of freedom in the residual, this design gives poor power to the tests.

Exercises

1. Aluminium is produced industrially by electrolysis in reduction cells. The ‘current efficiency’ of a reduction cell is the yield of aluminium as a percentage of the yield predicted in an ideal cell by Faraday’s law. Improvements in the computer control system could improve current efficiency. Current efficiency also varies from one cell to another. In an experiment, three control schemes were compared. Each control scheme was applied to each of ten cells in random order. The current efficiency was measured over one week’s operation in each case. The data are as follows. Here the cells are ‘blocks.’ 

   Control	Cell (block)
   Scheme	1	2	3	4	5	6	7	8	9	10
   A	80.27	79.44	81.59	79.78	80.39	81.92	82.87	82.04	83.41	84.52
   B	84.31	83.33	86.57	84.49	84.15	85.45	85.05	83.62	85.96	85.62
   C	83.59	80.36	84.55	80.03	81.59	80.75	82.61	85.20	84.29	81.60

   What conclusions can be drawn about the effect of the control schemes on current efficiency? Use the 5% level of significance.

2. Plastic tubing is made by an extrusion process, starting with the plastic in pellet form. The tensile strength of the tubes may depend on which of four extrusion processes is used. Tubes were made using each of the four methods and samples tested for tensile strength. Four different batches of plastic pellets were used and each method was used once with plastic from each batch. The data are given below.

   Batch	Tensile Strength Method
   (Block)	1	2	3	4
   1	202	186	182	180
   2	200	195	224	199
   3	259	243	225	223
   4	252	227	276	251

   What conclusions may be drawn about the effect of the extrusion process on the resulting tubing? Use the 5% level of significance.

3. Crash tests with dummies are used to investigate the effects of different car seat-belt mechanisms. The response variable of interest is the maximum acceleration, in units of g, of the dummy’s head. Four different variations of the mechanism, A, B, C, D, are compared in a Latin Square design with four dummies and four different impact angles, but all at the same impact speed.

   The data are as follows:

   		Dummy
   		1	2	3	4
   		$B$	$C$	$D$	$A$
   	1	3.84	3.75	4.26	3.97
   		$A$	$B$	$C$	$D$
   Impact	2	3.44	3.93	4.18	3.36
   Angle		$C$	$D$	$A$	$B$
   	3	3.82	3.04	3.77	3.34
   		$D$	$A$	$B$	$C$
   	4	3.43	2.93	3.77	3.41

   Use an analysis of variance to test for the effects of the three factors. Use the 5% level of significance. Treat all three factors as “fixed.”

   What assumptions have to be made to justify your analysis?

   How might the design be altered to allow investigation of the possibility of interaction effects between the factors?

Answer