1 Goodness-of-fit tests
The aim of a goodness-of-fit test is to determine the underlying nature of the probability distribution describing the population from which a random sample has been drawn. For example, we may wish to determine whether the population from which a sample has been drawn has a normal, binomial or Poisson distribution. While a variety of goodness-of-fit tests exist, the test described here depends on the -distribution and is usually called the chi-squared test.
We assume that a random sample of size has been drawn from a population with an unknown probability distribution and that we wish to determine the nature of that distribution.
- Firstly, if the data are continuous we organize the data into intervals (often equal but not necessarily so) in order that we can write down the observed frequency, say , of the th interval for .
- Secondly, we form a hypothesis about the nature of the unknown distribution. That is, we assume that it is normal, binomial, Poisson or some other appropriate probability distribution.
-
Thirdly, we calculate, on the basis of the hypothesis outlined above, the expected
frequency, say
,
of the
th
interval for
.
The values of
are calculated using the formula
where is the probability associated with the interval .
- Fourthly, we calculate the goodness-of-fit statistic as defined in Key Point 1.
It can be shown that, if the assumption made about the nature of the population (normal, binomial, Poisson etc.) is true then follows (approximately) a chi-squared distribution with degrees of freedom. Note that represents the number of parameters needed to describe the probability distribution of the population which we have to estimate from the data. For example the normal distribution has two parameters and , the binomial distribution has two parameters and but we usually only need to estimate , while the Poisson distribution has one parameter, .
- Fifthly, we reject the hypothesis concerning the nature of the underlying probability distribution if the calculated value of exceeds the value of where is the area in the tail of the -distribution, typically 5% or 1%.
Notes
- The larger the sample, the more reliable the result since the assertion that follows (approximately) a chi-squared distribution improves with increasing sample size.
- The size of the expected frequencies should be monitored carefully. Various authors recommend that minimum expected frequencies of 3, 4 or 5 are acceptable. It is reasonably safe to accept expected frequencies provided that they are greater than 5 and 10 is certainly acceptable.
- Some authors recommend that the intervals into which the data are organized are chosen so that the frequencies in each interval are roughly equal - remember that equal intervals are not necessary for the test to be performed.
We will now look at two examples of goodness-of-fit tests, the first uses a (discrete) Poisson distribution and the second uses a (continuous) normal distribution. Each worked Example is immediately followed by a Task for you to do.
Example 1
A manufacturer produces high-quality sheet aluminium for use in highly stressed aircraft wings. A random sample of 100 sheets is inspected and the number of faults per sheet recorded. The results are given in the table below.
Number of Faults per Sheet | Frequency of Occurrence |
0 | 50 |
1 | 24 |
2 | 14 |
3 | 8 |
4 | 4 |
Suggest a possible probability distribution from which the sample may have been drawn and perform a chi-squared test to determine the validity of your suggestion.
Solution
The data are already given in 5 classes with observed frequencies as shown. We will assume that the underlying distribution is Poisson and calculate the expected frequencies accordingly using the Poisson formula We need the value of the mean.
This is calculated as
Hence the Poisson probabilities and the corresponding expected frequencies are:
Note that in calculating we have ensured that our probabilities sum to unity.
Since the last frequency is very small we will combine the last two and use 4 classes so that and .
The test statistic is
and the number of degrees of freedom is so that the critical value from Table 1 (at the end of the Workbook) is . Clearly and we must reject the null hypothesis that the underlying distribution is Poisson.
Task!
A manufacturer produces electronic components for use in computer controlled monitoring systems. A random sample of 100 components is inspected and the number of faults per component recorded. The results are given in the table below.
Number of Faults per Component | Frequency of Occurrence |
0 | 45 |
1 | 35 |
2 | 16 |
3 | 4 |
Perform a chi-squared test to determine the validity of the assumption that the occurrence of faults in the components is Poisson.
The data are given in 4 classes with observed frequencies as shown. The expected frequencies using the Poisson formula with a mean
are
The last frequency is small but since it is greater than 3 we will allow its use.
The test statistic is
and the number of degrees of freedom is so that the critical value from tables is . Clearly and we accept the null hypothesis that the underlying distribution is Poisson. Note that the decision to accept the value is fairly marginal and that some personal judgement in such situations as to whether such values should be accepted or combined with another class is unavoidable.
Task!
Using the data of the previous Task but combining the expected frequencies of the last two classes, perform a chi-squared test to determine the validity of the assumption that the occurrence of faults in the components is Poisson.
The data are given in 4 classes with observed frequencies as shown. The expected frequencies using the Poisson formula with a mean
are
We will combine the expected frequencies of the last two classes and use 3 classes in total with expected frequencies of .
The test statistic is
and the number of degrees of freedom is so that the critical value from Table 1 (at the end of the Workbook) is . Clearly and we accept the null hypothesis that the underlying distribution is Poisson. Note that the decision to combine the last two classes has not, in this case, affected the acceptance of the null hypothesis.
Example 2
A quality control engineer is given the job of checking the voltage output characteristics of a circuit component in a CD player. After checking 100 randomly selected components and plotting a histogram of the results, the engineer concludes that the mean output of the 100 checked components is volts, that the standard deviation is volts and that the voltage distribution is probably normal. Choose a suitable test to decide whether the assumption of normality is valid at the 5% level of significance.
Solution
The engineer decides to use a chi-squared test to test the assumption of normality and follow the (common) practice of ensuring that the expected frequencies are equal. To do this, the data are put into eight equal classes and the class boundaries calculated as follows.
From the standard normal distribution the values corresponding to class boundaries giving a probability of 0.125 (i.e. 1/8) may be read off from tables as and for positive values and and for negative values. Using
the class boundaries are calculated to be: . This gives the eight classes, the observed frequencies found by the engineer (you are given this information here), and the expected frequencies as:
Classes | Observed Frequencies | Expected Frequencies |
8 | 12.5 | |
11 | 12.5 | |
16 | 12.5 | |
19 | 12.5 | |
18 | 12.5 | |
13 | 12.5 | |
9 | 12.5 | |
6 | 12.5 |
The hypotheses are: : distribution is normal, distribution is not normal
The test statistic is
and the number of degrees of freedom is so that the critical value from Table 1 is .
Since we have sufficient evidence to reject the null hypothesis and so the engineer should conclude that the distribution of voltages is not normal.
Task!
An electrical engineer working for a Health and Safety Executive measures the radiation emitted through the closed doors of 100 used microwave ovens. The measurements, in , are given in the table below.
0.19 | 0.16 | 0.14 | 0.20 | 0.17 | 0.21 | 0.18 | 0.22 | 0.26 | 0.23 |
0.13 | 0.17 | 0.16 | 0.21 | 0.18 | 0.22 | 0.20 | 0.23 | 0.16 | 0.26 |
0.19 | 0.16 | 0.14 | 0.20 | 0.18 | 0.21 | 0.19 | 0.22 | 0.27 | 0.24 |
0.12 | 0.17 | 0.15 | 0.20 | 0.18 | 0.22 | 0.19 | 0.23 | 0.29 | 0.25 |
0.06 | 0.16 | 0.14 | 0.20 | 0.17 | 0.21 | 0.18 | 0.22 | 0.26 | 0.23 |
0.13 | 0.17 | 0.16 | 0.20 | 0.18 | 0.22 | 0.19 | 0.23 | 0.30 | 0.25 |
0.19 | 0.17 | 0.14 | 0.20 | 0.18 | 0.21 | 0.19 | 0.22 | 0.27 | 0.24 |
0.11 | 0.17 | 0.15 | 0.20 | 0.18 | 0.21 | 0.19 | 0.23 | 0.27 | 0.24 |
0.13 | 0.17 | 0.16 | 0.21 | 0.18 | 0.22 | 0.19 | 0.23 | 0.33 | 0.25 |
0.13 | 0.17 | 0.16 | 0.21 | 0.18 | 0.22 | 0.19 | 0.23 | 0.36 | 0.26 |
The mean radiation of the checked ovens is , and the standard deviation is . Verify that the table below giving the eight classes corresponding to the observed and expected frequencies shown is correct.
Classes | Observed Frequencies | Expected Frequencies |
11 | 12.5 | |
10 | 12.5 | |
19 | 12.5 | |
10 | 12.5 | |
16 | 12.5 | |
17 | 12.5 | |
6 | 12.5 | |
11 | 12.5 |
Use a chi-squared test to decide whether the radiation readings obtained from the ovens are normally distributed at the 5% level of significance.
Although the choice of class boundaries is arbitrary, for convenience we choose boundaries to make eight classes with equal probabilities of 0.125.
From the standard normal distribution the values corresponding to class boundaries giving a probability of 0.125 may be read off from tables as and for positive values and and for negative values. Using
the class boundaries are calculated to be:
This gives the eight classes, the observed frequencies found by the engineer and the expected frequencies as given in the table above.
The hypotheses are: : distribution is normal, distribution is not normal.
The test statistic is
and the number of degrees of freedom is so that the critical value from Table 1 is .
Since we do not have sufficient evidence to reject the null hypothesis and so the engineer should conclude that the distribution of microwave radiation readings taken from the ovens is normal.
Exercises
-
A factory produces portable CD players. Every week a sample of ten players is selected
and subjected to 100 hours of continuous use. At the end of this time the players are
tested and the number not reaching a specified standard is recorded. The numbers
recorded in 100 consecutive weeks are given below. Test the hypothesis that the data
come from a binomial distribution. Use the 5% level of significance.
Number failing standard 0 1 2 3 4 5 Number of weeks 34 24 19 14 9 0 -
A highway engineer records the numbers of vehicles passing a point in a road in 120
consecutive one-minute intervals, as follows. Test the hypothesis that the data come from a
Poisson distribution. Use the 5% level of significance.
Number of vehicles 0 1 2 3 4 5 6 7 8 9 10 11 Number of intervals 0 5 10 20 30 20 15 7 6 4 2 1 -
In a test of a device to generate electricity from wave power at sea,
60 observations are made of the root mean square bending moment
of a component (in newton metres). The data are summarised as follows. The
sample mean is 5.08 and the sample variance is 3.29. Test the hypothesis that
has a normal distribution. Use the 5% level of significance.
Class Frequency Class Frequency 1 5 4 4 12 2 18 2 11 1 -
Eighty aircraft components are tested until they fail. The failure times
in
hours are summarised as follows. The sample mean is 6434. Test the hypothesis that the distribution
of
is
exponential. Use the 5% level of significance.
Class Frequency Class Frequency 11 3 21 5 19 1 9 3 4 4
-
Total number of failures:
Mean number of failures per week:
Estimate of
Use binomial distribution.
No. failing Probability FrequencyExpected Observed 0 0.1935 19.35 34 1 0.3762 37.62 24 2 0.2926 29.26 19 3 0.1138 11.38 14 4 0.0221 2.21 9 5 0.0017 0.17 0 Some expected frequencies are too small so we combine neighbouring classes.
No. failing Probability FrequencyExpected Observed 0 0.1935 19.35 34 1 0.3762 37.62 24 2 0.2926 29.26 19 3,4,5 0.1376 13.76 23 Test statistic:
Degrees of freedom: (4 classes, 1 estimated parameter).
Critical value:
The test statistic is significant at the 5% level. We reject the null hypothesis. We conclude that the data do not come from a binomial distribution. There seems to be an excess of large and small counts.
-
Total number of vehicles:
Mean number of vehicles per minute:
Use Poisson distribution.
No. vehicles Probability FrequencyExpected Observed 0 0.00949 1.14 0 1 0.04418 5.30 5 2 0.10290 12.35 10 3 0.15977 19.17 20 4 0.18606 22.33 30 5 0.17333 20.80 20 6 0.13456 16.15 15 7 0.08954 10.74 7 8 0.05214 6.26 6 9 0.02698 3.24 4 10 0.01257 1.51 2 0.00848 1.02 1 Some expected frequencies are too small so we combine neighbouring classes.
No. vehicles Probability FrequencyExpected Observed 0,1 0.05367 6.44 5 2 0.10290 12.35 10 3 0.15977 19.17 20 4 0.18606 22.33 30 5 0.17333 20.80 20 6 0.13456 16.15 15 7 0.08954 10.74 7 8 0.05214 6.26 6 0.04803 5.76 7 Test statistic:
Degrees of freedom: (9 classes, 1 estimated parameter).
Critical value:
The test statistic is not significant at the 5% level. We do not reject the null hypothesis. There is insufficient evidence to conclude that the data do not come from a Poisson distribution.
-
Using a
distribution we can calculate the probabilities for the various class intervals. For
example,
Bending moment Probability FrequencyExpected Observed 0.045 2.70 1 0.081 4.86 4 0.150 9.00 12 0.206 12.36 18 0.212 12.72 11 0.161 9.66 5 0.091 5.46 4 0.039 2.34 2 0.012 0.72 2 0.003 0.18 1 Some expected frequencies are too small so we combine neighbouring classes.
Bending moment Probability FrequencyExpected Observed 0.126 7.56 5 0.150 9.00 12 0.206 12.36 18 0.212 12.72 11 0.161 9.66 5 0.091 5.46 4 0.054 3.24 5 Test statistic:
Degrees of freedom: (7 classes, 2 estimated parameters).
Critical value:
The test statistic is not significant at the 5% level. We do not reject the null hypothesis. There is insufficient evidence to conclude that the data do not come from a normal distribution.
-
The sample mean is 6434. We estimate
using
We use
an exponential
distribution. For example
Failure time Probability FrequencyExpected Observed 0.267 21.36 11 0.196 15.68 21 0.143 11.44 19 0.106 8.48 9 0.077 6.16 4 0.056 4.48 3 0.041 3.28 5 0.031 2.48 1 0.022 1.76 3 0.061 4.88 4 Some expected frequencies are too small so we combine neighbouring classes.
Failure time Probability FrequencyExpected Observed 0.267 21.36 11 0.196 15.68 21 0.143 11.44 19 0.106 8.48 9 0.077 6.16 4 0.056 4.48 3 0.041 3.28 5 0.053 4.24 4 0.061 4.88 4 Test statistic:
Degrees of freedom: (9 classes, 1 estimated parameter).
Critical value:
The test statistic is significant at the 5% level. We reject the null hypothesis. We conclude that the data do not come from an exponential distribution. The observed frequency in the first class seems to be too small.