1 Correlation

So far we have assumed that we have a random variable $Y$ related to an independent variable $x$ which can be measured with some accuracy. In the equation below, the dependent variable $Y$ is a random variable whose value, for a fixed value of $x$ depends on a random error component say $e$ and we have

$Y=mx+c+e$

In some situations, both $X$ and $Y$ are random variables and you should note that we can still use a regression line of $y$ on $x$ if we are required to predict values of $y$ from observations made on $x$ . In this case the variables $x$ and $y$ play different roles. In correlation, the two variables are interchangeable. Examples involving two random variables often quoted are the shear strength ( $y$ ) and diameter of spot welds ( $x$ ) (neither can be precisely controlled) and the bending moment ( $y$ ) and shear ( $x$ ) at the fixed point of a beam as illustrated below

Figure 6

Again, neither variable (shear or moment) can be precisely controlled, each is a random variable. In cases such as these, we turn to the correlation coefficient (sometimes called Pearson’s coefficient of correlation or simply Pearson’s $r$ ) defined as

$r=\frac{{\sigma }_{xy}}{{\sigma }_x{\sigma }_y}$

where ${\sigma }_{xy}$ is the covariance between $X$ and $Y$ and ${\sigma }_x$ and ${\sigma }_y$ are the standard deviations of $X$ and $Y$ . We need to express this formula in terms of quantities which facilitate the easy calculation of the correlation coefficient.

Further, it can also be shown that $-1\le r\le 1$ and that:

1. $r=-1$ represents perfect negative correlation with all $\left(x,y\right)$ lying on a straight line with negative gradient;
2. $r=1$ represents perfect positive correlation with all $\left(x,y\right)$ lying on a straight line with positive gradient;
3. $r=0$ represents the situation where either there is no linear relationship between the variables or that any relationship existing is non-linear.

1.1 The calculation of Pearson’s $r$

The worked example below shows the setting out of a table which will facilitate the easy calculation of Pearson’s $r$ .

Example 3

Find the value of Pearson’s $r$ for the following set of data obtained by reading seven torque values ( $x$ ) from an electric motor using current ( $y$ ).

Solution

The calculation is done as follows:

Substituting in the formula we developed for $r$ gives the result:

$r=\frac{752\times 7-70\times 70}{\sqrt{\left(7\times 812-70^2\right)\left(7\times 812-70^2\right)}}=0.46$

In practice, one would set up a spreadsheet or use a specialist statistical software package to do the calculations.

Comment

Any value of $r$ calculated says something about the degree of correlation present between the two independent random variables present in the calculation. In order to give real meaning to the value of the correlation coefficient we should test the significance of the value of $r$ , in this case 0.46.

1.2 The significance of Pearson’s $r$

In order to test the significance of a calculated value of $r$ we assume that both $x$ and $y$ are normally distributed and set up the hypotheses:

$H_0:\rho =0H_1:\rho \ne 0$

where $\rho$ is the ‘true’ value of the population correlation. If the assumption of normality is false the test must not be used. We know that the value of $-1\le r\le 1$ and we wish to know whether our correlation coefficient is significantly different to zero.

Note that many authors simply miss out the modulus sign and ignore the sign of $r$ should it be negative. The test statistic is then written

$r_{test}=\frac{r\sqrt{n-2}}{\sqrt{1-r^2}}$

and critical values depending on the level of significance required are read off from $t$ -tables in the usual way. A copy of $t$ -distribution tables is included at the end of this Workbook (Table 2).

Example 4

Test the significance of the value of $r$ obtained from Example 3 concerning electric motor torque values. Use the 5% level of significance.

Solution

The sample size is 7 so we have 5 degrees of freedom. The value of $r_{test}$ is given by

$r_{test}=\frac{r\sqrt{n-2}}{\sqrt{1-r^2}}=\frac{0.46\times \sqrt{7-2}}{\sqrt{1-0.46^2}}=1.158$

From Table 2, the critical value for a two-sided test at the 5% level of significance is 2.571. In this case, since $1.158<2.571$ we cannot reject the null hypothesis at the 5% level of significance and conclude that for the motor under investigation, there is no evidence of a relationship between torque produced and current used.

Task!

Hooke’s law relates the extension of a spring under load to its extended length. The following results were obtained experimentally.

Calculate Pearson’s r and test its significance at the 5% level. What conclusion can you draw?

Answer

Comments on interpretation

Some care should always be taken when interpreting results obtained from correlation coefficient calculations.

1. A high correlation does not necessarily imply that a causal relationship exists between the variables considered. For example, it may be that a high degree of correlation exists between the number of road accidents in a particular city and the number of late trains arriving at a station in another city both over the same time period. In general one would not expect to find a causal relation between the variables involved. Similar comments apply to, for example, water hardness and average income for towns in the UK.
2. When considering the behaviour of two variables, one should realize that it is possible that both variables may change because of the influence of a third variable. An example often quoted in this context is the Gas law

   $\frac{PV}{T}=\text{constant}$

   where say, pressure and volume may change because of a change in temperature.

3. A low value of the correlation coefficient does not necessarily imply that no relationship exists between the variables being considered. Remember that the correlation coefficient is indicative of a linear relationship only and that a low or zero value of $r$ may indicate that a non-linear relationship exists. For example a set of points lying on the curve $y=x^2$ might (see the Tasks below) result in a zero value of $r$ .

Task!

Write down five $\left(x,y\right)$ points (symmetrical about zero) lying on the parabola $y=x^2$ . Show that the correlation coefficient between $x$ and $y$ is zero.

Answer

Task!

Write down five $\left(x,y\right)$ points (all involving positive values of $x$ and $y$ ) lying on the parabola $y=x^2$ . Show that the correlation coefficient between $x$ and $y$ is non-zero.

Answer

1.3 Spearman’s coefficient of correlation

There are times when data cannot be expressed in terms of numbers directly. For example, an audio engineer might be asked to give an opinion on the quality of sound produced by three sets of speakers. The results will represent a judgement made by the engineer. The engineer could adopt a set of criteria including, for example, the clarity of the treble, the power of the base and the ability of the speakers to distinguish between instruments. Suppose the results are as follows:

Note that the results are not numeric in an arithmetic sense so you cannot do meaningful arithmetic using the results. In order to see this, just ask what a calculation based on the ranks such as

$\frac{1+2^2}{3}$

would actually mean. The answer is, of course, nothing!

During your career as an engineer you may be asked to rank data in a similar way to that outlined above. You may be asked to assess the work of colleagues for promotion purposes or give an opinion on the visual appeal of alternative designs of manufactured objects such as mobile telephones, food containers or television sets.

Assigning numbers to data in order of size (often called ranking methods) can also be useful if one does not wish to make assumptions about the nature of the distributions underlying the data. (For example whenever at least one of the distributions describing the behaviour of the variables may not be normal.) In order to check the level of correlation between results obtained by ranking data we calculate Spearman’s coefficient of correlation.

The formula indicates that the differences of each pair of ranked values are to be found, squared and summed. It is worth noting that even though it is not obvious, Spearman’s coefficient is just Pearson’s coefficient applied to ranks.

1.4 The calculation of Spearman’s $R$

The following worked example illustrates the procedure.

Example 5

A production engineer is asked to grade, on the basis of 12 criteria $A$ to $L$ , a junior colleague who has applied for promotion. In order to try to ensure that he treats the colleague fairly, the engineer repeats his gradings after a few days. On the basis of the results below, can you conclude that the results are consistent? The gradings are percentages.

Solution

The calculation may be set out as follows:

Substituting in the formula for $R$ gives the value

$R=1-\frac{6\times 38}{12\times 143}=0.87$

Note that we have not made any attempt to interpret the meaning of this figure of 0.87. Methods for doing this are discussed below.

1.5 The significance of spearman’s $R$

Like Pearson’s $r$ the value of $R$ may be shown to lie in the range $-1\le R\le 1$ and in order to test the significance of a calculated value of $R$ we set up the hypotheses

$H_0:\rho =0H_1:\rho \ne 0$

Critical values depending on the level of significance required are read from $t$ -tables. When dealing with Spearman’s coefficient of correlation, the size of the sample is important. Different authors recommend different minimum sample sizes, a common recommendation being a minimum of $n=10$ . Even though they are not used here, you should note that tables are available which allow us to read critical values corresponding to small sample sizes.

Example 6

A production engineer is asked to grade, on the basis of 12 criteria (say) $A$ to $L$ a junior colleague who has applied for promotion. He repeats his gradings after a few days. The results (calculated in Example 5) gave a value of $R=0.87$ . Test at the 5% level to determine whether the results are consistent.

Solution

The calculation is:

$R_{test}=\frac{R\sqrt{n-2}}{\sqrt{1-R^2}}=\frac{0.87\times \sqrt{12-2}}{\sqrt{1-0.87^2}}=5.580$

The 5% critical value for a two sided test read from tables is 2.228 and since $5.580>2.228$ we conclude that we must reject the null hypothesis that the correlation coefficient is zero.

Task!

As a result of two tests given to 10 students studying laboratory safety, the students were placed in the following class order.

Use Spearman’s $R$ to discuss the consistency of their performances. Can you make any meaningful comment regarding the two tests as a means of assessing laboratory safety?

Answer

As part of an educational research project, twelve engineering students were given an intelligence test (IQ score) at the start of their first year course. At the end of the first year their results in engineering science (ES score) were noted down on the expectation that they would correlate with the results of the intelligence test. The results were as follows:

Calculate Pearson’s r for these data. Can you conclude that there is a linear relationship between IQ scores and ES scores? You may assume that the IQ scores and the ES scores are each normally distributed.

Answer

1.7 Table 2: Critical points of student’s $t$ distribution