The theorem of total probability

1 The theorem of total probability

To establish this result we start with the definition of a partition of a sample space.

1.1 A partition of a sample space

The collection of events $A_{1}, A_{2}, \dots A_{n}$ is said to partition a sample space $S$ if

$A_{1} \cup A_{2} \cup \dots \cup A_{n} = S$
$A_{i} \cap A_{j} = \emptyset$ for all $i, j$
$A_{i} \neq \emptyset$ for all $i$

In essence, a partition is a collection of non-empty, non-overlapping subsets of a sample space whose union is the sample space itself. The definition is illustrated by Figure 10.

Figure 10

No alt text was set. Please request alt text from the person who provided you with this resource.

If $B$ is any event within $S$ then we can express $B$ as the union of subsets:

$B = (B \cap A_{1}) \cup (B \cap A_{2}) \cup \dots \cup (B \cap A_{n})$

The definition is illustrated in Figure 11 in which an event $B$ in $S$ is represented by the shaded region.

Figure 11

No alt text was set. Please request alt text from the person who provided you with this resource.

The bracketed events $(B \cap A_{1}), (B \cap A_{2}) \dots (B \cap A_{n})$ are mutually exclusive (if one occurs then none of the others can occur) and so, using the addition law of probability for mutually exclusive events:

$P (B) = P (B \cap A_{1}) + P (B \cap A_{2}) + \dots + P (B \cap A_{n})$

Each of the probabilities on the right-hand side may be expressed in terms of conditional probabilities:

$P (B \cap A_{i}) = P (B | A_{i}) P (A_{i}) for all i$

Using these in the expression for $P (B)$ , above, gives:

\begin{array}{rcl} P (B) & = & P (B | A_{1}) P (A_{1}) + P (B | A_{2}) P (A_{2}) + \dots + P (B | A_{n}) P (A_{n}) \\ = & \sum_{i = 1}^{n} P (B | A_{i}) P (A_{i}) \end{array}

This is the theorem of Total Probability. A related theorem with many applications in statistics can be deduced from this, known as Bayes’ theorem.