The theorem of total probability

1 The theorem of total probability

To establish this result we start with the definition of a partition of a sample space.

1.1 A partition of a sample space

The collection of events $A_{1}, A_{2}, \dots A_{n}$ is said to partition a sample space $S$ if

$A_{1} \cup A_{2} \cup \dots \cup A_{n} = S$
$A_{i} \cap A_{j} = \emptyset$ for all $i, j$
$A_{i} \neq \emptyset$ for all $i$

In essence, a partition is a collection of non-empty, non-overlapping subsets of a sample space whose union is the sample space itself. The definition is illustrated by Figure 10.

Figure 10

A diagram illustrating a rectangular sample space S divided into several overlapping and non-overlapping regions labeled A_1 through A_6.

Long description

If $B$ is any event within $S$ then we can express $B$ as the union of subsets:

$B = (B \cap A_{1}) \cup (B \cap A_{2}) \cup \dots \cup (B \cap A_{n})$

The definition is illustrated in Figure 11 in which an event $B$ in $S$ is represented by the shaded region.

Figure 11

A diagram illustrates a set labeled B overlapping with several other labeled sets within a larger region S.

Long description

The bracketed events $(B \cap A_{1}), (B \cap A_{2}) \dots (B \cap A_{n})$ are mutually exclusive (if one occurs then none of the others can occur) and so, using the addition law of probability for mutually exclusive events:

$P (B) = P (B \cap A_{1}) + P (B \cap A_{2}) + \dots + P (B \cap A_{n})$

Each of the probabilities on the right-hand side may be expressed in terms of conditional probabilities:

$P (B \cap A_{i}) = P (B | A_{i}) P (A_{i}) for all i$

Using these in the expression for $P (B)$ , above, gives:

\begin{array}{rcl} P (B) & = & P (B | A_{1}) P (A_{1}) + P (B | A_{2}) P (A_{2}) + \dots + P (B | A_{n}) P (A_{n}) \\ = & \sum_{i = 1}^{n} P (B | A_{i}) P (A_{i}) \end{array}

This is the theorem of Total Probability. A related theorem with many applications in statistics can be deduced from this, known as Bayes’ theorem.