3 Do matrices always have an LU decomposition?

No. Sometimes it is impossible to write a matrix in the form “lower triangular" × “upper triangular" .

3.1 Why not?

An invertible matrix A has an L U decomposition provided that all its leading submatrices have non-zero determinants. The k th leading submatrix of A is denoted A k and is the k × k matrix found by looking only at the top k rows and leftmost k columns. For example if

A = 1 2 4 3 8 14 2 6 13

then the leading submatrices are

A 1 = 1 , A 2 = 1 2 3 8 , A 3 = 1 2 4 3 8 14 2 6 13 .

The fact that this matrix A has an L U decomposition can be guaranteed in advance because none of these determinants is zero:

A 1 = 1 , A 2 = ( 1 × 8 ) ( 2 × 3 ) = 2 , | A 3 | = 8 14 6 13 2 3 14 2 13 + 4 3 8 2 6 = 20 ( 2 × 11 ) + ( 4 × 2 ) = 6

(where the 3 × 3 determinant was found by expanding along the top row).

Example 7

Show that 1 2 3 2 4 5 1 3 4 does not have an L U decomposition.

Solution

The second leading submatrix has determinant equal to

1 2 2 4 = ( 1 × 4 ) ( 2 × 2 ) = 0

which means that an L U decomposition is not possible in this case.

Task!

Which, if any, of these matrices have an L U decomposition?  

  1. A = 3 2 0 1 ,
  2. A = 0 1 3 2 ,
  3. A = 1 3 7 2 6 1 0 3 2 .

| A 1 | = 3 and | A 2 | = | A | = 3 . Neither of these is zero, so A does have an L U decomposition.

A 1 = 0 so A does not have an L U decomposition.

| A 1 | = 1 , | A 2 | = 6 6 = 0 , so A does not have an L U decomposition.

3.2 Can we get around this problem?

Yes. It is always possible to re-order the rows of an invertible matrix so that all of the submatrices have non-zero determinants.

Example 8

Reorder the rows of A = 1 2 3 2 4 5 1 3 4 so that the reordered matrix has an L U decomposition.

Solution

Swapping the first and second rows does not help us since the second leading submatrix will still have a zero determinant. Let us swap the second and third rows and consider

B = 1 2 3 1 3 4 2 4 5

the leading submatrices are

B 1 = 1 , B 2 = 1 2 1 3 , B 3 = B .

Now | B 1 | = 1 , | B 2 | = 3 × 1 2 × 1 = 1 and (expanding along the first row)

B 3 = 1 ( 15 16 ) 2 ( 5 8 ) + 3 ( 4 6 ) = 1 + 6 6 = 1.

All three of these determinants are non-zero and we conclude that B does have an L U decomposition.

Task!

Reorder the rows of A = 1 3 7 2 6 1 0 3 2 so that the reordered matrix has an L U decomposition.

Let us swap the second and third rows and consider

B = 1 3 7 0 3 2 2 6 1

the leading submatrices are

B 1 = 1 , B 2 = 1 3 0 3 , B 3 = B

which have determinants 1 , 3 and 45 respectively. All of these are non-zero and we conclude that B does indeed have an L U decomposition.

Exercises
  1. Calculate L U decompositions for each of these matrices
    1. A = 2 1 4 6
    2. A = 2 1 4 2 2 2 6 3 11
    3. A = 1 3 2 2 8 5 1 11 4
  2. Check each answer in Question 1, by multiplying out L U to show that the product equals A .
  3. Using the answers obtained in Question 1, solve the following systems of equations.
    1. 2 1 4 6 x 1 x 2 = 1 2
    2. 2 1 4 2 2 2 6 3 11 x 1 x 2 x 3 = 4 0 11
    3. 1 3 2 2 8 5 1 11 4 x 1 x 2 x 3 = 2 3 0
  4. Consider A = 1 6 2 2 12 5 1 3 1
    1. Show that A does not have an L U decomposition.
    2. Re-order the rows of A and find an L U decomposition of the new matrix.
    3. Hence solve x 1 + 6 x 2 + 2 x 3 = 9 2 x 1 + 12 x 2 + 5 x 3 = 4 x 1 3 x 2 x 3 = 17
    1. We let

      2 1 4 6 = L U = 1 0 L 21 1 U 11 U 12 0 U 22 = U 11 U 12 L 21 U 11 L 21 U 12 + U 22 .

      Comparing the left-hand and right-hand sides row by row gives us that U 11 = 2 , U 12 = 1 , L 21 U 11 = 4 which implies that L 21 = 2 and, finally, L 21 U 12 + U 22 = 6 from which we see that U 22 = 4 . Hence

      2 1 4 6 = 1 0 2 1 2 1 0 4

      is an L U decomposition of the given matrix.

    2. We let

      2 1 4 2 2 2 6 3 11 = L U = U 11 U 12 U 13 L 21 U 11 L 21 U 12 + U 22 L 21 U 13 + U 23 L 31 U 11 L 31 U 12 + L 32 U 22 L 31 U 13 + L 32 U 23 + U 33 .

      Looking at the top row we see that U 11 = 2 , U 12 = 1 and U 13 = 4 . Now, from the second row, L 21 = 1 , U 22 = 1 and U 23 = 2 . The last three unknowns come from the bottom row: L 31 = 3 , L 32 = 0 and U 33 = 1 . Hence

      2 1 4 2 2 2 6 3 11 = 1 0 0 1 1 0 3 0 1 2 1 4 0 1 2 0 0 1

      is an L U decomposition of the given matrix.

    3. We let

      1 3 2 2 8 5 1 11 4 = L U = U 11 U 12 U 13 L 21 U 11 L 21 U 12 + U 22 L 21 U 13 + U 23 L 31 U 11 L 31 U 12 + L 32 U 22 L 31 U 13 + L 32 U 23 + U 33 .

      Looking at the top row we see that U 11 = 1 , U 12 = 3 and U 13 = 2 . Now, from the second row, L 21 = 2 , U 22 = 2 and U 23 = 1 . The last three unknowns come from the bottom row: L 31 = 1 , L 32 = 4 and U 33 = 2 . Hence

      1 3 2 2 8 5 1 11 4 = 1 0 0 2 1 0 1 4 1 1 3 2 0 2 1 0 0 2

      is an L U decomposition of the given matrix.

  1. Direct multiplication provides the necessary check.
    1. We begin by solving

      1 0 2 1 y 1 y 2 = 1 2

      Clearly y 1 = 1 and therefore y 2 = 4 . The values y 1 and y 2 appear on the right-hand side of the second system we need to solve:

      2 1 0 4 x 1 x 2 = 1 4

      The second equation implies that x 2 = 1 and therefore, from the first equation, x 1 = 1 .

    2. We begin by solving the system

      1 0 0 1 1 0 3 0 1 y 1 y 2 y 3 = 4 0 11 .

      Starting with the top equation we see that y 1 = 4 . The second equation then implies that y 2 = 4 and then, from the third equation, y 3 = 1 . These values now appear on the right-hand side of the second system

      2 1 4 0 1 2 0 0 1 x 1 x 2 x 3 = 4 4 1 .

      The bottom equation shows us that x 3 = 1 . Moving up to the middle equation we obtain x 2 = 2 . The top equation yields x 1 = 1 .

    3. We begin by solving the system

      1 0 0 2 1 0 1 4 1 y 1 y 2 y 3 = 2 3 0 .

      Starting with the top equation we see that y 1 = 2 . The second equation then implies that y 2 = 1 and then, from the third equation, y 3 = 2 . These values now appear on the right-hand side of the second system

      1 3 2 0 2 1 0 0 2 x 1 x 2 x 3 = 2 1 2 .

      The bottom equation shows us that x 3 = 1 . Moving up to the middle equation we obtain x 2 = 0 . The top equation yields x 1 = 4 .

    1. The second leading submatrix has determinant 1 × 12 6 × 2 = 0 and this implies that A has no L U decomposition.
    2. Swapping the second and third rows gives 1 6 2 1 3 1 2 12 5 . We let

      1 6 2 1 3 1 2 12 5 = L U = U 11 U 12 U 13 L 21 U 11 L 21 U 12 + U 22 L 21 U 13 + U 23 L 31 U 11 L 31 U 12 + L 32 U 22 L 31 U 13 + L 32 U 23 + U 33 .

      Looking at the top row we see that U 11 = 1 , U 12 = 6 and U 13 = 2 . Now, from the second row, L 21 = 1 , U 22 = 3 and U 23 = 1 . The last three unknowns come from the bottom row: L 31 = 2 , L 32 = 0 and U 33 = 1 . Hence

      1 6 2 1 3 1 2 12 5 = 1 0 0 1 1 0 2 0 1 1 6 2 0 3 1 0 0 1

      is an L U decomposition of the given matrix.

    3. We begin by solving the system

      1 0 0 1 1 0 2 0 1 y 1 y 2 y 3 = 9 17 4 .

      (Note that the second and third rows of the right-hand side vector have been swapped too.) Starting with the top equation we see that y 1 = 9 . The second equation then implies that y 2 = 26 and then, from the third equation, y 3 = 22 . These values now appear on the right-hand side of the second system

      1 6 2 0 3 1 0 0 1 x 1 x 2 x 3 = 9 26 22 .

      The bottom equation shows us that x 3 = 22 . Moving up to the middle equation we obtain x 2 = 16 . The top equation yields x 1 = 43 .