1 Matrix norms

The norm of a square matrix $A$ is a non-negative real number denoted $∥A∥$ . There are several different ways of defining a matrix norm, but they all share the following properties:

1. $∥A∥\ge 0$ for any square matrix $A$ .
2. $∥A∥=0$ if and only if the matrix $A=0$ .
3. $\parallel kA\parallel =\left|k\right|\parallel A\parallel$ , for any scalar $k$ .
4. $\parallel A+B\parallel \le \parallel A\parallel +\parallel B\parallel$ .
5. $\parallel AB\parallel \le \parallel A\parallel \parallel B\parallel$ .

The norm of a matrix is a measure of how large its elements are. It is a way of determining the “size" of a matrix that is not necessarily related to how many rows or columns the matrix has.

Anticipating the places where we will use norms later, it is sufficient at this stage to restrict our attention to matrices with only real-valued entries. There is no need to consider complex numbers at this stage.

In the definitions of norms below we will use this notation for the elements of an $n\times n$ matrix $A$ where

$A=\left[\begin{array}{ccccc}\hfill a_{11}\hfill & \hfill a_{12}\hfill & \hfill a_{13}\hfill & \hfill \dots \hfill & \hfill a_{1n}\hfill \\ \hfill a_{21}\hfill & \hfill a_{22}\hfill & \hfill a_{23}\hfill & \hfill \dots \hfill & \hfill a_{2n}\hfill \\ \hfill a_{31}\hfill & \hfill a_{32}\hfill & \hfill a_{33}\hfill & \hfill \dots \hfill & \hfill a_{3n}\hfill \\ \hfill ⋮\hfill & \hfill ⋮\hfill & \hfill ⋮\hfill & \hfill \ddots \hfill & \hfill ⋮\hfill \\ \hfill a_{n1}\hfill & \hfill a_{n2}\hfill & \hfill a_{n3}\hfill & \hfill \dots \hfill & \hfill a_{nn}\hfill \end{array}\right]$

The subscripts on $a$ have the r ow number first, then the c olumn number. The fact that

$a_{rc}$

is reminiscent of the word “arc" may be a help in remembering how the notation goes.

In this Section we will define three commonly used norms. We distinguish them with a subscript. All three of them satisfy the five conditions listed above, but we will not concern ourselves with verifying that fact.

1.1 The 1-norm

${∥A∥}_1=\underset{1\le j\le n}{max}\left({\sum }_{i=1}^n\left|a_{ij}\right|\right)$

(the maximum absolute column sum). Put simply, we sum the absolute values down each column and then take the biggest answer.

Example 9

Calculate the 1-norm of $A=\left[\begin{array}{cc}\hfill 1\hfill & \hfill -7\hfill \\ \hfill -2\hfill & \hfill -3\hfill \end{array}\right]$ .

Solution

The absolute column sums of $A$ are $1+|-2|=1+2=3$ and $|-7|+|-3|=7+3=10$ . The larger of these is 10 and therefore $\parallel A{\parallel }_1=10$ .

Example 10

Calculate the 1-norm of $B=\left[\begin{array}{ccc}\hfill 5\hfill & \hfill -4\hfill & \hfill 2\hfill \\ \hfill -1\hfill & \hfill 2\hfill & \hfill 3\hfill \\ \hfill -2\hfill & \hfill 1\hfill & \hfill 0\hfill \end{array}\right]$ .

Solution

Summing down the columns of $B$ we find that

1.2 The infinity-norm

${∥A∥}_{\infty }=\underset{1\le i\le n}{max}\left({\sum }_{j=1}^n\left|a_{ij}\right|\right)$

(the maximum absolute row sum). Put simply, we sum the absolute values along each row and then take the biggest answer.

Example 11

Calculate the infinity-norm of $A=\left[\begin{array}{cc}\hfill 1\hfill & \hfill -7\hfill \\ \hfill -2\hfill & \hfill -3\hfill \end{array}\right]$ .

Solution

The absolute row sums of $A$ are $1+|-7|=8$ and $|-2|+|-3|=5$ . The larger of these is 8 and therefore $\parallel A{\parallel }_{\infty }=8$ .

Example 12

Calculate the infinity-norm of $B=\left[\begin{array}{ccc}\hfill 5\hfill & \hfill -4\hfill & \hfill 2\hfill \\ \hfill -1\hfill & \hfill 2\hfill & \hfill 3\hfill \\ \hfill -2\hfill & \hfill 1\hfill & \hfill 0\hfill \end{array}\right]$ .

Solution

Summing along the rows of $B$ we find that

1.3 The Euclidean norm

${∥A∥}_E=\sqrt{{\sum }_{i=1}^n{\sum }_{j=1}^n{\left(a_{ij}\right)}^2}$

(the square root of the sum of all the squares). This is similar to ordinary “Pythagorean" length where the size of a vector is found by taking the square root of the sum of the squares of all the elements.

Example 13

Calculate the Euclidean norm of $A=\left[\begin{array}{cc}\hfill 1\hfill & \hfill -7\hfill \\ \hfill -2\hfill & \hfill -3\hfill \end{array}\right]$ .

Solution

Example 14

Calculate the Euclidean norm of $B=\left[\begin{array}{ccc}\hfill 5\hfill & \hfill -4\hfill & \hfill 2\hfill \\ \hfill -1\hfill & \hfill 2\hfill & \hfill 3\hfill \\ \hfill -2\hfill & \hfill 1\hfill & \hfill 0\hfill \end{array}\right]$ .

Solution

Task!

Calculate the norms indicated of these matrices

$A=\left[\begin{array}{cc}\hfill 2\hfill & \hfill -8\hfill \\ \hfill 3\hfill & \hfill 1\hfill \end{array}\right]\text{(1-norm)},B=\left[\begin{array}{ccc}\hfill 3\hfill & \hfill 6\hfill & \hfill -1\hfill \\ \hfill 3\hfill & \hfill 1\hfill & \hfill 0\hfill \\ \hfill 2\hfill & \hfill 4\hfill & \hfill -7\hfill \end{array}\right]\text{(infinity-norm)},$

$C=\left[\begin{array}{ccc}\hfill 1\hfill & \hfill 7\hfill & \hfill 3\hfill \\ \hfill 4\hfill & \hfill -2\hfill & \hfill -2\hfill \\ \hfill -2\hfill & \hfill -1\hfill & \hfill 1\hfill \end{array}\right]\text{(Euclidean-norm)}.$

Answer

1.4 Other norms

Any definition you can think of which satisifes the five conditions mentioned at the beginning of this Section is a definition of a norm. There are many many possibilities, but the three given above are among the most commonly used.