2 Symmetric matrices

Symmetric matrices have a number of useful properties which we will investigate in this Section.

Task!

Consider the following four matrices

$A_1=\left[\begin{array}{cc}\hfill 3\hfill & \hfill 1\hfill \\ \hfill 4\hfill & \hfill 5\hfill \end{array}\right]A_2=\left[\begin{array}{cc}\hfill 3\hfill & \hfill 1\hfill \\ \hfill 1\hfill & \hfill 5\hfill \end{array}\right]$

$A_3=\left[\begin{array}{ccc}\hfill 5& \hfill 8\hfill & \hfill 7\hfill \\ \hfill -1& \hfill 6\hfill & \hfill 8\hfill \\ \hfill 3& \hfill 4\hfill & \hfill 0\hfill \end{array}\right]A_4=\left[\begin{array}{ccc}\hfill 5\hfill & \hfill 8\hfill & \hfill 7\hfill \\ \hfill 8\hfill & \hfill 6\hfill & \hfill 4\hfill \\ \hfill 7\hfill & \hfill 4\hfill & \hfill 0\hfill \end{array}\right]$

What property do the matrices $A_2$ and $A_4$ possess that $A_1$ and $A_3$ do not?

Answer

Calculating the eigenvalues of an $n\times n$ matrix with real elements involves, in principle at least, solving an $n^{\text{ th}}$ order polynomial equation, a quadratic equation if $n=2$ , a cubic equation if $n=3$ , and so on. As is well known, such equations sometimes have only real solutions, but complex solutions (occurring as complex conjugate pairs) can also arise. This situation can therefore arise with the eigenvalues of matrices.

Task!

Consider the non-symmetric matrix

$A=\left[\begin{array}{cc}\hfill 2\hfill & \hfill -1\\ \hfill 5\hfill & \hfill -2\end{array}\right]$

Obtain the eigenvalues of $A$ and show that they form a complex conjugate pair.

Answer

In particular any $2\times 2$ matrix of the form

$A=\left[\begin{array}{cc}\hfill a& \hfill b\hfill \\ \hfill -b& \hfill a\hfill \end{array}\right]$

has complex conjugate eigenvalues $a\pm ib$ .

A $3\times 3$ example of a matrix with some complex eigenvalues is

$B=\left[\begin{array}{ccc}\hfill 1\hfill & \hfill -1& \hfill -1\\ \hfill 1\hfill & \hfill -1& \hfill 0\\ \hfill 1\hfill & \hfill 0& \hfill -1\end{array}\right]$

A straightforward calculation shows that the eigenvalues of $B$ are

$\lambda =-1$ (real), $\lambda =\pm i$ (complex conjugates).

With symmetric matrices on the other hand, complex eigenvalues are not possible.

The general proof of this result in Key Point 6 is beyond our scope but a simple proof for symmetric $2\times 2$ matrices is straightforward.

Let $A=\left[\begin{array}{cc}\hfill a\hfill & \hfill b\hfill \\ \hfill b\hfill & \hfill c\hfill \end{array}\right]$ be any $2\times 2$ symmetric matrix, $a,b,c$ being real numbers.

The characteristic equation for $A$ is

$\left(a-\lambda \right)\left(c-\lambda \right)-b^2=0$ or, expanding: ${\lambda }^2-\left(a+c\right)\lambda +ac-b^2=0$

from which

$\lambda =\frac{\left(a+c\right)\pm \sqrt{{\left(a+c\right)}^2-4ac+4b^2}}{2}$

The quantity under the square root sign can be treated as follows:

${\left(a+c\right)}^2-4ac+4b^2=a^2+c^2+2ac-4ac+b^2={\left(a-c\right)}^2+4b^2$

which is always positive and hence $\lambda$ cannot be complex .

Task!

Obtain the eigenvalues and the eigenvectors of the symmetric $2\times 2$ matrix

$A=\left[\begin{array}{cc}\hfill 4& \hfill -2\\ \hfill -2& \hfill 1\end{array}\right]$

Answer

We now look more closely at the eigenvectors $X$ and $Y$ in the last task. In particular we consider the product $X^{T}Y$ .

Task!

Evaluate $X^{T}Y$ from the previous task i.e. where

$X=\left[\begin{array}{c}\hfill 2\\ \hfill -1\end{array}\right]Y=\left[\begin{array}{c}\hfill 1\hfill \\ \hfill 2\hfill \end{array}\right]$

Answer

Task!

We obtained earlier in Section 22.1 Example 6 the eigenvalues of the matrix

$A=\left[\begin{array}{ccc}\hfill 2& \hfill -1& \hfill 0\\ \hfill -1& \hfill 2& \hfill -1\\ \hfill 0& \hfill -1& \hfill 2\end{array}\right]$

which, as we now emphasize, is symmetric . We found that the eigenvalues were $2,2+\sqrt{2},2-\sqrt{2}$ which are real and distinct. The corresponding eigenvectors were, respectively

$X=\left[\begin{array}{c}\hfill 1\\ \hfill 0\\ \hfill -1\end{array}\right]Y=\left[\begin{array}{c}\hfill 1\hfill \\ \hfill -\sqrt{2}\hfill \\ \hfill 1\hfill \end{array}\right]Z=\left[\begin{array}{c}\hfill 1\hfill \\ \hfill \sqrt{2}\hfill \\ \hfill 1\hfill \end{array}\right]$

(or, as usual, any multiple of these).

Show that these three eigenvectors $X,Y,Z$ are mutually orthogonal .

Answer

The following proof that eigenvectors corresponding to distinct eigenvalues of a symmetric matrix are orthogonal is straightforward and you are encouraged to follow it through.

Let $A$ be a symmetric $n\times n$ matrix and let ${\lambda }_1,{\lambda }_2$ be two distinct eigenvalues of $A$ i.e. ${\lambda }_1\ne {\lambda }_2$ with associated eigenvectors $X,Y$ respectively. We have seen that ${\lambda }_1$ and ${\lambda }_2$ must be real since $A$ is symmetric. Then

$AX={\lambda }_{1}XAY={\lambda }_{2}Y$ (1)

Transposing the first of there results gives

$X^T{A}^T={\lambda }_1{X}^T$ (2)

(Remember that for any two matrices the transpose of a product is the product of the transposes in reverse order .)

We now multiply both sides of (2) on the right by $Y$ (as well as putting $A^T=A$ , since $A$ is symmetric) to give:

$X^{T}AY={\lambda }_1{X}^{T}Y$ (3)

But, using the second eigenvalue equation of (1), equation (3) becomes

$X^T{\lambda }_{2}Y={\lambda }_1{X}^{T}Y$

or, since ${\lambda }_2$ is just a number,

${\lambda }_2{X}^{T}Y={\lambda }_1{X}^{T}Y$

Taking all terms to the same side and factorising gives

$\left({\lambda }_2-{\lambda }_1\right)X^{T}Y=0$

from which, since by assumption ${\lambda }_1\ne {\lambda }_2$ , we obtain the result

$X^{T}Y=0$

and the orthogonality has been proved.

The reader familiar with the algebra of vectors will recall that for two vectors whose Cartesian forms are

$\underset{̲}{a}=a_x\underset{̲}{i}+a_y\underset{̲}{j}+a_z\underset{̲}{k}\underset{̲}{b}=b_x\underset{̲}{i}+b_y\underset{̲}{j}+b_z\underset{̲}{k}$

the scalar (or dot) product is

$\underset{̲}{a}\cdot \underset{̲}{b}=a_x{b}_x+a_y{b}_y+a_z{b}_z.$

Furthermore, if $\underset{̲}{a}$ and $\underset{̲}{b}$ are mutually perpendicular then $\underset{̲}{a}\cdot \underset{̲}{b}=0$ . (The word ‘orthogonal’ is sometimes used instead of perpendicular in the case.) Our result, that two column vectors are orthogonal if $X^{T}Y=0$ , may thus be considered as a generalisation of the 3-dimensional result $\underset{̲}{a}\cdot \underset{̲}{b}=0$ .

2.2 Diagonalization of symmetric matrices

Recall from our earlier work that

1. We can always diagonalize a matrix with distinct eigenvalues (whether these are real or complex).
2. We can sometimes diagonalize a matrix with repeated eigenvalues. (The condition for this to be possible is that any eigenvalue of multiplicity $m$ had to have associated with it $m$ linearly independent eigenvectors.)

The situation with symmetric matrices is simpler. Basically we can diagonalize any symmetric matrix. To take the discussions further we first need the concept of an orthogonal matrix.

A square matrix $A$ is said to be orthogonal if its inverse (if it exists) is equal to its transpose:

$A^{-1}=A^T$ or, equivalently, $A{A}^T=A^{T}A=I.$

Example

An important example of an orthogonal matrix is

$A=\left[\begin{array}{cc}\hfill cos\varphi & \hfill sin\varphi \hfill \\ \hfill -sin\varphi & \hfill cos\varphi \hfill \end{array}\right]$

which arises when we use matrices to describe rotations in a plane.

It is clear that $A^{T}A=I$ also, so $A$ is indeed orthogonal.

It can be shown, but we omit the details, that any $2\times 2$ matrix which is orthogonal can be written in one of the two forms.

$\left[\begin{array}{cc}\hfill cos\varphi & \hfill sin\varphi \hfill \\ \hfill -sin\varphi & \hfill cos\varphi \hfill \end{array}\right]$ or $\left[\begin{array}{cc}\hfill cos\varphi \hfill & \hfill -sin\varphi \hfill \\ \hfill sin\varphi \hfill & \hfill cos\varphi \hfill \end{array}\right]$

If we look closely at either of these matrices we can see that

1. The two columns are mutually orthogonal e.g. for the first matrix we have

   $\left(cos\varphi -sin\varphi \right)\left[\begin{array}{c}\hfill sin\varphi \hfill \\ \hfill cos\varphi \hfill \end{array}\right]=cos\varphi sin\varphi -sin\varphi cos\varphi =0$

2. Each column has magnitude 1 (because $\sqrt{{cos}^2\varphi +{sin}^2\varphi }=1$ )

Although we shall not prove it, these results are necessary and sufficient for any order square matrix to be orthogonal.

Task!

For each of the following matrices verify that the two properties above are satisfied. Then check in both cases that $A{A}^T=A^{T}A=I$ i.e. that $A^T=A^{-1}$ .

1. $A=\left[\begin{array}{cc}\hfill \frac{\sqrt{3}}{2}\hfill & \hfill -\frac{1}{2}\hfill \\ \hfill \hfill \\ \hfill \frac{1}{2}\hfill & \hfill \frac{\sqrt{3}}{2}\hfill \end{array}\right]$
2. $A=\left[\begin{array}{ccc}\hfill \frac{1}{\sqrt{2}}\hfill & \hfill 0\hfill & \hfill -\frac{1}{\sqrt{2}}\hfill \\ \hfill 0\hfill & \hfill 1\hfill & \hfill 0\hfill \\ \hfill -\frac{1}{\sqrt{2}}\hfill & \hfill 0\hfill & \hfill \frac{1}{\sqrt{2}}\hfill \end{array}\right]$

Answer

The following is the key result of this Section.

It should be clear to the reader that Key Point 10 is a very powerful result for any applications that involve diagonalization of a symmetric matrix. Further, if we do need to find the inverse of $P$ , then this is a trivial process since $P^{-1}=P^T\text{(Key Point 9)}.$

Task!

The symmetric matrix

$A=\left[\begin{array}{ccc}\hfill 1\hfill & \hfill 0\hfill & \hfill \sqrt{2}\hfill \\ \hfill 0\hfill & \hfill 2\hfill & \hfill 0\hfill \\ \hfill \sqrt{2}\hfill & \hfill 0\hfill & \hfill 0\hfill \end{array}\right]$

has eigenvalues $2,2,-1$ (i.e. eigenvalue 2 is repeated with multiplicity 2.)

Associated with the non-repeated eigenvalue $-1$ is an eigenvector:

$X=\left[\begin{array}{c}\hfill 1\hfill \\ \hfill 0\hfill \\ \hfill -\sqrt{2}\hfill \end{array}\right]$ (or any multiple)

1. Normalize the eigenvector $X$ :

   Answer

   Normalizing $X$ which has magnitude $\sqrt{1^2+{\left(-\sqrt{2}\right)}^2}=\sqrt{3}$ gives

   $1∕\sqrt{3}\left[\begin{array}{c}\hfill 1\hfill \\ \hfill 0\hfill \\ \hfill -\sqrt{2}\hfill \end{array}\right]=\left[\begin{array}{c}\hfill 1∕\sqrt{3}\hfill \\ \hfill \hfill \\ \hfill 0\hfill \\ \hfill \hfill \\ \hfill -\sqrt{2∕3}\hfill \end{array}\right]$

2. Investigate the eigenvectors associated with the repeated eigenvalue 2:

   Answer

   The eigenvectors associated with $\lambda =2$ satisfy $AY=2Y$

   which gives $\left[\begin{array}{ccc}\hfill -1\hfill & \hfill 0\hfill & \hfill \sqrt{2}\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill \sqrt{2}\hfill & \hfill 0\hfill & \hfill -2\hfill \end{array}\right]\left[\begin{array}{c}\hfill x\hfill \\ \hfill y\hfill \\ \hfill z\hfill \end{array}\right]=\left[\begin{array}{c}\hfill 0\hfill \\ \hfill 0\hfill \\ \hfill 0\hfill \end{array}\right]$

   The first and third equations give

   $-x+\sqrt{2}z=0$

   $\sqrt{2}x-2z=0$ i.e. $x=\sqrt{2}z$

   The equations give us no information about $y$ so its value is arbitrary.

   Thus $Y$ has the form $Y=\left[\begin{array}{c}\hfill \sqrt{2}\beta \hfill \\ \hfill \alpha \hfill \\ \hfill \beta \hfill \end{array}\right]$  where both $\alpha$ and $\beta$ are arbitrary.

A certain amount of care is now required in the choice of $\alpha$ and $\beta$ if we are to find an orthogonal modal matrix to diagonalize $A$ .

For any choice

$X^{T}Y=\left(10-\sqrt{2}\right)\left[\begin{array}{c}\hfill \sqrt{2}\beta \hfill \\ \hfill \alpha \hfill \\ \hfill \beta \hfill \end{array}\right]=\sqrt{2}\beta -\sqrt{2}\beta =0.$

So $X$ and $Y$ are orthogonal. (The normalization of $X$ does not affect this.)

However, we also need two orthogonal eigenvectors of the form $\left[\begin{array}{c}\hfill \sqrt{2}\beta \hfill \\ \hfill \alpha \hfill \\ \hfill \beta \hfill \end{array}\right]$ . Two such are

$Y^{\left(1\right)}=\left[\begin{array}{c}\hfill 0\hfill \\ \hfill 1\hfill \\ \hfill 0\hfill \end{array}\right]\left(\text{ choosing}\beta =0,\alpha =1\right)\text{ and}{Y}^{\left(2\right)}=\left[\begin{array}{c}\hfill \sqrt{2}\hfill \\ \hfill 0\hfill \\ \hfill 1\hfill \end{array}\right]\left(\text{ choosing}\alpha =0,\beta =1\right)$

After normalization, these become $Y^{\left(1\right)}=\left[\begin{array}{c}\hfill 0\hfill \\ \hfill 1\hfill \\ \hfill 0\hfill \end{array}\right]Y^{\left(2\right)}=\left[\begin{array}{c}\hfill \sqrt{2∕3}\hfill \\ \hfill 0\hfill \\ \hfill 1∕\sqrt{3}\hfill \end{array}\right]$

Hence the matrix $P=\left[X⋮Y^{\left(1\right)}⋮Y^{\left(2\right)}\right]=\left[\begin{array}{ccc}\hfill 1∕\sqrt{3}\hfill & \hfill 0\hfill & \hfill \sqrt{2∕3}\hfill \\ \hfill 0\hfill & \hfill 1\hfill & \hfill 0\hfill \\ \hfill -\sqrt{2∕3}\hfill & \hfill 0\hfill & \hfill 1∕\sqrt{3}\hfill \end{array}\right]$

is orthogonal and diagonalizes $A$ :

$P^{T}AP=\left[\begin{array}{ccc}\hfill -1& \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0& \hfill 2\hfill & \hfill 0\hfill \\ \hfill 0& \hfill 0\hfill & \hfill 2\hfill \end{array}\right]$

2.3 Hermitian matrices

In some applications, of which quantum mechanics is one, matrices with complex elements arise.

If $A$ is such a matrix then the matrix ${\bar{A}}^T$ is the conjugate transpose of $A$ , i.e. the complex conjugate of each element of $A$ is taken as well as $A$ being transposed. Thus if

$A=\left[\begin{array}{cc}\hfill 2+i\hfill & \hfill 2\hfill \\ \hfill 3i\hfill & \hfill 5-2i\hfill \end{array}\right]\text{ then}{\bar{A}}^T=\left[\begin{array}{cc}\hfill 2-i\hfill & \hfill -3i\hfill \\ \hfill 2\hfill & \hfill 5+2i\hfill \end{array}\right]$

An Hermitian matrix is one satisfying

${\bar{A}}^T=A$

The matrix $A$ above is clearly non-Hermitian. Indeed the most obvious features of an Hermitian matrix is that its diagonal elements must be real. (Can you see why?) Thus

$A=\left[\begin{array}{cc}\hfill 6\hfill & \hfill 4+i\hfill \\ \hfill 4-i\hfill & \hfill -2\hfill \end{array}\right]$

is Hermitian.

A $3\times 3$ example of an Hermitian matrix is

$A=\left[\begin{array}{ccc}\hfill 1\hfill & \hfill i\hfill & \hfill 5-2i\hfill \\ \hfill -i\hfill & \hfill 3\hfill & \hfill 0\hfill \\ \hfill 5+2i\hfill & \hfill 0\hfill & \hfill 2\hfill \end{array}\right]$

An Hermitian matrix is in fact a generalization of a symmetric matrix. The key property of an Hermitian matrix is the same as that of a real symmetric matrix – i.e. the eigenvalues are always real .