2 General eigenvalue problems

Consider a given square matrix $A$ . If $X$ is a column vector and $\lambda$ is a scalar (a number) then the relation.

$AX=\lambda X$ (4)

is called an eigenvalue problem . Our purpose is to carry out an analysis of this equation in a manner similar to the example above. However, we will attempt a more general approach which will apply to all problems of this kind.

Firstly, we can spot an obvious solution (for $X$ ) to these equations. The solution $X=0$ is a possibility (for then both sides are zero). We will not be interested in these trivial solutions of the eigenvalue problem. Our main interest will be in the occurrence of non-trivial solutions for $X$ . These may exist for special values of $\lambda$ , called the eigenvalues of the matrix $A$ . We proceed as in the previous example:

take all unknowns to one side:

$\left(A-\lambda I\right)X=0$ (5)

where $I$ is a unit matrix with the same dimensions as $A$ . (Note that $AX-\lambda X=0$ does not simplify to $\left(A-\lambda \right)X=0$ as you cannot subtract a scalar $\lambda$ from a matrix $A$ ). This equation (5) is a homogeneous system of equations. In the notation of the earlier discussion $C\equiv A-\lambda I$ and $K\equiv 0$ . For such a system we know that non-trivial solutions will only exist if the determinant of the coefficient matrix is zero:

$det\; <!--nolimits\; -->\left(A-\lambda I\right)=0$ (6)

Equation (6) is called the characteristic equation of the eigenvalue problem. We see that the characteristic equation only involves one unknown $\lambda$ . The characteristic equation is generally a polynomial in $\lambda$ , with degree being the same as the order of $A$ (so if $A$ is $2\times 2$ the characteristic equation is a quadratic, if $A$ is a $3\times 3$ it is a cubic equation, and so on). For each value of $\lambda$ that is obtained the corresponding value of $X$ is obtained by solving the original equations (4). These $X$ ’s are called eigenvectors .

N.B. We shall see that eigenvectors are only unique up to a multiplicative factor: i.e. if $X$ satisfies $AX=\lambda X$ then so does $kX$ when $k$ is any constant.

Example 5

Find the eigenvalues and eigenvectors of the matrix $A=\left[\begin{array}{cc}\hfill 1& \hfill 0\\ \hfill 1& \hfill 2\end{array}\right]$

Solution

The eigenvalues and eigenvectors are found by solving the eigenvalue probelm

$AX=\lambda XX=\left[\begin{array}{c}\hfill x\hfill \\ \hfill y\hfill \end{array}\right]$ i.e. $\left(A-\lambda I\right)X=0.$

Non-trivial solutions will exist if $det\; <!--nolimits\; -->\left(A-\lambda I\right)=0$

that is, $det\; <!--nolimits\; -->\left\{\left[\begin{array}{cc}\hfill 1& \hfill 0\\ \hfill 1& \hfill 2\end{array}\right]-\lambda \left[\begin{array}{cc}\hfill 1\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 1\hfill \\ \hfill \hfill \end{array}\right]\right\}=0,$ $\therefore \left|\begin{array}{cc}\hfill 1-\lambda \hfill & \hfill 0\hfill \\ \hfill 1\hfill & \hfill 2-\lambda \hfill \end{array}\right|=0,$

expanding this determinant: $\left(1-\lambda \right)\left(2-\lambda \right)=0.$  Hence the solutions for $\lambda$ are: $\lambda =1$ and $\lambda =2$ .

So we have found two values of $\lambda$ for this $2\times 2$ matrix $A$ . Since these are unequal they are said to be distinct eigenvalues.

To each value of $\lambda$ there corresponds an eigenvector. We now proceed to find the eigenvectors.

2.1 Case 1

$\lambda =1$ (smaller eigenvalue). Then our original eigenvalue problem becomes:  $AX=X$ .  In full this is

Simplifying

All we can deduce here is that $x=-y$ $\therefore X=\left[\begin{array}{c}\hfill x\hfill \\ \hfill -x\hfill \end{array}\right]$ for any $x\ne 0$

(We specify $x\ne 0$ as, otherwise, we would have the trivial solution.)

So the eigenvectors corresponding to eigenvalue $\lambda =1$ are all proportional to $\left[\begin{array}{c}\hfill 1\hfill \\ \hfill -1\hfill \end{array}\right]$ , e.g. $\left[\begin{array}{c}\hfill 2\hfill \\ \hfill -2\hfill \end{array}\right]$ , $\left[\begin{array}{c}\hfill -1\hfill \\ \hfill 1\hfill \end{array}\right]$ etc.

Sometimes we write the eigenvector in normalised form that is, with modulus or magnitude 1. Here, the normalised form of $X$ is

$\frac{1}{\sqrt{2}}\left[\begin{array}{c}\hfill 1\hfill \\ \hfill -1\hfill \end{array}\right]$

which  is unique .

Case 2 Now we consider the larger eigenvalue $\lambda =2$ . Our original eigenvalue problem $AX=\lambda X$ becomes $AX=2X$ which gives the following equations:

$\left[\begin{array}{cc}\hfill 1& \hfill 0\\ \hfill 1& \hfill 2\end{array}\right]\left[\begin{array}{c}\hfill x\hfill \\ \hfill y\hfill \end{array}\right]=2\left[\begin{array}{c}\hfill x\hfill \\ \hfill y\hfill \end{array}\right]$

i.e.

These equations imply that $x=0$ whilst the variable $y$ may take any value whatsoever (except zero as this gives the trivial solution).

Thus the eigenvector corresponding to eigenvalue $\lambda =2$ has the form $\left[\begin{array}{c}\hfill 0\\ \hfill y\end{array}\right]$ , e.g. $\left[\begin{array}{c}\hfill 0\\ \hfill 1\end{array}\right]$ , $\left[\begin{array}{c}\hfill 0\\ \hfill 2\end{array}\right]$  etc. The normalised eigenvector here is  $\left[\begin{array}{c}\hfill 0\\ \hfill 1\end{array}\right]$ .

In conclusion: the matrix $A=\left[\begin{array}{cc}\hfill 1& \hfill 0\\ \hfill 1& \hfill 2\end{array}\right]$ has two eigenvalues and two associated normalised eigenvectors:

${\lambda }_1=1,{\lambda }_2=2$

$X_1=\frac{1}{\sqrt{2}}\left[\begin{array}{c}\hfill 1\hfill \\ \hfill -1\hfill \end{array}\right]X_2=\left[\begin{array}{c}\hfill 0\\ \hfill 1\end{array}\right]$

Example 6

Find the eigenvalues and eigenvectors of the $3\times 3$ matrix

$A=\left[\begin{array}{ccc}\hfill 2& \hfill -1& \hfill 0\\ \hfill -1& \hfill 2& \hfill -1\\ \hfill 0& \hfill -1& \hfill 2\end{array}\right]$

Solution

The eigenvalues and eigenvectors are found by solving the eigenvalue problem

$AX=\lambda XX=\left[\begin{array}{c}\hfill x\hfill \\ \hfill y\hfill \\ \hfill z\hfill \end{array}\right]$

Proceeding as in Example 5:

$\left(A-\lambda I\right)X=0$   and non-trivial solutions for $X$ will exist if $det\; <!--nolimits\; -->\left(A-\lambda I\right)=0$

that is,

$det\; <!--nolimits\; -->\left\{\left[\begin{array}{ccc}\hfill 2& \hfill -1& \hfill 0\\ \hfill -1& \hfill 2& \hfill -1\\ \hfill 0& \hfill -1& \hfill 2\end{array}\right]-\lambda \left[\begin{array}{ccc}\hfill 1\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 1\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill 1\hfill \end{array}\right]\right\}=0$

$\text{ i.e.}\left|\begin{array}{ccc}\hfill 2-\lambda \hfill & \hfill -1\hfill & \hfill 0\hfill \\ \hfill -1\hfill & \hfill 2-\lambda \hfill & \hfill -1\hfill \\ \hfill 0\hfill & \hfill -1\hfill & \hfill 2-\lambda \hfill \end{array}\right|=0.$

Expanding this determinant we find:

$\left(2-\lambda \right)\left|\begin{array}{cc}\hfill 2-\lambda \hfill & \hfill -1\hfill \\ \hfill -1\hfill & \hfill 2-\lambda \hfill \end{array}\right|+\left|\begin{array}{cc}\hfill -1\hfill & \hfill -1\hfill \\ \hfill 0\hfill & \hfill 2-\lambda \hfill \end{array}\right|=0$

that is,

$\left(2-\lambda \right)\left\{{\left(2-\lambda \right)}^2-1\right\}-\left(2-\lambda \right)=0$

Taking out the common factor $\left(2-\lambda \right)$ :

$\left(2-\lambda \right)\left\{4-4\lambda +{\lambda }^2-1-1\right\}$

which gives: $\left(2-\lambda \right)\left[{\lambda }^2-4\lambda +2\right]=0.$

This is easily solved to give: $\lambda =2$ or $\lambda =\frac{4\pm \sqrt{16-8}}{2}=2\pm \sqrt{2}$ .

So (typically) we have found three possible values of $\lambda$ for this $3\times 3$ matrix $A$ .

To each value of $\lambda$ there corresponds an eigenvector.

Case 1: $\lambda =2-\sqrt{2}$  (lowest eigenvalue)

Then $AX=\left(2-\sqrt{2}\right)X$ implies

Simplifying

We conclude the following:

$\text{ (c)}\Rightarrow y=\sqrt{2}z\text{ (a)}\Rightarrow y=\sqrt{2}x$

$\therefore \text{ these two relations give}x=z\text{ then}\text{ (b)}\Rightarrow -x+2x-x=0$

The last equation gives us no information; it simply states that $0=0$ .

$\therefore X=\left[\begin{array}{c}\hfill x\hfill \\ \hfill \sqrt{2}x\hfill \\ \hfill x\hfill \end{array}\right]$ for any $x\ne 0$ (otherwise we would have the trivial solution). So the eigenvectors corresponding to eigenvalue $\lambda =2-\sqrt{2}$ are all proportional to $\left[\begin{array}{c}\hfill 1\hfill \\ \hfill \sqrt{2}\hfill \\ \hfill 1\hfill \end{array}\right]$ .

In normalised form we have an eigenvector $\frac{1}{2}\left[\begin{array}{c}\hfill 1\hfill \\ \hfill \sqrt{2}\hfill \\ \hfill 1\hfill \end{array}\right].$

Case 2:   $\lambda =2$

Here $AX=2X$ implies $\left[\begin{array}{ccc}\hfill 2& \hfill -1& \hfill 0\\ \hfill -1& \hfill 2& \hfill -1\\ \hfill 0& \hfill -1& \hfill 2\end{array}\right]\left[\begin{array}{c}\hfill x\hfill \\ \hfill y\hfill \\ \hfill z\hfill \end{array}\right]=2\left[\begin{array}{c}\hfill x\hfill \\ \hfill y\hfill \\ \hfill z\hfill \end{array}\right]$

i.e.

After simplifying the equations become:

(a), (c) imply $y=0$ : (b) implies $x=-z$

$\therefore$ eigenvector has the form $\left[\begin{array}{c}\hfill x\\ \hfill 0\\ \hfill -x\end{array}\right]$ for any $x\ne 0$ .

That is, eigenvectors corresponding to $\lambda =2$ are all proportional to $\left[\begin{array}{c}\hfill 1\\ \hfill 0\\ \hfill -1\end{array}\right]$ .

In normalised form we have an eigenvector $\frac{1}{\sqrt{2}}\left[\begin{array}{c}\hfill 1\\ \hfill 0\\ \hfill -1\end{array}\right].$

Case 3:   $\lambda =2+\sqrt{2}$  (largest eigenvalue)

Proceeding along similar lines to cases 1,2 above we find that the eigenvectors corresponding to $\lambda =2+\sqrt{2}$ are each proportional to $\left[\begin{array}{c}\hfill 1\hfill \\ \hfill -\sqrt{2}\hfill \\ \hfill 1\hfill \end{array}\right]$ with normalised eigenvector $\frac{1}{2}\left[\begin{array}{c}\hfill 1\hfill \\ \hfill -\sqrt{2}\hfill \\ \hfill 1\hfill \end{array}\right].$

In conclusion the matrix $A$ has three distinct eigenvalues:

${\lambda }_1=2-\sqrt{2},{\lambda }_2=2{\lambda }_3=2+\sqrt{2}$

and three corresponding normalised eigenvectors:

$X_1=\frac{1}{2}\left[\begin{array}{c}\hfill 1\hfill \\ \hfill \sqrt{2}\hfill \\ \hfill 1\hfill \end{array}\right],X_2=\frac{1}{\sqrt{2}}\left[\begin{array}{c}\hfill 1\\ \hfill 0\\ \hfill -1\end{array}\right],X_3=\frac{1}{2}\left[\begin{array}{c}\hfill 1\hfill \\ \hfill -\sqrt{2}\hfill \\ \hfill 1\hfill \end{array}\right]$

Exercise

Find the eigenvalues and eigenvectors of each of the following matrices $A$ :

1. $\left[\begin{array}{cc}\hfill 4\hfill & \hfill -2\hfill \\ \hfill 1\hfill & \hfill 1\hfill \end{array}\right]$
2. $\left[\begin{array}{cc}\hfill 1\hfill & \hfill 2\hfill \\ \hfill -8\hfill & \hfill 11\hfill \end{array}\right]$
3. $\left[\begin{array}{ccc}\hfill 2\hfill & \hfill 0\hfill & \hfill -2\hfill \\ \hfill 0\hfill & \hfill 4\hfill & \hfill 0\hfill \\ \hfill -2\hfill & \hfill 0\hfill & \hfill 5\hfill \\ \hfill \hfill \end{array}\right]$
4. $\left[\begin{array}{ccc}\hfill 10\hfill & \hfill -2\hfill & \hfill 4\hfill \\ \hfill -20\hfill & \hfill 4\hfill & \hfill -10\hfill \\ \hfill -30\hfill & \hfill 6\hfill & \hfill -13\hfill \\ \hfill \hfill \end{array}\right]$

Answer

(eigenvectors are written in normalised form)

1. 3 and 2; $\left[\begin{array}{c}\hfill 2∕\sqrt{5}\hfill \\ \hfill 1∕\sqrt{5}\hfill \\ \hfill \hfill \end{array}\right]$ and $\left[\begin{array}{c}\hfill 1∕\sqrt{2}\hfill \\ \hfill 1∕\sqrt{2}\hfill \\ \hfill \hfill \end{array}\right]$
2. 3 and 9; $\frac{1}{\sqrt{2}}\left[\begin{array}{c}\hfill 1\hfill \\ \hfill 1\hfill \\ \hfill \hfill \end{array}\right]$ and $\frac{1}{\sqrt{17}}\left[\begin{array}{c}\hfill 1\hfill \\ \hfill 4\hfill \\ \hfill \hfill \end{array}\right]$
3. 1, 4 and 6; $\frac{1}{\sqrt{5}}\left[\begin{array}{c}\hfill 2\hfill \\ \hfill 0\hfill \\ \hfill 1\hfill \\ \hfill \hfill \end{array}\right];\left[\begin{array}{c}\hfill 0\hfill \\ \hfill 1\hfill \\ \hfill 0\hfill \\ \hfill \hfill \end{array}\right];\frac{1}{\sqrt{5}}\left[\begin{array}{c}\hfill 1\hfill \\ \hfill 0\hfill \\ \hfill -2\hfill \\ \hfill \hfill \end{array}\right]$
4. 0, $-1$ and 2; $\frac{1}{\sqrt{26}}\left[\begin{array}{c}\hfill 1\hfill \\ \hfill 5\hfill \\ \hfill 0\hfill \\ \hfill \hfill \end{array}\right];\frac{1}{\sqrt{5}}\left[\begin{array}{c}\hfill 0\hfill \\ \hfill 2\hfill \\ \hfill 1\hfill \\ \hfill \hfill \end{array}\right];\frac{1}{\sqrt{5}}\left[\begin{array}{c}\hfill 1\hfill \\ \hfill 0\hfill \\ \hfill -2\hfill \\ \hfill \hfill \end{array}\right]$