Eigenvalues and eigenvectors of a matrix tell us a lot about the matrix. On the other hand, if we know our matrix is somehow special (say symmetric) it will tell us some information about how its eigenvalues and eigenvectors look like.
Let us begin with a definition. Given a matrix , the vector is an eigenvector of and has a corresponding eigenvalue , if
The eigenvectors of a matrix are exactly those vectors which when transformed by the mapping defined by are only scaled by , but their direction does not change.
Eigenvalues and eigenvectors of a projection matrix
To understand what eigenvectors are and how they behave, let us consider a projection matrix . What are ’s and ’s for a projection matrix?
The key property we’ll use is . This is because when we project a vector onto a plane to get , that is , we would expect that projecting again to do nothing, since it already lies in the plane, that is
Now thinking about eigenvectors as those vectors which don’t change direction when a projection matrix is applied, we can deduce two cases:
- Any already in the plane: .
- Any perpendicular to the plane: .
As a result, a projection matrix has two eigenvalues, and , and two sets of eigenvectors. Those that lie in the projection plane, and those that are perpendicular to it.
Eigenvalues of a permutation matrix
One more small example, consider a permutation matrix .
We can find the eigenvectors straight away, at least the first one, which is simply , since , and so its corresponding eigenvalue is .
If we think a little harder, we can guess the second eigenvector to be , since with an eigenvalue .
Computing eigenvalues and eigenvectors
We can re-arrange the terms in our definition to get a direct way to compute eigenvalues and eigenvectors of a matrix . Simply move to the left
and then notice that must be singular, because lies in its nullspace. We know that singular matrices have a zero determinant, and we can use this to compute the eigenvalues simply by writing
This is called the characteristic equation. The equation gives us a polynomial of degree , which we can use to find solutions . These need not be different, and can even be complex numbers. But once we obtain the ’s we can plug them back into the characteristic equation and one by one obtain their corresponding eigenvectors .
Eigenvalues and eigenvectors of an upper triangular matrix
For a triangular matrix, the determinant is just the diagonal
which means solving the characteristic equation of simply amounts to multiplying out the diagonal
which gives us a factored polynomial , from which we immediately see that the eigenvalues are the diagonal elements.
Suppose we have linearly independent eigenvectors of . Put them int the columns of . We now write
where is a diagonal matrix of eigenvalues. Thus we get . If we have independent eigenvectors in , we also get
The matrix is sure to have independent eigenvectors (and be diagonalizable) if all the ’s are different (no repeated ’s). Repeated eigenvalues mean may or may not have independent eigenvectors.
Proof (ref G. Strang, Introduction to LA): Suppose . Multiply by to find . Multiply by to find . Now subtracting these two equations gives us
Since and , we conclude . We can derive the same way. Since are the only coefficients for which , we see that and are linearly independent.
The same argument can be extended to eigenvectors and eigenvalues.
Sum of eigenvalues equlas the trace
Another very useful fact is that the sum of the eigenvalues equals the sum of the main diagonal (called the trace of ), that is
To prove this we’ll first show that .
To get a single element on the diagonal of we simply write
and to get the trace we just sum over all possible as
On the other hand, the -th element on the diagonal of is
and the trace is
But since we can swap the order of summation and also swap the order of multiplication, we get
Now consider we have different eigenvalues. We can diagonalize the matrix
where is a diagonal matrix of eigenvalues of . Using our trace trick we can write
and thus the sum of eigenvalues is equal the trace of . We’ve only shown this for the case of different eigenvalues. This property does hold in general, but requires some properties we haven’t proven yet (Jordan normal form), and thus we skip the rest of the proof.
Powers of a matrix
If , then we multiply by and get
or in general
Theorem: as if all .
and share the same independent eigenvectors if and only if .
This is true because and since
But this only holds if and share the same eigenvectors!
One last interesting fact we can show is what happens to the eigenvalues when we add a constant to the matrix . The proof is rather trivial, if , then
Adding a constant to a matrix causes its eigenvalues to increase by exactly that constant.
References and visualizations
Share on Twitter and Facebook
Discussion of "Eigenvalues and Eigenvectors: Basic Properties"
If you have any questions, feedback, or suggestions, please do share them in the comments! I'll try to answer each and every one. If something in the article wasn't clear don't be afraid to mention it. The goal of these articles is to be as informative as possible.
If you'd prefer to reach out to me via email, my address is loading ..