The Spectral Theorem

Author

Quasar

Published

July 23, 2024

Spectral Theorem

Every real, symmetric matrix is orthogonally diagonalizable.

Theorem 1 (Spectral Theorem) Every real symmetric matrix is diagonalizable.

Let A be a n×n real symmetric matrix. Then,

  1. The eigenvalues of A are real.
  2. There exists an orthonormal basis {q1,,qn} for Rn consisting of the eigenvectors of A. That is, there is an orthogonal matrix Q so that A=QAQ1.
Tip 1: Spectral values

The term spectrum refers to the eigenvalues of a matrix, or more, generally a linear operator. In Physics, the spectral energy lines of atoms (e.g. Balmer lines of the Hydrogen atom), are characterized as the eigenvalues of the governing quantum mechanical Schrodinger operator.

Proof.

Claim. The eigenvalues of A are real.

Ax,y=(Ax)y=xAy=x,Ay

Since, for a symmetric matrix A, A=A, it follows that:

Ax,y=x,Ay

Or using the dot-product notation, we could write:

(1)(Ax)y=x(Ay)

Suppose v0 is a non-zero vector in Rn such that there exists a complex scalar λ, satisfying:

(2)Av=λv

We can now take the complex conjugate of the eigenvalue equation. Remember that A is a real matrix, so A¯=A. Thus, we have the conjugated version of the eigenvalue equation:

(3)(Av)=Av=Av=λv=λv

Using the eigenvalue equation (), we can write:

(Av)v=(λv)v=λ(vv)

Alternatively, using and , we have:

(Av)v=v(Av)=v(λv)=λ(vv)

Consequently,

(λλ)(vv)=0

Since, v0, λ=λ. Therefore, λR.

Claim. A is orthogonally diagonalizable.

We proceed by induction.

For n=1, A and v are scalars, so Av=λv, where λ=A. Thus, we can pick any non-zero scalar v to form a basis in R. And A=P1ΛP, where P=I and Λ=A.

Inductive hypotheis. Every k×k matrix is diagonalisable for k=1,2,3,,n1.

Claim. Let ARn×n be symmetric. Then, we are interested to prove that A is diagonalizable. We break the induction part into 3 steps.

Every square matrix A has atleast one eigenvalue. Suppose λ1 is an eigenvalue of the matrix A and has a corresponding eigenvector v1. By part (I), we know that λ1R. We can normalize v1 as q1=v1/||v1||, so that it is an eigenvector with eigenvalue λ1. (Obviously, this is no problem, since if Av1=λ1v1, it implies A(v1/||v1||)=λ1(v1/||v1||). It follows that, Aq1=λ1q1. )

Now, we can extend this to a basis {q1,w2,,wn} of Rn. By the Gram-Schmidt orthogonalization algorithm, given the basis {q1,w2,,wn}, we can find a corresponding orthonormal basis {q1,q2,,qn} of Rn.

Now, we huddle these basis vectors together as column-vectors of a matrix and formulate the matrix P.

P=[q1q2qn]

By definition, P is an orthogonal matrix. So, P1=PT.

Define

B=P1AP

Step I. B is symmetric.

We have:

BT=(P1AP)T=(PTAP)T{P1=PT}=PTAT(PT)T=PTATP=PTAP{A is symmetric}=B

We are now going to try and write B in the block form to try to see the structure that this matrix must have and hope that it looks like, it is going to be diagonal.

Step II. The structure of B.

The way we do this, is to consider the matrix B post-multiplied by e1. Consider Be1. This should actually give us the first column of B. Now, we also know that B=PTAP. So, we could actually say, well,

PTAPe1=PTAq1

Now, remember that q1 is the normalized eigenvector corresponding to the eigenvalue λ1. So, Aq1=λ1q1. That means, this is equal to:

PTAq1=PTλ1q1=λ1Ptq1=λ1[q1Tq2TqnT]q1=λ1[q1Tq1q2Tq1qnTq1]=λ1[100]=[λ1000]

This is the first column of the matrix B. Since B=BT, the first row should also be

[λ1000]

So, we can write the matrix B in the block form:

B=[λ1OOC]

The first row and the first column are satisying the need to be diagonal.

Step III.

We know that C is a n1×n1 symmetric matrix. By the induction hypothesis, there exists an orthogonal matrix Q such that D=Q1CQ=QTCQ.

Now, define the matrix R as:

R:=P[101×n10n1×1Q]

Claim. Our claim is that R is orthogonal and R1AR is diagonal.

  1. We have:

R1=[101×n10n1×1Q1]P1{Reverse order law}=[101×n10n1×1QT]PT{P and Q are orthogonal}

But,

RT=[101×n10n1×1QT]PT

So,

RT=R1

Thus, R is orthogonal.

  1. Well, let’s compute R1AR.

R1AR=RTAR{R is orthogonal}=[101×n10n1×1QT]PTAP[101×n10n1×1Q]=[101×n10n1×1QT]B[101×n10n1×1Q]=[101×n10n1×1QT][λ101×n10n1×1C][101×n10n1×1Q]=[λ101×n10n1×1QTC][101×n10n1×1Q]=[λ101×n10n1×1QTCQ]

Since QTCQ is diagonal, it follows that R1AR is diagonal. This closes the proof.