Spectral Theorem

1. Self-Adjoint Operators

Definition 7.16: Adjoint Operator

Let $T: V \to V$ be a linear operator on inner product space $V$ . The adjoint $T^*$ is the unique operator satisfying:

\langle Tx, y \rangle = \langle x, T^* y \rangle \quad \text{for all } x, y \in V

Remark 7.22: Matrix Representation

For matrices with standard inner product:

Real: $A^* = A^T$ (transpose)
Complex: $A^* = \bar{A}^T = A^H$ (conjugate transpose)

Definition 7.17: Self-Adjoint Operator

An operator $T$ is self-adjoint (or Hermitian) if:

T = T^*

Equivalently: $\langle Tx, y \rangle = \langle x, Ty \rangle$ for all $x, y$ .

Example 7.35: Symmetric Matrices are Self-Adjoint

A real matrix $A$ is self-adjoint iff $A = A^T$ (symmetric).

A = \begin{pmatrix} 2 & 1 \\ 1 & 3 \end{pmatrix}

is symmetric, hence self-adjoint.

Example 7.36: Hermitian Matrix

A complex matrix is Hermitian (self-adjoint) iff $A = \bar{A}^T$ :

A = \begin{pmatrix} 2 & 1-i \\ 1+i & 3 \end{pmatrix}

Note: diagonal entries must be real.

Theorem 7.28a: Existence of Adjoint

For any linear operator $T$ on a finite-dimensional inner product space, the adjoint $T^*$ exists and is unique.

Proof:

For fixed $y$ , the map $x \mapsto \langle Tx, y \rangle$ is a linear functional.

By the Riesz Representation Theorem, there exists unique $z_y$ such that $\langle Tx, y \rangle = \langle x, z_y \rangle$ .

Define $T^* y = z_y$ . This $T^*$ is linear and unique.

∎

Remark 7.22a: Properties of Adjoint

$(T^*)^* = T$
$(S + T)^* = S^* + T^*$
$(\alpha T)^* = \bar{\alpha} T^*$
$(ST)^* = T^* S^*$ (order reverses)

Example 7.36a: Computing Adjoint

For $T: \mathbb{R}^2 \to \mathbb{R}^2$ with matrix $A = \begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}$ :

A^* = A^T = \begin{pmatrix} 1 & 3 \\ 2 & 4 \end{pmatrix}

Definition 7.17a: Skew-Adjoint Operator

An operator $T$ is skew-adjoint (or anti-Hermitian) if:

T^* = -T

Skew-adjoint operators have purely imaginary eigenvalues.

Example 7.36b: Skew-Symmetric Matrix

A = \begin{pmatrix} 0 & 1 \\ -1 & 0 \end{pmatrix}

This is skew-symmetric ( $A^T = -A$ ). Eigenvalues are $\pm i$ .

Definition 7.17b: Normal Operator

An operator $T$ is normal if:

TT^* = T^*T

Self-adjoint, skew-adjoint, and unitary operators are all normal.

Remark 7.22b: Why Normal Matters

Normal operators share the key property with self-adjoint ones:

They have an orthonormal basis of eigenvectors
The Complex Spectral Theorem applies
But eigenvalues may be complex (for non-self-adjoint normal operators)

2. Key Properties

Theorem 7.29: Eigenvalues are Real

All eigenvalues of a self-adjoint operator are real.

Proof:

Let $Tv = \lambda v$ with $v \neq 0$ . Then:

\lambda \|v\|^2 = \lambda \langle v, v \rangle = \langle \lambda v, v \rangle = \langle Tv, v \rangle = \langle v, Tv \rangle = \langle v, \lambda v \rangle = \bar{\lambda} \|v\|^2

Since $\|v\|^2 > 0$ , we have $\lambda = \bar{\lambda}$ , so $\lambda \in \mathbb{R}$ .

∎

Theorem 7.30: Eigenvectors are Orthogonal

Eigenvectors corresponding to distinct eigenvalues of a self-adjoint operator are orthogonal.

Proof:

Let $Tv_1 = \lambda_1 v_1$ and $Tv_2 = \lambda_2 v_2$ with $\lambda_1 \neq \lambda_2$ .

\lambda_1 \langle v_1, v_2 \rangle = \langle Tv_1, v_2 \rangle = \langle v_1, Tv_2 \rangle = \lambda_2 \langle v_1, v_2 \rangle

So $(\lambda_1 - \lambda_2) \langle v_1, v_2 \rangle = 0$ . Since $\lambda_1 \neq \lambda_2$ , we have $\langle v_1, v_2 \rangle = 0$ .

∎

Corollary 7.6: Orthonormal Eigenvectors

We can always choose orthonormal eigenvectors for a self-adjoint operator.

Theorem 7.29a: Invariant Subspace Theorem

If $T$ is self-adjoint and $W$ is a $T$ -invariant subspace, then $W^\perp$ is also $T$ -invariant.

Proof:

Let $v \in W^\perp$ . For any $w \in W$ :

\langle Tv, w \rangle = \langle v, Tw \rangle = 0

since $Tw \in W$ (W is T-invariant) and $v \perp W$ . Thus $Tv \perp W$ , so $Tv \in W^\perp$ .

∎

Example 7.36c: Checking Self-Adjoint

Verify $A = \begin{pmatrix} 1 & 2 & 3 \\ 2 & 4 & 5 \\ 3 & 5 & 6 \end{pmatrix}$ is self-adjoint:

Check $A = A^T$ : ✓ (symmetric)

Eigenvalues will be real, eigenvectors orthogonalizable.

Remark 7.22c: Geometric Interpretation

Self-adjoint operators preserve angles in a specific sense:

\langle Tx, y \rangle = \langle x, Ty \rangle

The "shadow" of $Tx$ on $y$ equals the shadow of $x$ on $Ty$ .

Theorem 7.29b: Positive Semi-Definiteness

A self-adjoint operator $T$ is positive semi-definite iff:

\langle Tv, v \rangle \geq 0 \quad \text{for all } v

Equivalently: all eigenvalues are non-negative.

Definition 7.18: Positive Definite

$T$ is positive definite if $\langle Tv, v \rangle > 0$ for all $v \neq 0$ .

Equivalently: all eigenvalues are strictly positive.

Example 7.36d: Positive Definite Test

Is $A = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}$ positive definite?

Eigenvalues: $\lambda = 3, 1$ (both positive) ✓

Or: $\langle Av, v \rangle = 2x^2 + 2xy + 2y^2 = 2(x + y/2)^2 + (3/2)y^2 > 0$ ✓

3. The Spectral Theorem

Theorem 7.31: Spectral Theorem (Finite Dimensions)

Let $T$ be a self-adjoint operator on finite-dimensional inner product space $V$ . Then:

All eigenvalues of $T$ are real
$V$ has an orthonormal basis of eigenvectors of $T$

Equivalently, $T$ is orthogonally diagonalizable.

Theorem 7.32: Matrix Form

A real matrix $A$ is symmetric iff there exists orthogonal $Q$ and diagonal $D$ with:

A = QDQ^T

where columns of $Q$ are orthonormal eigenvectors and $D$ has real eigenvalues.

Example 7.37: Orthogonal Diagonalization

Diagonalize $A = \begin{pmatrix} 3 & 1 \\ 1 & 3 \end{pmatrix}$ .

Eigenvalues: $\det(A - \lambda I) = (3-\lambda)^2 - 1 = 0$

$\lambda_1 = 4, \lambda_2 = 2$

Eigenvectors:

v_1 = \frac{1}{\sqrt{2}}(1, 1)^T, \quad v_2 = \frac{1}{\sqrt{2}}(1, -1)^T

Diagonalization:

A = \frac{1}{2}\begin{pmatrix} 1 & 1 \\ 1 & -1 \end{pmatrix} \begin{pmatrix} 4 & 0 \\ 0 & 2 \end{pmatrix} \begin{pmatrix} 1 & 1 \\ 1 & -1 \end{pmatrix}

Proof:

Proof of Spectral Theorem: By induction on dimension. Base case: $\dim V = 1$ is trivial.

Induction: Let $\lambda_1$ be an eigenvalue of $T$ (exists over $\mathbb{C}$ , real for self-adjoint).

Let $v_1$ be a unit eigenvector for $\lambda_1$ . Set $W = \text{span}\{v_1\}^\perp$ .

By the Invariant Subspace Theorem, $W$ is $T$ -invariant.

Apply induction to $T|_W$ : get orthonormal eigenbasis for $W$ .

Together with $v_1$ , this gives orthonormal eigenbasis for $V$ .

∎

Example 7.37a: Complete 3×3 Example

Orthogonally diagonalize $A = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 1 \\ 0 & 1 & 1 \end{pmatrix}$ :

Step 1: Characteristic polynomial: $\det(A - \lambda I) = (1 - \lambda)[(1-\lambda)^2 - 1] = 0$

Eigenvalues: $\lambda = 1, 2, 0$

Step 2: Find eigenvectors:

$\lambda = 1$ : $v_1 = (1, 0, 0)^T$
$\lambda = 2$ : $v_2 = (0, 1, 1)^T / \sqrt{2}$
$\lambda = 0$ : $v_3 = (0, 1, -1)^T / \sqrt{2}$

Step 3: Form Q and D.

Theorem 7.31a: Complex Spectral Theorem

A linear operator $T$ on a complex inner product space is normal iff $V$ has an orthonormal basis of eigenvectors of $T$ .

Remark 7.23a: Real vs Complex

Real case: Self-adjoint ⟹ orthonormal eigenbasis, real eigenvalues
Complex case: Normal ⟹ orthonormal eigenbasis, complex eigenvalues

Corollary 7.7: Powers and Functions

If $A = QDQ^T$ , then:

A^n = QD^n Q^T, \quad f(A) = Qf(D)Q^T

where $f(D)$ applies $f$ to diagonal entries.

Example 7.37b: Computing A¹⁰⁰

For $A = \begin{pmatrix} 3 & 1 \\ 1 & 3 \end{pmatrix}$ with $A = QDQ^T$ :

A^{100} = Q D^{100} Q^T = Q \begin{pmatrix} 4^{100} & 0 \\ 0 & 2^{100} \end{pmatrix} Q^T

4. Spectral Decomposition

Theorem 7.33: Spectral Decomposition

A self-adjoint operator $T$ can be written as:

T = \sum_{i=1}^{k} \lambda_i P_i

where $\lambda_i$ are distinct eigenvalues and $P_i$ is the orthogonal projection onto the $\lambda_i$ -eigenspace.

Remark 7.23: Projection Properties

$P_i^2 = P_i$ (idempotent)
$P_i P_j = 0$ for $i \neq j$ (orthogonal projections)
$\sum P_i = I$ (complete)

Example 7.38: Computing Spectral Decomposition

For $A = \begin{pmatrix} 3 & 1 \\ 1 & 3 \end{pmatrix}$ with $\lambda_1 = 4, \lambda_2 = 2$ :

P_1 = \frac{1}{2}\begin{pmatrix} 1 & 1 \\ 1 & 1 \end{pmatrix}, \quad P_2 = \frac{1}{2}\begin{pmatrix} 1 & -1 \\ -1 & 1 \end{pmatrix}

A = 4P_1 + 2P_2

Theorem 7.33a: Functional Calculus

For any function $f$ defined on the spectrum of $T$ :

f(T) = \sum_{i=1}^{k} f(\lambda_i) P_i

This allows computing matrix functions like $e^A$ , $\sqrt{A}$ , $\log A$ .

Example 7.38a: Matrix Exponential

Compute $e^A$ for $A = \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix}$ :

Eigenvalues: $\lambda_1 = 1, \lambda_2 = -1$

e^A = e^1 P_1 + e^{-1} P_2 = e \cdot \frac{1}{2}\begin{pmatrix} 1 & 1 \\ 1 & 1 \end{pmatrix} + \frac{1}{e} \cdot \frac{1}{2}\begin{pmatrix} 1 & -1 \\ -1 & 1 \end{pmatrix}

= \begin{pmatrix} \cosh 1 & \sinh 1 \\ \sinh 1 & \cosh 1 \end{pmatrix}

Example 7.38b: Matrix Square Root

Compute $\sqrt{A}$ for positive definite $A = \begin{pmatrix} 4 & 0 \\ 0 & 9 \end{pmatrix}$ :

\sqrt{A} = \begin{pmatrix} 2 & 0 \\ 0 & 3 \end{pmatrix}

For non-diagonal: $\sqrt{A} = Q \sqrt{D} Q^T$

Remark 7.23b: Resolvent

The resolvent $(\lambda I - T)^{-1}$ for $\lambda$ not an eigenvalue is:

(\lambda I - T)^{-1} = \sum_i \frac{1}{\lambda - \lambda_i} P_i

Definition 7.19: Spectral Radius

The spectral radius of $T$ is:

\rho(T) = \max_i |\lambda_i|

For normal operators: $\|T\| = \rho(T)$ .

5. Applications

Quantum Mechanics

Observables are self-adjoint. Real eigenvalues = measurement outcomes. Eigenstates are orthogonal.

Principal Component Analysis

Covariance matrix is symmetric. Eigenvectors give principal components for dimensionality reduction.

Vibration Analysis

Stiffness/mass matrices are symmetric. Eigenvalues give natural frequencies, eigenvectors give mode shapes.

Quadratic Forms

Classify $x^T A x$ using eigenvalue signs. Positive definite iff all eigenvalues positive.

Example 7.39: Quantum Mechanics Application

In quantum mechanics, the energy observable is represented by the Hamiltonian $H$ (self-adjoint).

Eigenvalues $E_n$ = possible energy measurements (real!)
Eigenvectors $\psi_n$ = stationary states (orthogonal!)
Any state = superposition: $\psi = \sum c_n \psi_n$

Example 7.39a: Principal Component Analysis

Given data matrix $X$ , the covariance matrix $C = X^T X / n$ is symmetric positive semi-definite.

Spectral decomposition $C = QDQ^T$ :

Columns of $Q$ = principal components (orthonormal)
Diagonal of $D$ = variance in each direction
Project data onto top $k$ principal components for dimensionality reduction

Example 7.39b: Coupled Oscillators

Two masses connected by springs: $M\ddot{x} = -Kx$ where $K$ is symmetric.

Natural frequencies: $\omega_i = \sqrt{\lambda_i}$ (eigenvalues of $M^{-1}K$ )

Mode shapes: eigenvectors describe how masses move together.

Remark 7.24: Why Self-Adjoint Matters Physically

Self-adjoint operators guarantee:

Real measurements: Eigenvalues are measurable quantities
Complete set of states: Eigenvectors span the space
Orthogonal decomposition: Different modes don't interfere
Conservation laws: Related to symmetries via Noether's theorem

6. Normal Operators

Theorem 7.34: Characterization of Normal

An operator $T$ is normal iff:

\|Tv\| = \|T^* v\| \quad \text{for all } v

Proof:

$T$ normal ⟹ $\|Tv\|^2 = \langle Tv, Tv \rangle = \langle T^* Tv, v \rangle = \langle TT^* v, v \rangle = \|T^* v\|^2$

∎

Example 7.40: Unitary Operators

Unitary operators (like rotations) satisfy $UU^* = U^* U = I$ .

They are normal with $|\lambda| = 1$ (eigenvalues on unit circle).

U = \begin{pmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{pmatrix}

Eigenvalues: $e^{\pm i\theta}$

Remark 7.25: Classes of Normal Operators

Type	Definition	Eigenvalues
Self-adjoint	$T = T^*$	Real
Skew-adjoint	$T = -T^*$	Purely imaginary
Unitary	$T^* T = I$	On unit circle
Projection	$T^2 = T = T^*$	0 or 1

Theorem 7.35: Simultaneous Diagonalization

Two self-adjoint operators $A, B$ can be simultaneously diagonalized (share orthonormal eigenbasis) iff they commute: $AB = BA$ .

Example 7.40a: Commuting Observables

In quantum mechanics, position and momentum don't commute ( $[\hat{x}, \hat{p}] = i\hbar$ ), so they can't be simultaneously measured precisely (Heisenberg uncertainty).

7. Common Mistakes

Assuming all matrices are orthogonally diagonalizable

Only symmetric/Hermitian matrices are! Non-symmetric matrices may not even be diagonalizable.

Forgetting to normalize eigenvectors

For A = QDQᵀ, Q must be orthogonal. This requires normalizing each eigenvector to unit length.

Confusing symmetric with Hermitian

Symmetric: A = Aᵀ (real). Hermitian: A = Āᵀ (complex, requires conjugate).

Forgetting to orthogonalize within eigenspaces

If eigenvalue has multiplicity >1, apply Gram-Schmidt to get orthonormal basis for that eigenspace.

Thinking normal = self-adjoint

Normal means TT* = T*T. Self-adjoint is the special case T = T*. Unitary and skew-adjoint are also normal.

Remark 7.26: Debugging Tips

When diagonalizing symmetric matrices:

Check A = Aᵀ first (symmetry)
Verify eigenvalues are real
Check eigenvectors from different eigenvalues are orthogonal
Normalize all eigenvectors
Verify Q is orthogonal: QᵀQ = I
Confirm A = QDQᵀ by matrix multiplication

8. Key Formulas Summary

Self-Adjoint

• $T = T^*$
• Real: $A = A^T$
• Complex: $A = \bar{A}^T$

• $A = QDQ^T$
• $A = \sum \lambda_i P_i$
• Eigenvalues real, eigenvectors orthogonal

Orthogonal Diagonalization Algorithm

Verify: Check A = Aᵀ (symmetric)
Eigenvalues: Solve det(A − λI) = 0
Eigenvectors: Find null(A − λI) for each λ
Orthogonalize: Apply Gram-Schmidt within each eigenspace
Normalize: Make each eigenvector unit length
Assemble: Q = [v₁ | v₂ | ... | vₙ], D = diag(λ₁, ..., λₙ)
Verify: Check A = QDQᵀ

9. More Worked Examples

Example 7.41: Complete 2×2 Diagonalization

Orthogonally diagonalize $A = \begin{pmatrix} 5 & 4 \\ 4 & 5 \end{pmatrix}$ :

Step 1: Check symmetry: $A = A^T$ ✓

Step 2: Eigenvalues:

\det(A - \lambda I) = (5-\lambda)^2 - 16 = \lambda^2 - 10\lambda + 9 = 0

\lambda_1 = 9, \quad \lambda_2 = 1

Step 3: Eigenvectors:

\lambda = 9: (A - 9I)v = 0 \Rightarrow v_1 = (1, 1)^T / \sqrt{2}

\lambda = 1: (A - I)v = 0 \Rightarrow v_2 = (1, -1)^T / \sqrt{2}

Step 4: Result:

A = \frac{1}{2}\begin{pmatrix} 1 & 1 \\ 1 & -1 \end{pmatrix} \begin{pmatrix} 9 & 0 \\ 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 1 \\ 1 & -1 \end{pmatrix}

Example 7.42: Repeated Eigenvalue

Diagonalize $A = \begin{pmatrix} 2 & 0 & 0 \\ 0 & 3 & 1 \\ 0 & 1 & 3 \end{pmatrix}$ :

Eigenvalues: $\lambda = 2$ (multiplicity 1), $\lambda = 4, 2$ (from lower-right block)

Note: $\lambda = 2$ has multiplicity 2 but the eigenspace is 2-dimensional.

Eigenvectors for $\lambda = 2$ : $(1, 0, 0)^T$ and $(0, 1, -1)^T / \sqrt{2}$

Example 7.43: Quadratic Form Classification

Classify $Q(x, y) = 3x^2 + 4xy + 3y^2$ :

Matrix form: $A = \begin{pmatrix} 3 & 2 \\ 2 & 3 \end{pmatrix}$

Eigenvalues: $\lambda = 5, 1$ (both positive)

Conclusion: $Q$ is positive definite.

In principal axes: $Q = 5u^2 + v^2$ where $(u, v)$ are rotated coordinates.

Example 7.44: Matrix Logarithm

Compute $\log A$ for positive definite $A = \begin{pmatrix} 2 & 0 \\ 0 & 3 \end{pmatrix}$ :

\log A = \begin{pmatrix} \ln 2 & 0 \\ 0 & \ln 3 \end{pmatrix}

For non-diagonal A: $\log A = Q (\log D) Q^T$

10. Connections and Extensions

Building on the Spectral Theorem

SVD (LA-8.1)

SVD generalizes spectral theorem to non-square matrices. A = UΣVᵀ uses orthonormal bases.

Quadratic Forms (LA-8.2)

Spectral theorem classifies quadratic forms. Sign of eigenvalues determines definiteness.

Polar Decomposition

Every matrix A = UP where U is unitary and P is positive semi-definite.

Infinite Dimensions

Spectral theorem extends to compact self-adjoint operators on Hilbert spaces.

Theorem 7.36: Polar Decomposition

Any invertible operator $A$ can be written as:

A = UP

where $U$ is unitary and $P = \sqrt{A^* A}$ is positive definite.

Remark 7.27: Physical Significance

The spectral theorem is one of the most important results in mathematics because:

Quantum mechanics: observables are self-adjoint, eigenvalues are measurable
Vibrations: normal modes are eigenvectors of symmetric operators
Statistics: PCA uses eigendecomposition of covariance
Graph theory: Laplacian spectral theory describes graph structure

11. Quick Reference

Key Definitions

• Adjoint: ⟨Tx, y⟩ = ⟨x, T*y⟩
• Self-adjoint: T = T*
• Normal: TT* = T*T
• Positive definite: ⟨Tv, v⟩ > 0

Key Results

• Eigenvalues of self-adjoint are real
• Eigenvectors for distinct λ are orthogonal
• Self-adjoint ⟹ orthonormal eigenbasis
• A = QDQᵀ (orthogonal diagonalization)

Applications

• Powers: Aⁿ = QDⁿQᵀ
• Functions: f(A) = Qf(D)Qᵀ
• Quadratic forms: xᵀAx = Σλᵢyᵢ²
• PCA, vibration analysis, quantum mechanics

Verification Checklist

✓ A = Aᵀ (symmetric)
✓ λᵢ ∈ ℝ (real eigenvalues)
✓ QᵀQ = I (orthogonal Q)
✓ A = QDQᵀ (decomposition)

Remark 7.28: Historical Note

The spectral theorem has roots in the work of Cauchy, Hermite, and Hilbert. The term "spectrum" was introduced by David Hilbert around 1900, inspired by the discrete frequencies (spectral lines) observed in atomic spectra.

Notation Summary

T*	Adjoint of T
Aᵀ	Transpose (real)
A*	Conjugate transpose (complex)
Pᵢ	Projection onto λᵢ-eigenspace
ρ(T)	Spectral radius

12. Additional Practice Problems

Example 7.45: Practice Problem 1

Orthogonally diagonalize $A = \begin{pmatrix} 1 & 2 \\ 2 & 1 \end{pmatrix}$ :

Solution outline:

Check symmetry: A = Aᵀ ✓
Find eigenvalues: λ = 3, −1
Find eigenvectors and normalize
Form Q and D

Example 7.46: Practice Problem 2

Show that $A = \begin{pmatrix} 1 & 2 \\ 0 & 3 \end{pmatrix}$ is NOT orthogonally diagonalizable:

Solution: A ≠ Aᵀ (not symmetric), so spectral theorem doesn't apply.

A is diagonalizable (distinct eigenvalues), but not orthogonally diagonalizable.

Example 7.47: Practice Problem 3

Determine if $B = \begin{pmatrix} 4 & 2 & 0 \\ 2 & 1 & 0 \\ 0 & 0 & 3 \end{pmatrix}$ is positive definite:

Solution: Find eigenvalues of the 2×2 block: λ = 5, 0

Since one eigenvalue is 0, B is positive semi-definite, not positive definite.

Example 7.48: Practice Problem 4

For rotation $R = \begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix}$ (90° rotation):

(a) Show R is normal but not self-adjoint

(b) Find eigenvalues (complex)

(c) Verify eigenvalues are on unit circle

Remark 7.29: Practice Strategy

For mastery of spectral theorem:

Practice 2×2 and 3×3 examples by hand
Verify every step: symmetry, eigenvalues real, eigenvectors orthogonal
Connect to applications: quadratic forms, PCA
Understand proof structure (induction on dimension)

13. Chapter Summary

Key Takeaways

Self-Adjoint Operators

• T = T* (equals its adjoint)
• Real matrices: A = Aᵀ (symmetric)
• Complex matrices: A = Ā ᵀ (Hermitian)
• All eigenvalues are real

Spectral Theorem

• Orthonormal basis of eigenvectors exists
• A = QDQᵀ with Q orthogonal
• A = Σλᵢ Pᵢ (spectral decomposition)
• Enables matrix function calculus

Applications

• Quantum mechanics: observables
• PCA: dimensionality reduction
• Vibrations: normal modes
• Quadratic forms: classification

Normal Operators

• TT* = T*T (commute with adjoint)
• Includes self-adjoint, unitary, skew-adjoint
• Have orthonormal eigenbases
• Eigenvalues may be complex

Remark 7.30: Mastery Checklist

You've mastered the spectral theorem when you can:

✓ Identify self-adjoint/symmetric/Hermitian operators
✓ Prove eigenvalues are real
✓ Show eigenvectors for distinct eigenvalues are orthogonal
✓ Orthogonally diagonalize symmetric matrices
✓ Compute spectral decomposition A = ΣλᵢPᵢ
✓ Apply to matrix functions and quadratic forms
✓ Distinguish normal from self-adjoint operators

Pro Tips for Exams

• Always check symmetry first: if A ≠ Aᵀ, spectral theorem doesn't apply
• For eigenspaces with multiplicity > 1, apply Gram-Schmidt
• Verify Q is orthogonal by checking QᵀQ = I
• For matrix functions, diagonalize first: f(A) = Qf(D)Qᵀ
• Positive definite ⟺ all eigenvalues positive
• Normal ≠ self-adjoint (unitary is normal but not self-adjoint)

14. Additional Theoretical Results

Theorem 7.37: Schur Decomposition

Every square matrix $A$ is unitarily similar to an upper triangular matrix:

A = UTU^*

where $U$ is unitary and $T$ is upper triangular with eigenvalues on diagonal.

Remark 7.31: Schur vs Spectral

Compare the two decompositions:

Schur: Always exists, T is triangular
Spectral: Only for normal, D is diagonal
Both use unitary/orthogonal similarity transforms

Theorem 7.38: Cayley-Hamilton Connection

For self-adjoint $A$ with spectral decomposition $A = \sum \lambda_i P_i$ :

p(A) = \sum p(\lambda_i) P_i

This gives an alternative proof of Cayley-Hamilton for self-adjoint operators.

Example 7.49: Verifying Cayley-Hamilton

For $A = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}$ with $\lambda = 3, 1$ :

Characteristic polynomial: $p(\lambda) = (\lambda - 3)(\lambda - 1) = \lambda^2 - 4\lambda + 3$

p(A) = A^2 - 4A + 3I = 0

Via spectral: $p(A) = p(3)P_1 + p(1)P_2 = 0 \cdot P_1 + 0 \cdot P_2 = 0$ ✓

Definition 7.20: Rayleigh Quotient

For self-adjoint $A$ and nonzero $x$ , the Rayleigh quotient is:

R(x) = \frac{\langle Ax, x \rangle}{\langle x, x \rangle} = \frac{x^T A x}{x^T x}

Theorem 7.39: Min-Max Characterization

For self-adjoint $A$ with eigenvalues $\lambda_1 \leq \lambda_2 \leq \cdots \leq \lambda_n$ :

\lambda_1 = \min_{x \neq 0} R(x), \quad \lambda_n = \max_{x \neq 0} R(x)

The extrema are achieved at the corresponding eigenvectors.

Example 7.50: Finding Eigenvalue Bounds

For $A = \begin{pmatrix} 3 & 1 \\ 1 & 3 \end{pmatrix}$ :

Rayleigh quotient at $x = (1, 0)^T$ : $R = 3$

Rayleigh quotient at $x = (1, 1)^T$ : $R = 4$ (max eigenvalue)

Rayleigh quotient at $x = (1, -1)^T$ : $R = 2$ (min eigenvalue)

Theorem 7.40: Courant-Fischer Theorem

The k-th eigenvalue can be characterized as:

\lambda_k = \min_{\dim W = k} \max_{x \in W, x \neq 0} R(x)

Remark 7.32: Applications of Min-Max

The min-max principle is used in:

Numerical analysis: Eigenvalue estimation
Physics: Ground state energy bounds
Graph theory: Cheeger inequality, spectral clustering

15. Glimpse: Infinite Dimensions

Remark 7.33: Hilbert Space Spectral Theory

The spectral theorem extends to infinite-dimensional Hilbert spaces:

Compact operators: Spectrum is countable, accumulates only at 0
Unbounded operators: Continuous and point spectrum
Spectral measure: Replaces sum with integral

Theorem 7.41: Spectral Theorem (Compact Case)

A compact self-adjoint operator $T$ on Hilbert space has:

T = \sum_{n=1}^{\infty} \lambda_n P_n

where $\lambda_n \to 0$ and $P_n$ are finite-rank projections onto eigenspaces.

Example 7.51: Integral Operators

The integral operator $(Kf)(x) = \int_0^1 k(x, y) f(y) \, dy$ with symmetric kernel is compact self-adjoint.

Its eigenvalues and eigenfunctions solve:

\int_0^1 k(x, y) \phi_n(y) \, dy = \lambda_n \phi_n(x)

Remark 7.34: Looking Ahead

The infinite-dimensional spectral theorem is fundamental to:

Quantum mechanics: Observables as unbounded operators
PDE theory: Eigenfunction expansions
Harmonic analysis: Fourier theory
Functional analysis: C*-algebras, operator theory

16. Computational Aspects

Algorithms for Symmetric Eigenproblems

Algorithm	Complexity	Best For
Power iteration	$O(n^2)$ per iter	Largest eigenvalue only
QR algorithm	$O(n^3)$	All eigenvalues
Jacobi method	$O(n^3)$	High accuracy
Divide-and-conquer	$O(n^3)$	Tridiagonal
Lanczos	$O(n^2 k)$	Sparse, few eigenvalues

Example 7.52: Power Iteration

To find largest eigenvalue of symmetric $A$ :

Start with random $v^{(0)}$
Iterate: $v^{(k+1)} = Av^{(k)} / \|Av^{(k)}\|$
Converges to eigenvector for $\lambda_{\max}$
Rayleigh quotient gives eigenvalue estimate

Remark 7.35: Symmetric Advantage

For symmetric matrices, eigenvalue algorithms are:

More stable: Real eigenvalues, orthogonal eigenvectors
Faster: Can exploit structure (tridiagonalize first)
Parallel: Divide-and-conquer works well

Example 7.53: Tridiagonalization

Before diagonalization, reduce to tridiagonal form:

A = QTQ^T \quad \text{where } T \text{ is tridiagonal}

This costs $O(n^3)$ but makes subsequent steps much faster.

17. Spectral Graph Theory

Definition 7.21: Graph Laplacian

For graph $G$ with adjacency matrix $A$ and degree matrix $D$ :

L = D - A

$L$ is symmetric positive semi-definite with eigenvalues $0 = \lambda_1 \leq \lambda_2 \leq \cdots$ .

Theorem 7.42: Spectral Properties of Laplacian

$\lambda_1 = 0$ with eigenvector $\mathbf{1}$ (all ones)
Multiplicity of 0 = number of connected components
$\lambda_2$ (algebraic connectivity) measures connectivity

Example 7.54: Path Graph

For path graph $P_3$ (3 vertices in a line):

L = \begin{pmatrix} 1 & -1 & 0 \\ -1 & 2 & -1 \\ 0 & -1 & 1 \end{pmatrix}

Eigenvalues: $0, 1, 3$

Remark 7.36: Applications of Graph Laplacian

The graph Laplacian is used in:

Spectral clustering: Partition using eigenvectors
Graph embedding: Low-dimensional representation
Random walks: Transition probabilities
Graph drawing: Spectral layouts

18. Final Synthesis

The Big Picture

The Spectral Theorem is one of the most beautiful and useful results in linear algebra. It says that symmetric/Hermitian operators are completely determined by their eigenvalues and can be decomposed into simple, orthogonal pieces.

Structure

A = QDQᵀ reveals the "principal axes" of a linear transformation.

Computation

Powers, functions, and systems become easy in the eigenbasis.

Applications

From quantum mechanics to machine learning, the spectral theorem is everywhere.

Remark 7.37: Looking Forward

The spectral theorem leads naturally to:

SVD (LA-8.1): Spectral theory for rectangular matrices
Quadratic forms (LA-8.2): Classification using eigenvalue signs
Functional analysis: Operator theory on Hilbert spaces
Representation theory: Group actions and characters

Spectral Theorem Practice

12

Questions

0

Correct

0%

Accuracy

1

A real symmetric matrix has:

Easy

Not attempted

2

The adjoint

T^*

of operator

T

satisfies:

Medium

Not attempted

3

Self-adjoint means:

Easy

Not attempted

4

For real matrices, self-adjoint equals:

Easy

Not attempted

5

Eigenvectors for distinct eigenvalues of self-adjoint

T

are:

Medium

Not attempted

6

The spectral theorem says self-adjoint

T

has:

Medium

Not attempted

7

Orthogonal diagonalization means

A = QDQ^T

where:

Medium

Not attempted

8

Which matrix is NOT orthogonally diagonalizable?

Medium

Not attempted

9

For Hermitian matrix

A

:

Medium

Not attempted

10

A normal operator satisfies:

Hard

Not attempted

11

The spectral decomposition

A = \sum \lambda_i P_i

uses:

Hard

Not attempted

12

Physical systems use symmetric matrices because:

Hard

Not attempted

Frequently Asked Questions

What does 'spectral' mean?

The 'spectrum' of an operator is its set of eigenvalues. The spectral theorem describes how the operator decomposes into eigenspaces—like white light splitting into its spectral colors.

Why are eigenvalues of symmetric matrices real?

If Av = λv with A symmetric, then λ||v||² = ⟨Av, v⟩ = ⟨v, Av⟩ = ⟨v, λv⟩ = λ̄||v||². So λ = λ̄, meaning λ is real.

What's the difference between adjoint and transpose?

For real matrices with standard inner product, they're the same: T* = Tᵀ. For complex matrices, adjoint means conjugate transpose: T* = T̄ᵀ = Tᴴ.

Why does orthogonal diagonalization matter?

It means A = QDQᵀ where Q⁻¹ = Qᵀ (easy to compute!). Powers become Aⁿ = QDⁿQᵀ. Functions like eᴬ are easy to compute.

What about non-symmetric matrices?

They may not be orthogonally diagonalizable, or even diagonalizable at all. Normal matrices (TT* = T*T) are the largest class with orthonormal eigenbases.

How is this used in quantum mechanics?

Observables are self-adjoint operators. The spectral theorem guarantees real measurement outcomes (eigenvalues) and orthogonal states (eigenvectors).

What's the connection to quadratic forms?

For xᵀAx with symmetric A, diagonalization gives Σλᵢyᵢ² in new coordinates. Sign of eigenvalues determines if form is positive/negative definite.

Can I always find an orthonormal eigenbasis?

For self-adjoint operators: yes! That's the spectral theorem. For non-normal operators: generally no.

What are projection matrices in spectral decomposition?

Pᵢ = (1/||vᵢ||²)vᵢvᵢᵀ projects onto eigenspace for λᵢ. They satisfy Pᵢ² = Pᵢ, PᵢPⱼ = 0 for i≠j, and ΣPᵢ = I.

Why is A = QDQᵀ and not QDQ⁻¹?

For orthogonal Q, Qᵀ = Q⁻¹, so they're the same! The Qᵀ notation emphasizes that Q is orthogonal and no matrix inversion is needed.