Special Matrices | Matrices | Linear Algebra

1. Diagonal Matrices

Definition 4.6.1: Diagonal Matrix

A square matrix $D = (d_{ij}) \in M_n(F)$ is diagonal if $d_{ij} = 0$ whenever $i \neq j$ .

D = \begin{pmatrix} d_1 & 0 & \cdots & 0 \\ 0 & d_2 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & d_n \end{pmatrix} = \text{diag}(d_1, d_2, \ldots, d_n)

Theorem 4.6.1: Properties of Diagonal Matrices

Let $D = \text{diag}(d_1, \ldots, d_n)$ and $E = \text{diag}(e_1, \ldots, e_n)$ . Then:

$D + E = \text{diag}(d_1 + e_1, \ldots, d_n + e_n)$
$DE = \text{diag}(d_1 e_1, \ldots, d_n e_n)$
$D^k = \text{diag}(d_1^k, \ldots, d_n^k)$ for any $k \geq 0$
$D$ is invertible iff all $d_i \neq 0$ , with $D^{-1} = \text{diag}(d_1^{-1}, \ldots, d_n^{-1})$
$\det(D) = d_1 d_2 \cdots d_n$
The eigenvalues of $D$ are $d_1, d_2, \ldots, d_n$

Proof:

All properties follow from direct computation. For (2): $(DE)_{ij} = \sum_k D_{ik} E_{kj}$ . Since both are diagonal, only $k = i = j$ contributes, giving $d_i e_i$ .

For (6): $De_i = d_i e_i$ where $e_i$ is the $i$ -th standard basis vector.

∎

Example 4.6.1: Diagonal Matrix Operations

Let $D = \text{diag}(2, -1, 3)$ and $E = \text{diag}(4, 2, -1)$ .

DE = \text{diag}(8, -2, -3), \quad D^2 = \text{diag}(4, 1, 9)

D^{-1} = \text{diag}(1/2, -1, 1/3), \quad \det(D) = 2 \cdot (-1) \cdot 3 = -6

Remark 4.6.1: Scalar Matrices

A scalar matrix $\lambda I$ is diagonal with all entries equal. Scalar matrices commute with all matrices:

(\lambda I) A = \lambda A = A(\lambda I)

In fact, scalar matrices are the only matrices that commute with all $n \\times n$ matrices.

Example 4.6.1b: Matrix Functions of Diagonal Matrices

For $D = \text{diag}(1, 2, 3)$ , compute $e^D$ :

e^D = \text{diag}(e^1, e^2, e^3) = \text{diag}(e, e^2, e^3)

In general, $f(D) = \text{diag}(f(d_1), \ldots, f(d_n))$ for any function $f$ .

Example 4.6.1c: Diagonal Matrix Power

For $D = \text{diag}(2, 3, -1)$ , compute $D^{100}$ :

D^{100} = \text{diag}(2^{100}, 3^{100}, (-1)^{100}) = \text{diag}(2^{100}, 3^{100}, 1)

This is trivial for diagonal matrices but would be hard to compute directly for non-diagonal matrices.

Theorem 4.6.1b: Diagonal Matrices Commute

Any two diagonal matrices commute: if $D_1, D_2$ are diagonal, then $D_1 D_2 = D_2 D_1$ .

Proof:

$(D_1 D_2)_{ij} = (d_1)_i (d_2)_j \delta_{ij} = (d_2)_j (d_1)_i \delta_{ij} = (D_2 D_1)_{ij}$ .

∎

Remark 4.6.1b: Block Diagonal Matrices

A block diagonal matrix has square blocks along the diagonal:

\text{diag}(A_1, A_2, \ldots, A_k) = \begin{pmatrix} A_1 & 0 & \cdots & 0 \\ 0 & A_2 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & A_k \end{pmatrix}

Properties: $\det = \prod_i \det(A_i)$ , inverse exists iff each block is invertible.

Example 4.6.1d: Block Diagonal Operations

For $M = \text{diag}(A, B)$ where $A$ is $2 \times 2$ and $B$ is $3 \times 3$ :

M^{-1} = \text{diag}(A^{-1}, B^{-1}), \quad M^n = \text{diag}(A^n, B^n)

2. Triangular Matrices

Definition 4.6.2: Upper and Lower Triangular Matrices

A matrix $A = (a_{ij})$ is:

Upper triangular if $a_{ij} = 0$ for $i > j$ (zeros below diagonal)
Lower triangular if $a_{ij} = 0$ for $i < j$ (zeros above diagonal)
Strictly triangular if also $a_{ii} = 0$ for all $i$

\text{Upper: } \begin{pmatrix} * & * & * \\ 0 & * & * \\ 0 & 0 & * \end{pmatrix}, \quad \text{Lower: } \begin{pmatrix} * & 0 & 0 \\ * & * & 0 \\ * & * & * \end{pmatrix}

Theorem 4.6.2: Properties of Triangular Matrices

Let $U$ , $U_1$ , $U_2$ be upper triangular. Then:

$U_1 + U_2$ is upper triangular
$U_1 U_2$ is upper triangular
If $U$ is invertible, $U^{-1}$ is upper triangular
$\det(U) = u_{11} u_{22} \cdots u_{nn}$
The eigenvalues of $U$ are its diagonal entries

Proof:

For (2): Let $W = U_1 U_2$ . For $i > j$ :

w_{ij} = \sum_{k=1}^n (u_1)_{ik} (u_2)_{kj}

When $k < i$ , $(u_1)_{ik} = 0$ . When $k \geq i > j$ , $(u_2)_{kj} = 0$ . So $w_{ij} = 0$ .

∎

Example 4.6.2: Triangular Matrix Product

\begin{pmatrix} 2 & 1 & 3 \\ 0 & -1 & 2 \\ 0 & 0 & 4 \end{pmatrix} \begin{pmatrix} 1 & 0 & 1 \\ 0 & 3 & -1 \\ 0 & 0 & 2 \end{pmatrix} = \begin{pmatrix} 2 & 3 & 7 \\ 0 & -3 & 5 \\ 0 & 0 & 8 \end{pmatrix}

The product is upper triangular with diagonal entries $(2, -3, 8)$ .

Corollary 4.6.1: Invertibility of Triangular Matrices

A triangular matrix is invertible if and only if all diagonal entries are nonzero.

Remark 4.6.2: LU Decomposition

Many matrices can be factored as $A = LU$ where $L$ is lower triangular and $U$ is upper triangular. This is fundamental for solving linear systems efficiently.

Example 4.6.2b: Product of Diagonal Entries

For $U = \begin{pmatrix} 3 & 1 & 2 \\ 0 & 2 & 5 \\ 0 & 0 & -1 \end{pmatrix}$ :

\det(U) = 3 \cdot 2 \cdot (-1) = -6

Eigenvalues are 3, 2, -1 (the diagonal entries).

Example 4.6.2c: LU Factorization

\begin{pmatrix} 2 & 1 \\ 4 & 5 \end{pmatrix} = \begin{pmatrix} 1 & 0 \\ 2 & 1 \end{pmatrix} \begin{pmatrix} 2 & 1 \\ 0 & 3 \end{pmatrix}

Here $L$ has 1s on diagonal (unit lower triangular), $U$ is upper triangular.

Theorem 4.6.2b: Inverse of Triangular Matrix

If $U$ is upper triangular and invertible, then $U^{-1}$ can be computed in $O(n^2)$ operations using back-substitution, rather than $O(n^3)$ for general matrices.

Remark 4.6.2b: Block Triangular Matrices

A block upper triangular matrix has the form:

\begin{pmatrix} A_{11} & A_{12} & A_{13} \\ 0 & A_{22} & A_{23} \\ 0 & 0 & A_{33} \end{pmatrix}

Its determinant is $\det(A_{11}) \det(A_{22}) \det(A_{33})$ .

3. Symmetric and Skew-Symmetric Matrices

Definition 4.6.3: Symmetric Matrix

A matrix $A \in M_n(F)$ is symmetric if $A = A^T$ , i.e., $a_{ij} = a_{ji}$ for all $i, j$ .

Definition 4.6.4: Skew-Symmetric Matrix

A matrix $A \in M_n(F)$ is skew-symmetric if $A = -A^T$ , i.e., $a_{ij} = -a_{ji}$ .

Theorem 4.6.3: Properties of Symmetric Matrices

Let $A$ , $B$ be symmetric. Then:

$A + B$ is symmetric
$\lambda A$ is symmetric for any scalar $\lambda$
$A^k$ is symmetric for any $k \geq 0$
If $A$ is invertible, $A^{-1}$ is symmetric
$AB$ is symmetric iff $AB = BA$

Proof:

For (4): $(A^{-1})^T = (A^T)^{-1} = A^{-1}$ .

For (5): $(AB)^T = B^T A^T = BA$ . This equals $AB$ iff they commute.

∎

Theorem 4.6.4: Properties of Skew-Symmetric Matrices

Let $A$ be skew-symmetric. Then:

All diagonal entries of $A$ are zero
$A^2$ is symmetric
If $n$ is odd, then $\det(A) = 0$
$x^T A x = 0$ for all $x \in \mathbb{R}^n$

Proof:

For (1): $a_{ii} = -a_{ii}$ implies $a_{ii} = 0$ .

For (3): $\det(A) = \det(-A^T) = (-1)^n \det(A)$ . If $n$ is odd, $\det(A) = -\det(A)$ .

∎

Theorem 4.6.5: Symmetric-Skew Decomposition

Every matrix $A$ can be uniquely written as $A = S + K$ where:

S = \frac{1}{2}(A + A^T) \text{ (symmetric)}, \quad K = \frac{1}{2}(A - A^T) \text{ (skew-symmetric)}

Example 4.6.3: Symmetric-Skew Decomposition

For $A = \begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}$ :

S = \begin{pmatrix} 1 & 5/2 \\ 5/2 & 4 \end{pmatrix}, \quad K = \begin{pmatrix} 0 & -1/2 \\ 1/2 & 0 \end{pmatrix}

Example 4.6.3b: 3×3 Symmetric Matrix

A = \begin{pmatrix} 1 & 2 & 3 \\ 2 & 5 & 4 \\ 3 & 4 & 6 \end{pmatrix}

This is symmetric: each entry equals its reflection across the diagonal. All eigenvalues will be real.

Example 4.6.3c: 3×3 Skew-Symmetric Matrix

B = \begin{pmatrix} 0 & 2 & -3 \\ -2 & 0 & 4 \\ 3 & -4 & 0 \end{pmatrix}

Note the zero diagonal and $b_{ij} = -b_{ji}$ . Since $n = 3$ is odd, $\det(B) = 0$ .

Theorem 4.6.5b: Quadratic Forms and Symmetric Matrices

Every quadratic form $Q(x) = x^T A x$ can be written using a symmetric matrix:

Q(x) = x^T \left(\frac{A + A^T}{2}\right) x

The skew-symmetric part contributes nothing since $x^T K x = 0$ for skew-symmetric $K$ .

Remark 4.6.3c: Spectral Theorem Preview

The Spectral Theorem states: every real symmetric matrix $A$ is orthogonally diagonalizable:

A = Q \Lambda Q^T

where $Q$ is orthogonal and $\Lambda$ is diagonal. This is one of the most important theorems in linear algebra.

Example 4.6.3d: Spectral Decomposition

For $A = \begin{pmatrix} 3 & 1 \\ 1 & 3 \end{pmatrix}$ :

Eigenvalues: $\lambda = 4, 2$ . Eigenvectors: $(1,1)^T, (1,-1)^T$ .

A = \frac{1}{2}\begin{pmatrix} 1 & 1 \\ 1 & -1 \end{pmatrix} \begin{pmatrix} 4 & 0 \\ 0 & 2 \end{pmatrix} \begin{pmatrix} 1 & 1 \\ 1 & -1 \end{pmatrix}

4. Orthogonal Matrices

Definition 4.6.5: Orthogonal Matrix

A real matrix $Q \in M_n(\mathbb{R})$ is orthogonal if $Q^T Q = Q Q^T = I$ , equivalently $Q^{-1} = Q^T$ .

Theorem 4.6.6: Characterizations of Orthogonal Matrices

The following are equivalent for $Q \in M_n(\mathbb{R})$ :

$Q$ is orthogonal
The columns of $Q$ form an orthonormal basis
The rows of $Q$ form an orthonormal basis
$Q$ preserves dot products: $\langle Qx, Qy \rangle = \langle x, y \rangle$
$Q$ preserves lengths: $\|Qx\| = \|x\|$

Theorem 4.6.7: Properties of Orthogonal Matrices

$\det(Q) = \pm 1$
Products of orthogonal matrices are orthogonal
$Q^{-1} = Q^T$ is orthogonal
All eigenvalues $\lambda$ satisfy $|\lambda| = 1$

Proof:

For (1): $(\det Q)^2 = \det(Q^T Q) = \det(I) = 1$ .

For (4): If $Qv = \lambda v$ , then $\|v\| = \|Qv\| = |\lambda| \|v\|$ .

∎

Example 4.6.4: 2D Rotation Matrix

Rotation by angle $\theta$ counterclockwise:

R_\theta = \begin{pmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{pmatrix}

$\det(R_\theta) = \cos^2\theta + \sin^2\theta = 1$ (proper orthogonal = rotation).

Example 4.6.5: Reflection Matrix

Reflection across the line $y = x$ :

S = \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix}

$S^2 = I$ and $\det(S) = -1$ (improper orthogonal = reflection).

Example 4.6.5b: 3D Rotation Matrix

Rotation by $\theta$ around the $z$ -axis:

R_z(\theta) = \begin{pmatrix} \cos\theta & -\sin\theta & 0 \\ \sin\theta & \cos\theta & 0 \\ 0 & 0 & 1 \end{pmatrix}

This is orthogonal with $\det = 1$ . The axis of rotation is the $z$ -axis (eigenvector with eigenvalue 1).

Example 4.6.5c: Householder Reflection

Reflection through a hyperplane perpendicular to unit vector $u$ :

H = I - 2uu^T

Verify: $H^T H = (I - 2uu^T)^2 = I - 4uu^T + 4uu^T uu^T = I$ (using $u^T u = 1$ ).

Also $\det(H) = -1$ and $H^2 = I$ .

Theorem 4.6.7b: Structure of 2×2 Orthogonal Matrices

Every $2 \times 2$ orthogonal matrix is either:

\text{Rotation: } \begin{pmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{pmatrix} \quad \text{or} \quad \text{Reflection: } \begin{pmatrix} \cos\theta & \sin\theta \\ \sin\theta & -\cos\theta \end{pmatrix}

Remark 4.6.4b: Orthogonal Group O(n)

The set of $n \\times n$ orthogonal matrices forms a group under multiplication:

Closed: $(Q_1 Q_2)^T (Q_1 Q_2) = Q_2^T Q_1^T Q_1 Q_2 = I$
Identity: $I$ is orthogonal
Inverses: $Q^{-1} = Q^T$ is orthogonal

The subgroup $SO(n)$ with $\det = 1$ consists of rotations only.

Proof:

For characterization (4) ⇒ (5): Set $y = x$ to get $\|Qx\|^2 = \langle Qx, Qx \rangle = \langle x, x \rangle = \|x\|^2$ .

For (5) ⇒ (4): Use polarization identity:

\langle x, y \rangle = \frac{1}{4}(\|x+y\|^2 - \|x-y\|^2)

∎

5. Idempotent Matrices

Definition 4.6.6: Idempotent Matrix

A matrix $P$ is idempotent if $P^2 = P$ .

Theorem 4.6.8: Properties of Idempotent Matrices

$P^k = P$ for all $k \geq 1$
Eigenvalues are only 0 and 1
$I - P$ is also idempotent
$\text{rank}(P) = \text{tr}(P)$
$\mathbb{R}^n = \text{Im}(P) \oplus \ker(P)$

Proof:

For (2): If $Pv = \lambda v$ , then $\lambda v = Pv = P^2 v = \lambda^2 v$ , so $\lambda = \lambda^2$ .

For (3): $(I-P)^2 = I - 2P + P^2 = I - 2P + P = I - P$ .

∎

Example 4.6.6: Projection onto a Line

Project onto $\text{span}\{(1,1)^T\}$ :

P = \frac{1}{2}\begin{pmatrix} 1 & 1 \\ 1 & 1 \end{pmatrix}

Verify: $P^2 = P$ , eigenvalues are 0 and 1, trace = 1 = rank.

Remark 4.6.3: Orthogonal Projection

An idempotent $P$ is an orthogonal projection iff $P = P^T$ .

Theorem 4.6.8b: Complementary Projections

If $P$ is idempotent, then:

$\text{Im}(P) = \ker(I - P)$ and $\ker(P) = \text{Im}(I - P)$
$P$ and $I - P$ are complementary projections
$P(I - P) = (I - P)P = 0$

Example 4.6.6b: Projection onto a Plane

Project onto the $xy$ -plane in $\mathbb{R}^3$ :

P = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 0 \end{pmatrix}

Then $I - P = \text{diag}(0, 0, 1)$ projects onto the $z$ -axis. Both are orthogonal projections.

Example 4.6.6c: Oblique Projection

P = \begin{pmatrix} 1 & 2 \\ 0 & 0 \end{pmatrix}

Verify $P^2 = P$ . This projects onto $\text{span}\{(1,0)^T\}$ along $\text{span}\{(-2,1)^T\}$ . Note $P \neq P^T$ , so this is not an orthogonal projection.

Remark 4.6.3b: Applications of Projections

Idempotent matrices appear throughout mathematics:

Statistics: Hat matrix $H = X(X^T X)^{-1} X^T$ in linear regression
Quantum mechanics: Projection operators represent measurements
Signal processing: Filtering operations
Optimization: Projecting onto constraint sets

6. Nilpotent Matrices

Definition 4.6.7: Nilpotent Matrix

A matrix $N$ is nilpotent if $N^k = 0$ for some $k \geq 1$ . The smallest such $k$ is the index of nilpotency.

Theorem 4.6.9: Properties of Nilpotent Matrices

All eigenvalues are zero
$\det(N) = 0$ and $\text{tr}(N) = 0$
$I - N$ is invertible: $(I-N)^{-1} = I + N + N^2 + \cdots + N^{k-1}$
For $n \\times n$ matrix: index $\leq n$

Example 4.6.7: Nilpotent Matrix

N = \begin{pmatrix} 0 & 1 & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \end{pmatrix}, \quad N^2 = \begin{pmatrix} 0 & 0 & 1 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{pmatrix}, \quad N^3 = 0

Index of nilpotency is 3.

Proof:

For property (1): If $Nv = \lambda v$ with $v \neq 0$ , then:

0 = N^k v = \lambda^k v \implies \lambda^k = 0 \implies \lambda = 0

For property (3), verify by expanding:

(I - N)(I + N + N^2 + \cdots + N^{k-1}) = I - N^k = I

∎

Example 4.6.7b: 2×2 Nilpotent Matrices

All $2 \times 2$ nilpotent matrices have the form:

N = \begin{pmatrix} a & b \\ c & -a \end{pmatrix} \text{ where } a^2 + bc = 0

Example: $\begin{pmatrix} 1 & -1 \\ 1 & -1 \end{pmatrix}$ has $N^2 = 0$ .

Example 4.6.7c: Computing (I-N)^{-1}

For $N = \begin{pmatrix} 0 & 1 \\ 0 & 0 \end{pmatrix}$ with $N^2 = 0$ :

(I - N)^{-1} = I + N = \begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix}

Verify: $(I - N)(I + N) = I - N^2 = I$ .

Remark 4.6.5b: Nilpotent and Jordan Form

Nilpotent matrices are key building blocks in Jordan canonical form. Every nilpotent matrix is similar to a direct sum of nilpotent Jordan blocks:

J_k(0) = \begin{pmatrix} 0 & 1 & 0 & \cdots & 0 \\ 0 & 0 & 1 & \cdots & 0 \\ \vdots & & \ddots & \ddots & \vdots \\ 0 & 0 & \cdots & 0 & 1 \\ 0 & 0 & \cdots & 0 & 0 \end{pmatrix}_{k \times k}

Theorem 4.6.9b: Sum of Nilpotent Matrices

If $N_1$ and $N_2$ are nilpotent and $N_1 N_2 = N_2 N_1$ (they commute), then $N_1 + N_2$ is nilpotent.

Proof:

If $N_1^p = 0$ and $N_2^q = 0$ , use binomial theorem (valid since they commute):

(N_1 + N_2)^{p+q-1} = \sum_{k=0}^{p+q-1} \binom{p+q-1}{k} N_1^k N_2^{p+q-1-k}

Each term has either $k \geq p$ (so $N_1^k = 0$ ) or $p+q-1-k \geq q$ (so $N_2^{p+q-1-k} = 0$ ).

∎

7. Involutions and Other Special Types

Definition 4.6.8: Involution

A matrix $A$ is an involution if $A^2 = I$ .

Theorem 4.6.10: Properties of Involutions

$A^{-1} = A$
Eigenvalues are $+1$ or $-1$
$\det(A) = \pm 1$

Example 4.6.8: Involutions

\begin{pmatrix} 1 & 0 \\ 0 & -1 \end{pmatrix}, \quad \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix}, \quad -I

Remark 4.6.4: Normal Matrices

A complex matrix is normal if $A^* A = A A^*$ . Examples include symmetric, skew-symmetric, orthogonal, Hermitian, and unitary matrices. Normal matrices are unitarily diagonalizable.

Proof:

For involution eigenvalues: If $Av = \lambda v$ , then $v = A^2 v = \lambda^2 v$ , so $\lambda^2 = 1$ .

∎

Definition 4.6.9: Unitary Matrix

A complex matrix $U$ is unitary if $U^* U = U U^* = I$ , where $U^* = \overline{U}^T$ .

Unitary matrices are the complex analogue of orthogonal matrices.

Definition 4.6.10: Hermitian Matrix

A complex matrix $H$ is Hermitian if $H = H^*$ .

Hermitian matrices are the complex analogue of symmetric matrices. All eigenvalues are real.

Theorem 4.6.11: Properties of Hermitian Matrices

All eigenvalues of a Hermitian matrix are real
Eigenvectors for distinct eigenvalues are orthogonal
Hermitian matrices are unitarily diagonalizable
$x^* H x \in \mathbb{R}$ for all $x$ (Rayleigh quotient is real)

Example 4.6.9: Hermitian Matrix

H = \begin{pmatrix} 2 & 1+i \\ 1-i & 3 \end{pmatrix}

Note $H = H^*$ . The eigenvalues are $\frac{5 \pm \sqrt{5}}{2}$ , both real.

Remark 4.6.6: Positive Definite Matrices

A symmetric (or Hermitian) matrix $A$ is positive definite if $x^T A x > 0$ for all $x \neq 0$ . Equivalently:

All eigenvalues are positive
All leading principal minors are positive (Sylvester criterion)
$A = B^T B$ for some invertible $B$ (Cholesky)

Example 4.6.10: Positive Definite Matrix

A = \begin{pmatrix} 2 & 1 \\ 1 & 3 \end{pmatrix}

Check: $\det(2) = 2 > 0$ and $\det(A) = 5 > 0$ . Or eigenvalues $\frac{5 \pm \sqrt{5}}{2}$ are both positive.

8. Common Mistakes

Assuming products of symmetric matrices are symmetric

If $A$ , $B$ symmetric, $AB$ is symmetric only if they commute.

Confusing orthogonal with symmetric

Orthogonal: $Q^T Q = I$ . Symmetric: $A = A^T$ . Different conditions!

Thinking nilpotent means zero

Nilpotent means $N^k = 0$ , not $N = 0$ .

Forgetting skew-symmetric diagonals are zero

$a_{ii} = -a_{ii}$ forces $a_{ii} = 0$ .

9. Historical Notes

Symmetric Matrices

The study of symmetric matrices goes back to Lagrange and Laplace in the 18th century. The spectral theorem for symmetric matrices was developed by Cauchy (1829) and later refined by Sylvester and others.

Orthogonal Matrices

Orthogonal transformations were studied by Euler in relation to rigid body motion. The term "orthogonal" comes from Greek "orthos" (right) and "gonia" (angle), reflecting the right-angle preservation property.

Jordan Normal Form

Camille Jordan (1870) developed the canonical form that bears his name, giving a complete classification of linear transformations via nilpotent matrices and diagonal blocks.

Hermitian Matrices

Named after Charles Hermite (1822-1901), who studied quadratic forms over the complex numbers. Hermitian matrices are fundamental in quantum mechanics, where observables are represented by Hermitian operators.

10. Key Takeaways

Diagonal & Triangular

Eigenvalues on diagonal. Easy to invert, multiply, and solve systems.

Symmetric

Real eigenvalues, orthogonal eigenvectors. Diagonalizable by orthogonal matrix.

Orthogonal

Preserve lengths and angles. $Q^{-1} = Q^T$ . Rotations and reflections.

Idempotent & Nilpotent

Projections ( $P^2=P$ ) and nilpotent ( $N^k=0$ ) key in Jordan form.

11. Study Tips

Memorize Key Properties

Create flashcards for: symmetric ( $A = A^T$ ), orthogonal ( $Q^T Q = I$ ), idempotent ( $P^2 = P$ ), nilpotent ( $N^k = 0$ ).

Practice Verification

Given a matrix, practice quickly checking which special properties it has. Start with 2×2 matrices before moving to larger ones.

Connect to Geometry

Orthogonal = rotations/reflections. Projections = shadows. Visualizing helps remember properties.

Understand Eigenvalue Constraints

Many special matrices restrict possible eigenvalues: symmetric → real, orthogonal → |λ| = 1, idempotent → {0,1}, nilpotent → all 0.

12. What's Next?

In the next chapter on Determinants, you will see how special matrix structure simplifies determinant computation:

Determinants of triangular matrices are products of diagonal entries
Orthogonal matrices have $\det = \pm 1$
Block diagonal determinants factor as products
Nilpotent matrices always have $\det = 0$

Later, in Eigenvalues, the spectral theorem will show that symmetric matrices are always diagonalizable with orthogonal eigenvectors—one of the most important results in linear algebra.

13. Quick Reference Table

Type	Definition	Key Property	Eigenvalues
Diagonal	$a_{ij}=0$ for $i \neq j$	Powers/inverse easy	Diagonal entries
Triangular	Zeros above/below diagonal	Products stay triangular	Diagonal entries
Symmetric	$A = A^T$	Orthogonal eigenvectors	All real
Skew-symmetric	$A = -A^T$	Zero diagonal	Purely imaginary
Orthogonal	$Q^T Q = I$	Preserves length	$\|\lambda\| = 1$
Idempotent	$P^2 = P$	Projection	0 or 1
Nilpotent	$N^k = 0$	$I-N$ invertible	All 0
Involution	$A^2 = I$	$A^{-1} = A$	±1

14. Worked Examples

Example 1: Classify a Matrix

Given $A = \begin{pmatrix} 0 & 1 & -1 \\ -1 & 0 & 2 \\ 1 & -2 & 0 \end{pmatrix}$ , classify this matrix.

Solution:

Check symmetric: $A^T = \begin{pmatrix} 0 & -1 & 1 \\ 1 & 0 & -2 \\ -1 & 2 & 0 \end{pmatrix} = -A$ . So $A$ is skew-symmetric.
Since $n = 3$ is odd, $\det(A) = 0$ (all skew-symmetric matrices of odd order are singular).
All diagonal entries are 0 (as expected for skew-symmetric).
Not orthogonal, idempotent, or nilpotent (can verify).

Example 2: Find a Projection Matrix

Find the orthogonal projection onto $\text{span}\{(1, 2, 2)^T\}$ .

Solution:

For a line spanned by unit vector $u$ , projection is $P = uu^T$ .

First normalize: $\|v\| = \sqrt{1+4+4} = 3$ , so $u = \frac{1}{3}(1, 2, 2)^T$ .

P = uu^T = \frac{1}{9}\begin{pmatrix} 1 \\ 2 \\ 2 \end{pmatrix}\begin{pmatrix} 1 & 2 & 2 \end{pmatrix} = \frac{1}{9}\begin{pmatrix} 1 & 2 & 2 \\ 2 & 4 & 4 \\ 2 & 4 & 4 \end{pmatrix}

Verify: $P^2 = P$ , $P = P^T$ , $\text{tr}(P) = 1$ , eigenvalues are 0, 0, 1.

Example 3: Verify Orthogonality

Show that $Q = \frac{1}{3}\begin{pmatrix} 1 & -2 & 2 \\ 2 & 2 & 1 \\ 2 & -1 & -2 \end{pmatrix}$ is orthogonal.

Solution:

Check that columns are orthonormal:

Column 1: $\frac{1}{9}(1+4+4) = 1$ ✓
Column 2: $\frac{1}{9}(4+4+1) = 1$ ✓
Column 3: $\frac{1}{9}(4+1+4) = 1$ ✓
Col 1 · Col 2: $\frac{1}{9}(-2+4-2) = 0$ ✓
Col 1 · Col 3: $\frac{1}{9}(2+2-4) = 0$ ✓
Col 2 · Col 3: $\frac{1}{9}(-4+2+2) = 0$ ✓

Thus $Q^T Q = I$ . Also $\det(Q) = \frac{1}{27}(1 \cdot (-3) - (-2) \cdot (-6) + 2 \cdot (-6)) = 1$ , so $Q$ is a rotation.

Example 4: Decompose into Symmetric + Skew

Decompose $A = \begin{pmatrix} 1 & 2 & 3 \\ 0 & 4 & 5 \\ 0 & 0 & 6 \end{pmatrix}$ into symmetric + skew-symmetric.

Solution:

S = \frac{1}{2}(A + A^T) = \frac{1}{2}\begin{pmatrix} 2 & 2 & 3 \\ 2 & 8 & 5 \\ 3 & 5 & 12 \end{pmatrix} = \begin{pmatrix} 1 & 1 & 3/2 \\ 1 & 4 & 5/2 \\ 3/2 & 5/2 & 6 \end{pmatrix}

K = \frac{1}{2}(A - A^T) = \frac{1}{2}\begin{pmatrix} 0 & 2 & 3 \\ -2 & 0 & 5 \\ -3 & -5 & 0 \end{pmatrix} = \begin{pmatrix} 0 & 1 & 3/2 \\ -1 & 0 & 5/2 \\ -3/2 & -5/2 & 0 \end{pmatrix}

Verify: $S + K = A$ , $S = S^T$ , $K = -K^T$ .

Example 5: Nilpotent Matrix Computation

For $N = \begin{pmatrix} 1 & 1 \\ -1 & -1 \end{pmatrix}$ , show $N$ is nilpotent and find $(I - N)^{-1}$ .

Solution:

N^2 = \begin{pmatrix} 1 & 1 \\ -1 & -1 \end{pmatrix}\begin{pmatrix} 1 & 1 \\ -1 & -1 \end{pmatrix} = \begin{pmatrix} 0 & 0 \\ 0 & 0 \end{pmatrix}

So $N$ is nilpotent with index 2. Thus:

(I - N)^{-1} = I + N = \begin{pmatrix} 2 & 1 \\ -1 & 0 \end{pmatrix}

Verify: $(I-N)(I+N) = I - N^2 = I$ .

15. Applications Overview

Computer Graphics

Orthogonal matrices represent rotations and reflections in 3D graphics. Transformation matrices preserve shape when rendering objects.

Quantum Mechanics

Hermitian operators represent observables with real eigenvalues. Unitary operators represent time evolution preserving probability.

Statistics

Projection matrices compute least squares fits. Covariance matrices are symmetric positive semi-definite.

Numerical Analysis

LU decomposition uses triangular matrices for efficient solving. QR decomposition uses orthogonal matrices for stability.

Control Theory

Nilpotent matrices describe systems that "stop" after finite time. Stability analysis uses eigenvalue locations.

Machine Learning

Symmetric matrices appear in kernel methods. PCA uses eigenvectors of covariance (symmetric) matrices.

16. Relationships Between Special Matrix Types

Venn Diagram of Special Matrices

Understanding how special matrix types relate to each other:

Diagonal ⊂ Triangular: Every diagonal matrix is both upper and lower triangular.
Diagonal ⊂ Symmetric: Every diagonal matrix is symmetric.
Orthogonal ∩ Symmetric = Involutions: $Q^T = Q^{-1}$ and $Q^T = Q$ implies $Q^2 = I$ .
Symmetric ∩ Skew-symmetric = {0}: Only the zero matrix is both.
Nilpotent ⊂ Singular: All nilpotent matrices have det = 0.
Idempotent ≠ Nilpotent: Except $P = 0$ , these are distinct (eigenvalues 0,1 vs all 0).

Theorem 4.6.12: Orthogonal + Symmetric = Involution

A matrix is both orthogonal and symmetric if and only if it is an involution with eigenvalues $\pm 1$ .

Proof:

If $Q$ is orthogonal ( $Q^T Q = I$ ) and symmetric ( $Q = Q^T$ ), then $Q^2 = Q \cdot Q^T = I$ .

Conversely, if $Q^2 = I$ and $Q = Q^T$ , then $Q^T Q = Q \cdot Q = I$ , so $Q$ is orthogonal.

∎

Remark 4.6.7: Decomposition Theorems

Several important decomposition theorems relate special matrix types:

Schur: Every matrix is unitarily similar to upper triangular.
Spectral: Normal matrices are unitarily diagonalizable.
Polar: $A = QP$ with $Q$ orthogonal, $P$ symmetric positive semi-definite.
SVD: $A = U \Sigma V^T$ with $U, V$ orthogonal, $\Sigma$ diagonal.

17. Computational Considerations

Complexity Benefits

Matrix Type	Operation	General	Special
Diagonal	Inverse	$O(n^3)$	$O(n)$
Triangular	Solve $Ax=b$	$O(n^3)$	$O(n^2)$
Symmetric	Eigenvalues	$O(n^3)$	$O(n^3)$ (faster const)
Orthogonal	Inverse	$O(n^3)$	$O(n^2)$ (transpose)
Sparse	Multiply	$O(n^2)$	$O(nnz)$

Remark 4.6.8: Numerical Stability

Special structure often improves numerical stability:

Symmetric positive definite: Cholesky is stable without pivoting.
Orthogonal: Condition number is 1 (perfectly conditioned).
Triangular: Back-substitution is stable for well-conditioned systems.
Diagonal: Operations are exact (no accumulation of errors).

Example 4.6.11: Exploiting Structure

To solve $Ax = b$ where $A = LDL^T$ (symmetric factorization):

Solve $Ly = b$ (lower triangular) - $O(n^2)$
Solve $Dz = y$ (diagonal) - $O(n)$
Solve $L^T x = z$ (upper triangular) - $O(n^2)$

Total: $O(n^2)$ after factorization, vs $O(n^3)$ for general Gaussian elimination.

Remark 4.6.9: Storage Optimization

Special matrices can be stored more efficiently:

Diagonal: Store only $n$ entries instead of $n^2$ .
Triangular: Store $n(n+1)/2$ entries.
Symmetric: Store only upper (or lower) triangle - $n(n+1)/2$ entries.
Banded: For bandwidth $k$ , store $O(nk)$ entries.
Sparse: Use compressed formats (CSR, CSC) storing only nonzeros.

Special Matrices Practice

18

Questions

0

Correct

0%

Accuracy

1

A matrix

A

is symmetric if and only if:

Easy

Not attempted

2

If

A

is orthogonal, then

A^{-1} = ?

Easy

Not attempted

3

The product of two upper triangular matrices is:

Medium

Not attempted

4

If

A

is skew-symmetric, all diagonal entries are:

Medium

Not attempted

5

A matrix

P

is idempotent if:

Easy

Not attempted

6

If

N

is nilpotent with

N^3 = 0

but

N^2 \neq 0

, the index is:

Medium

Not attempted

7

The determinant of an orthogonal matrix is:

Medium

Not attempted

8

Every matrix can be written as

A = S + K

where:

Medium

Not attempted

9

All eigenvalues of a nilpotent matrix are:

Hard

Not attempted

10

The eigenvalues of an idempotent matrix are:

Hard

Not attempted

11

An upper triangular matrix is invertible iff:

Medium

Not attempted

12

If

A

is symmetric and

B

is any matrix, then

B^T A B

is:

Medium

Not attempted

13

The columns of an orthogonal matrix form:

Medium

Not attempted

14

If

A

is both symmetric and skew-symmetric, then:

Easy

Not attempted

15

Rotation matrices in

\mathbb{R}^2

are:

Easy

Not attempted

16

A Householder matrix

H = I - 2uu^T

with

\|u\|=1

is:

Hard

Not attempted

17

If

A

is skew-symmetric, then

e^A

is:

Hard

Not attempted

18

The rank of an

n \times n

idempotent matrix equals its:

Hard

Not attempted

Frequently Asked Questions

Why are special matrices important?

Special matrices have structure that simplifies computations and reveals properties. Diagonal matrices are easy to invert and power, triangular matrices make solving systems efficient, and symmetric matrices have real eigenvalues with orthogonal eigenvectors.

What's the geometric meaning of an orthogonal matrix?

Orthogonal matrices represent isometries—transformations that preserve lengths and angles. In 2D/3D, these are rotations (det = 1) and reflections (det = -1). They preserve the dot product: ⟨Ax, Ay⟩ = ⟨x, y⟩.

How do I check if a matrix is symmetric?

Check if A = A^T, i.e., whether a_{ij} = a_{ji} for all i, j. Visually, check if the matrix equals its reflection across the main diagonal.

What are idempotent matrices used for?

Idempotent matrices represent projections. If P² = P, then P projects vectors onto its image. Applying the projection twice gives the same result—hence P² = P. They're central in statistics and quantum mechanics.

What is a nilpotent matrix?

A matrix N is nilpotent if N^k = 0 for some positive integer k. The smallest such k is the index of nilpotency. Nilpotent matrices have all eigenvalues equal to zero and appear in Jordan forms.

Why do triangular matrices have eigenvalues on the diagonal?

For triangular A, det(A - λI) is the product of (a_{ii} - λ) terms. This equals zero exactly when λ equals some diagonal entry.

What's the symmetric-skew decomposition?

Every matrix A can be uniquely written as A = S + K where S = (A + A^T)/2 is symmetric and K = (A - A^T)/2 is skew-symmetric. This is analogous to decomposing a function into even and odd parts.

Are products of symmetric matrices symmetric?

Generally no! If A and B are symmetric, (AB)^T = B^T A^T = BA, which equals AB only if A and B commute. However, B^T A B is always symmetric when A is symmetric.

What matrices are both orthogonal and symmetric?

If A is orthogonal and symmetric, then A² = A·A^T = I. So A is an involution. The only such matrices have eigenvalues ±1: they're reflections across subspaces.

How do special matrices simplify eigenvalue problems?

Diagonal: eigenvalues are diagonal entries. Triangular: same. Symmetric: always diagonalizable with orthogonal eigenvectors. Orthogonal: eigenvalues have |λ| = 1. This structure makes analysis much easier.

What is the difference between orthogonal and unitary matrices?

Orthogonal matrices are for real vector spaces: Q^T Q = I. Unitary matrices are for complex spaces: U* U = I where U* is conjugate transpose. Both preserve inner products in their respective spaces.

How do I construct a projection matrix onto a subspace?

If V has orthonormal basis {v₁,...,vₖ}, then P = Σᵢ vᵢvᵢᵀ = VVᵀ where V = [v₁|...|vₖ]. If the basis isn't orthonormal, use P = V(VᵀV)⁻¹Vᵀ.

What is the Cholesky decomposition?

Every positive definite matrix A can be written as A = LLᵀ where L is lower triangular with positive diagonal. This is more efficient than LU for symmetric positive definite systems.

When is a matrix diagonalizable?

A matrix is diagonalizable iff it has n linearly independent eigenvectors. Sufficient conditions: distinct eigenvalues, or being normal (including symmetric, orthogonal, Hermitian, unitary).

What is a circulant matrix?

Each row is a cyclic shift of the row above. Circulants are diagonalized by the DFT matrix and appear in signal processing. They commute with each other.