LA-C7

Available

Course 7: Matrix Inverses, Elementary Matrices & Dual Spaces

This course deepens your understanding of matrix theory by covering inverses, the building blocks of row operations (elementary matrices), the fundamental concept of rank, and the elegant theory of dual spaces. These topics connect matrix computations to abstract linear algebra.

12-15 hours Advanced Core Level 10 Objectives

Learning Objectives

Define and compute matrix inverses using Gaussian elimination.
Understand the invertible matrix theorem and its many characterizations.
Master elementary matrices and their role in row operations.
Express invertible matrices as products of elementary matrices.
Define matrix rank and prove row rank equals column rank.
Apply rank to determine solvability of linear systems.
Understand rank inequalities and their applications.
Define dual spaces and linear functionals.
Construct dual bases and understand the natural isomorphism V ≅ V**.
Compute annihilators and understand their dimension formula.

Prerequisites

LA-C6: Matrix Theory
Matrix operations and representation
Gaussian elimination
Linear maps and kernel/image
Basis and dimension

Historical Context

The concept of matrix inverses emerged naturally from solving systems of linear equations, with Arthur Cayley (1821–1895) formalizing matrix algebra in 1858. Elementary matrices, though implicit in Gaussian elimination, were explicitly studied in the early 20th century. The rank of a matrix, a fundamental invariant, was recognized by James Joseph Sylvester in the 1850s. The beautiful theorem that row rank equals column rank was proven by various mathematicians, with Emmy Noetherproviding elegant abstract proofs. Dual spaces, introduced by Hermann Grassmannin the 1840s and later developed by Jean Dieudonné and others, provide a powerful framework for understanding linear maps and their adjoints.

1. Matrix Inverses

The inverse of a matrix $A$ is the unique matrix $A^{-1}$ such that $AA^{-1} = A^{-1}A = I$ . Invertible matrices correspond to isomorphisms and are fundamental for solving linear systems.

Definition 1.1: Matrix Inverse

An $n \times n$ matrix $A$ is invertible (or nonsingular) if there exists $A^{-1}$ such that $AA^{-1} = A^{-1}A = I_n$ .

Theorem 1.1: Uniqueness of Inverse

If an inverse exists, it is unique.

Proof:

If $B, C$ are both inverses of $A$ , then $B = BI = B(AC) = (BA)C = IC = C$ .

∎

Theorem 1.2: Invertible Matrix Theorem

For an $n \times n$ matrix $A$ , the following are equivalent:

$A$ is invertible
$\text{rank}(A) = n$
The columns of $A$ are linearly independent
$Ax = 0$ has only the trivial solution
$Ax = b$ has a unique solution for every $b$
$A$ is a product of elementary matrices
$A$ is row equivalent to $I_n$

Example 1.1: Computing Inverse via Gaussian Elimination

To find $A^{-1}$ , row reduce $[A|I]$ to $[I|A^{-1}]$ . For example, if $A = \begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}$ , then $A^{-1} = \begin{pmatrix} -2 & 1 \\ 3/2 & -1/2 \end{pmatrix}$ .

Example 1.2: Formula for 2×2 Inverse

For $A = \begin{pmatrix} a & b \\ c & d \end{pmatrix}$ with $ad - bc \neq 0$ :

A^{-1} = \frac{1}{ad - bc} \begin{pmatrix} d & -b \\ -c & a \end{pmatrix}

Remark 1.1: Computational Complexity

Computing $A^{-1}$ via Gaussian elimination is $O(n^3)$ operations. For large matrices, iterative methods or specialized algorithms may be more efficient.

Theorem 1.3: Properties of Inverse

$(A^{-1})^{-1} = A$
$(AB)^{-1} = B^{-1}A^{-1}$ (order reverses!)
$(A^T)^{-1} = (A^{-1})^T$
$(cA)^{-1} = \frac{1}{c}A^{-1}$ for $c \neq 0$

2. Elementary Matrices

Elementary matrices are obtained by applying one elementary row operation to the identity matrix. They are the building blocks of Gaussian elimination and provide a factorization of invertible matrices.

Definition 2.1: Elementary Matrix

An elementary matrix is obtained by applying one elementary row operation to $I$ .

Three types:

Type I (swap): $E_{ij}$ swaps rows $i$ and $j$
Type II (scale): $E_i(c)$ multiplies row $i$ by $c \neq 0$
Type III (add): $E_{ij}(c)$ adds $c$ times row $i$ to row $j$

Theorem 2.1: Elementary Matrices are Invertible

Every elementary matrix is invertible, and its inverse is also elementary.

Theorem 2.2: Row Operations via Multiplication

Left-multiplying by an elementary matrix performs the corresponding row operation:

$E_{ij}A$ swaps rows $i$ and $j$ of $A$
$E_i(c)A$ multiplies row $i$ of $A$ by $c$
$E_{ij}(c)A$ adds $c$ times row $i$ to row $j$ of $A$

Theorem 2.3: Factorization Theorem

An $n \times n$ matrix $A$ is invertible if and only if it is a product of elementary matrices.

Proof:

If $A$ is invertible, row reduce $A$ to $I$ : $E_k \cdots E_1 A = I$ , so $A = E_1^{-1} \cdots E_k^{-1}$ . Conversely, products of invertible matrices are invertible.

∎

Example 2.1: Gaussian Elimination as Matrix Multiplication

If $E_3 E_2 E_1 A = U$ (row echelon form), then $A = E_1^{-1} E_2^{-1} E_3^{-1} U$ . This factorization is the basis for LU decomposition.

Example 2.2: Elementary Matrix Examples

For $3 \times 3$ matrices:

Swap rows 1 and 2: $E_{12} = \begin{pmatrix} 0 & 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \end{pmatrix}$
Scale row 2 by 3: $E_2(3) = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 3 & 0 \\ 0 & 0 & 1 \end{pmatrix}$
Add 2×row 1 to row 3: $E_{13}(2) = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 2 & 0 & 1 \end{pmatrix}$

Remark 2.1: Inverse of Elementary Matrices

The inverse of an elementary matrix is also elementary:

$E_{ij}^{-1} = E_{ij}$ (swap is its own inverse)
$E_i(c)^{-1} = E_i(1/c)$
$E_{ij}(c)^{-1} = E_{ij}(-c)$

3. Matrix Rank

The rank of a matrix is the dimension of its row space (or column space). It is a fundamental invariant that determines solvability of linear systems and invertibility.

Definition 3.1: Matrix Rank

The rank of a matrix $A$ , denoted $\text{rank}(A)$ , is:

The dimension of the row space of $A$ (row rank)
The dimension of the column space of $A$ (column rank)
The number of pivots after row reduction

Theorem 3.1: Row Rank Equals Column Rank

For any matrix $A$ , row rank = column rank.

Proof:

Both equal $\dim(\text{im } T_A)$ where $T_A: x \mapsto Ax$ is the linear map. Alternatively, row operations preserve row rank and show column rank equals it.

∎

Theorem 3.2: Rank and Invertibility

An $n \times n$ matrix $A$ is invertible if and only if $\text{rank}(A) = n$ .

Theorem 3.3: Rank Inequalities

$\text{rank}(AB) \leq \min(\text{rank}(A), \text{rank}(B))$
$\text{rank}(A + B) \leq \text{rank}(A) + \text{rank}(B)$
$\text{rank}(A^T) = \text{rank}(A)$
$\text{rank}(A^T A) = \text{rank}(A)$

Example 3.1: Rank and Linear Systems

For $Ax = b$ :

Solvable iff $\text{rank}(A) = \text{rank}([A|b])$
Unique solution iff additionally $\text{rank}(A) = n$
Dimension of solution space = $n - \text{rank}(A)$

Corollary 3.1: Rank-Nullity for Matrices

For $A \in M_{m \times n}(F)$ , $n = \text{rank}(A) + \dim(\ker A)$ , where $\dim(\ker A)$ is the nullity of $A$ .

Theorem 3.4: Sylvester's Rank Inequality

For matrices $A$ and $B$ of compatible sizes:

\text{rank}(A) + \text{rank}(B) - n \leq \text{rank}(AB) \leq \min(\text{rank}(A), \text{rank}(B))

where $n$ is the number of columns of $A$ (or rows of $B$ ).

4. Dual Spaces

The dual space $V^*$ consists of all linear functionals (linear maps from $V$ to $F$ ). It has the same dimension as $V$ and provides a natural way to study vectors through their evaluations.

Definition 4.1: Linear Functional

A linear functional on $V$ is a linear map $\phi: V \to F$ .

Definition 4.2: Dual Space

The dual space of $V$ is $V^* = \mathcal{L}(V, F)$ , the space of all linear functionals on $V$ .

Theorem 4.1: Dimension of Dual Space

If $\dim V = n$ (finite), then $\dim V^* = n$ .

Proof:

$V^* = \mathcal{L}(V, F)$ , so $\dim V^* = \dim V \cdot \dim F = n \cdot 1 = n$ .

∎

Definition 4.3: Dual Basis

If $\mathcal{B} = \{v_1, \ldots, v_n\}$ is a basis for $V$ , the dual basis $\mathcal{B}^* = \{v_1^*, \ldots, v_n^*\}$ is defined by $v_i^*(v_j) = \delta_{ij}$ (Kronecker delta).

Theorem 4.2: Dual Basis is a Basis

The dual basis $\mathcal{B}^*$ is indeed a basis for $V^*$ .

Definition 4.4: Annihilator

For a subspace $W \subseteq V$ , the annihilator is $W^0 = \{\phi \in V^* : \phi(w) = 0 \text{ for all } w \in W\}$ .

Theorem 4.3: Dimension of Annihilator

If $\dim V = n$ and $\dim W = k$ , then $\dim(W^0) = n - k$ .

Definition 4.5: Double Dual

The double dual is $V^{**} = (V^*)^*$ . The evaluation map $\text{ev}: V \to V^{**}$ sends $v$ to $\text{ev}_v$ where $\text{ev}_v(\phi) = \phi(v)$ .

Theorem 4.4: Natural Isomorphism

For finite-dimensional $V$ , the evaluation map $V \to V^{**}$ is an isomorphism. This is a natural isomorphism (doesn't depend on basis choice).

Definition 4.6: Dual Map

For $T: V \to W$ , the dual map (or transpose) is $T^*: W^* \to V^*$ defined by $T^*(\phi) = \phi \circ T$ .

Theorem 4.5: Properties of Dual Map

$(ST)^* = T^* S^*$ (order reverses!)
$\ker(T^*) = (\text{im } T)^0$
$\text{rank}(T^*) = \text{rank}(T)$

Example 4.1: Dual Space of ℝⁿ

For $V = \mathbb{R}^n$ , every linear functional $\phi: \mathbb{R}^n \to \mathbb{R}$ is of the form:

\phi(x) = a_1 x_1 + a_2 x_2 + \cdots + a_n x_n = a^T x

for some $a = (a_1, \ldots, a_n)^T \in \mathbb{R}^n$ . So $(\mathbb{R}^n)^* \cong \mathbb{R}^n$ .

Remark 4.1: Dual Space in Coordinates

If $\mathcal{B} = \{v_1, \ldots, v_n\}$ is a basis of $V$ , then the dual basis $\mathcal{B}^*$ provides coordinates for $V^*$ . For $\phi \in V^*$ , $\phi = \sum \phi(v_i) v_i^*$ .

5. Applications of Matrix Inverses

Matrix inverses are fundamental for solving linear systems, computing matrix functions, and many other applications in mathematics and engineering.

Theorem 5.1: Solving Linear Systems

For an invertible $n \times n$ matrix $A$ , the system $Ax = b$ has the unique solution:

x = A^{-1} b

Proof:

Multiply both sides of $Ax = b$ by $A^{-1}$ :

A^{-1}(Ax) = A^{-1}b \implies (A^{-1}A)x = A^{-1}b \implies Ix = A^{-1}b \implies x = A^{-1}b

∎

Example 5.1: Solving a 2×2 System

Solve $\begin{cases} x + 2y = 5 \\ 3x + 4y = 6 \end{cases}$ .

In matrix form: $\begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} 5 \\ 6 \end{pmatrix}$

Using the inverse:

\begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} -2 & 1 \\ 3/2 & -1/2 \end{pmatrix} \begin{pmatrix} 5 \\ 6 \end{pmatrix} = \begin{pmatrix} -4 \\ 9/2 \end{pmatrix}

Remark 5.1: Computational Note

While $x = A^{-1}b$ is theoretically correct, in practice it's usually more efficient to solve $Ax = b$ directly via Gaussian elimination rather than computing $A^{-1}$ first. However, if you need to solve $Ax = b_i$ for many different $b_i$ , computing $A^{-1}$ once may be more efficient.

Theorem 5.2: Matrix Inversion and Linear Independence

The columns of an invertible matrix $A$ form a basis for $F^n$ . Conversely, if the columns of $A$ form a basis, then $A$ is invertible.

Example 5.2: Change of Coordinates

If $\mathcal{B} = \{v_1, \ldots, v_n\}$ is a basis, the change of basis matrix $P$ from $\mathcal{B}$ to standard basis has columns $v_1, \ldots, v_n$ .

Then $P^{-1}$ converts standard coordinates to $\mathcal{B}$ -coordinates: $[v]_\mathcal{B} = P^{-1}[v]_\mathcal{E}$ .

Theorem 5.3: Inverse and Matrix Equations

For invertible $A$ and $B$ , the matrix equation $AX = B$ has unique solution $X = A^{-1}B$ , and $XA = B$ has unique solution $X = BA^{-1}$ .

Example 5.3: Finding Matrix X

If $AX = B$ where $A$ is invertible, then $X = A^{-1}B$ .

This is useful for solving systems of matrix equations and finding matrix square roots (when $B = A$ ).

Course 7 Practice Quiz

Questions

Correct

Accuracy

A^{-1}

exists, what is

AA^{-1}

Easy

Not attempted

(AB)^{-1} = ?

Medium

Not attempted

Elementary matrices are always:

Easy

Not attempted

The rank of a matrix equals:

Easy

Not attempted

Row rank equals column rank:

Medium

Not attempted

What is

\dim(V^*)

\dim(V) = n

Easy

Not attempted

The dual of a linear map

T: V \to W

T^*: ? \to ?

Medium

Not attempted

\dim(V) = n

and

\dim(W) = k

, what is

\dim(W^0)

Hard

Not attempted

A square matrix is invertible iff its rank equals:

Easy

Not attempted

Every invertible matrix is a product of:

Medium

Not attempted

Frequently Asked Questions

What are the characterizations of invertibility?

For an n×n matrix A, invertibility is equivalent to: det(A) ≠ 0, rank(A) = n, columns/rows linearly independent, Ax = 0 has only trivial solution, Ax = b has unique solution for all b, A is product of elementary matrices, A is row equivalent to I, and more.

How do elementary matrices relate to Gaussian elimination?

Each step of Gaussian elimination is a row operation, which equals left-multiplying by an elementary matrix. The entire process is E_k ⋯ E_1 A = U, so Gaussian elimination is just matrix multiplication!

Why does row rank equal column rank?

This beautiful theorem has multiple proofs. One key insight: both equal dim(im A) = dim(im A^T), viewing A as a linear map. Row operations preserve row rank but show column rank equals it.

What is the dual space and why is it important?

The dual space V* is the space of all linear functionals (linear maps from V to F). It has the same dimension as V and provides a natural way to study vectors through their 'evaluations' on functionals. The double dual V** is naturally isomorphic to V.

How do I compute the annihilator of a subspace?

The annihilator W^0 of a subspace W ⊆ V is the set of all functionals that vanish on W. If dim(V) = n and dim(W) = k, then dim(W^0) = n - k. This follows from the rank-nullity theorem applied to the evaluation map.