1. Matrix Representation of Linear Maps

Definition 6.1: Matrix of a Linear Map

Let $T: V \to W$ be a linear map, $\mathcal{B} = \{v_1, \ldots, v_n\}$ a basis of $V$ , and $\mathcal{C} = \{w_1, \ldots, w_m\}$ a basis of $W$ .

The matrix of $T$ with respect to $\mathcal{B}$ and $\mathcal{C}$ , denoted $[T]_\mathcal{B}^\mathcal{C}$ , is the $m \times n$ matrix whose $j$ -th column is $[T(v_j)]_\mathcal{C}$ (the coordinate vector of $T(v_j)$ in basis $\mathcal{C}$ ).

Theorem 6.1: Matrix-Vector Multiplication

If $[T]_\mathcal{B}^\mathcal{C} = A$ and $[v]_\mathcal{B} = x$ , then:

[T(v)]_\mathcal{C} = Ax

Example 6.1: Finding Matrix Representation

Let $T: \mathbb{R}^2 \to \mathbb{R}^2$ be defined by $T(x, y) = (x + y, x - y)$ .

Using the standard basis $\{e_1, e_2\}$ :

$T(e_1) = T(1, 0) = (1, 1)$
$T(e_2) = T(0, 1) = (1, -1)$

So the matrix is:

[T] = \begin{pmatrix} 1 & 1 \\ 1 & -1 \end{pmatrix}

Theorem 6.2: Composition and Matrix Multiplication

If $T: V \to W$ and $S: W \to U$ are linear maps, then:

[S \circ T]_\mathcal{B}^\mathcal{D} = [S]_\mathcal{C}^\mathcal{D} [T]_\mathcal{B}^\mathcal{C}

where $\mathcal{B}, \mathcal{C}, \mathcal{D}$ are bases of $V, W, U$ respectively.

Remark 1.1: Matrix Representation is Basis-Dependent

The matrix representation of a linear map depends on the choice of bases. Changing bases changes the matrix, but the underlying linear map remains the same.

Example 1.2: Matrix of Zero and Identity Maps

The zero map $0: V \to W$ has matrix representation $0$ (zero matrix) in any bases
The identity map $I_V: V \to V$ has matrix representation $I_n$ (identity matrix) in any basis

2. Matrix Operations

Definition 6.2: Matrix Addition and Scalar Multiplication

For $m \times n$ matrices $A$ and $B$ over field $F$ :

Addition: $(A + B)_{ij} = A_{ij} + B_{ij}$
Scalar multiplication: $(\alpha A)_{ij} = \alpha A_{ij}$

Definition 6.3: Matrix Multiplication

If $A$ is $m \times n$ and $B$ is $n \times p$ , then $AB$ is $m \times p$ with:

(AB)_{ij} = \sum_{k=1}^n A_{ik} B_{kj}

This is the dot product of row $i$ of $A$ with column $j$ of $B$ .

Theorem 6.3: Properties of Matrix Multiplication

Matrix multiplication satisfies:

Associativity: $(AB)C = A(BC)$
Distributivity: $A(B + C) = AB + AC$ and $(A + B)C = AC + BC$
Scalar compatibility: $\alpha(AB) = (\alpha A)B = A(\alpha B)$
Identity: $AI = IA = A$ where $I$ is the identity matrix

Note: Matrix multiplication is NOT commutative in general.

Example 6.2: Non-Commutativity

Let $A = \begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix}$ and $B = \begin{pmatrix} 1 & 0 \\ 1 & 1 \end{pmatrix}$ .

Then:

AB = \begin{pmatrix} 2 & 1 \\ 1 & 1 \end{pmatrix}, \quad BA = \begin{pmatrix} 1 & 1 \\ 1 & 2 \end{pmatrix}

So $AB \neq BA$ .

Definition 6.4: Transpose

The transpose of an $m \times n$ matrix $A$ , denoted $A^T$ , is the $n \times m$ matrix with:

(A^T)_{ij} = A_{ji}

Theorem 6.4: Properties of Transpose

For matrices $A$ and $B$ and scalar $\alpha$ :

$(A^T)^T = A$
$(A + B)^T = A^T + B^T$
$(\alpha A)^T = \alpha A^T$
$(AB)^T = B^T A^T$ (note the order reversal)

Theorem 2.1: Matrix Multiplication and Linear Combinations

For $A \in M_{m \times n}(F)$ and $x = (x_1, \ldots, x_n)^T$ :

Ax = x_1 A_1 + x_2 A_2 + \cdots + x_n A_n

where $A_i$ is the $i$ -th column of $A$ . So $Ax$ is a linear combination of the columns of $A$ .

Example 2.1: Column Space

The column space of $A$ is $\text{span}\{A_1, \ldots, A_n\}$ , which equals $\text{im}(T_A)$ where $T_A(x) = Ax$ .

3. Special Matrices

Definition 6.5: Diagonal Matrix

A square matrix $D$ is diagonal if $D_{ij} = 0$ for all $i \neq j$ .

We write $D = \text{diag}(d_1, d_2, \ldots, d_n)$ .

Definition 6.6: Triangular Matrices

A square matrix $A$ is:

Upper triangular if $A_{ij} = 0$ for all $i > j$
Lower triangular if $A_{ij} = 0$ for all $i < j$

Definition 6.7: Symmetric and Skew-Symmetric

A square matrix $A$ is:

Symmetric if $A = A^T$
Skew-symmetric if $A = -A^T$

Theorem 6.5: Symmetric-Skew Decomposition

Every square matrix $A$ can be uniquely written as:

A = S + K

where $S = \frac{1}{2}(A + A^T)$ is symmetric and $K = \frac{1}{2}(A - A^T)$ is skew-symmetric.

Definition 6.8: Orthogonal Matrix

A square matrix $Q$ is orthogonal if:

Q^T Q = I

Equivalently, $Q^{-1} = Q^T$ .

Theorem 6.6: Properties of Orthogonal Matrices

For an orthogonal matrix $Q$ :

The columns (and rows) form an orthonormal set
$\det(Q) = \pm 1$
$Q$ preserves lengths: $\|Qx\| = \|x\|$
$Q$ preserves angles: $\langle Qx, Qy \rangle = \langle x, y \rangle$

Definition 6.9: Idempotent and Nilpotent

A square matrix $P$ is:

Idempotent if $P^2 = P$
Nilpotent if $P^k = 0$ for some positive integer $k$

Example 6.3: Special Matrix Examples

Diagonal: $\text{diag}(2, 3, 5)$
Symmetric: $\begin{pmatrix} 1 & 2 \\ 2 & 3 \end{pmatrix}$
Orthogonal: $\begin{pmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{pmatrix}$ (rotation)
Idempotent: $\begin{pmatrix} 1 & 0 \\ 0 & 0 \end{pmatrix}$ (projection onto x-axis)

Definition 3.1: Unitary and Hermitian Matrices

For complex matrices:

Unitary: $Q^* Q = I$ where $Q^* = \overline{Q^T}$ (conjugate transpose)
Hermitian: $A = A^*$ (self-adjoint)

Theorem 3.1: Properties of Special Matrices

Product of orthogonal matrices is orthogonal
Product of upper (lower) triangular matrices is upper (lower) triangular
Inverse of orthogonal matrix is orthogonal
Transpose of symmetric matrix is symmetric

4. Change of Basis for Matrices

When we change bases, the matrix representation of a linear map changes. Understanding this relationship is crucial for diagonalization, similarity, and many other applications.

Theorem 4.1: Change of Basis Formula

Let $T: V \to V$ be a linear operator, and let $\mathcal{B}$ and $\mathcal{C}$ be two bases of $V$ . If $P$ is the change of basis matrix from $\mathcal{B}$ to $\mathcal{C}$ , then:

[T]_\mathcal{C} = P^{-1} [T]_\mathcal{B} P

Proof:

For $v \in V$ , let $[v]_\mathcal{B}$ and $[v]_\mathcal{C}$ be coordinate vectors. Then:

[T(v)]_\mathcal{C} = [T]_\mathcal{C} [v]_\mathcal{C} = [T]_\mathcal{C} P [v]_\mathcal{B}

Also:

[T(v)]_\mathcal{C} = P [T(v)]_\mathcal{B} = P [T]_\mathcal{B} [v]_\mathcal{B}

Since this holds for all $v$ , we get $[T]_\mathcal{C} P = P [T]_\mathcal{B}$ , so $[T]_\mathcal{C} = P [T]_\mathcal{B} P^{-1}$ .

∎

Definition 4.1: Similar Matrices

Two $n \times n$ matrices $A$ and $B$ are similar if there exists an invertible matrix $P$ such that:

B = P^{-1} A P

Similar matrices represent the same linear operator in different bases.

Theorem 4.2: Properties of Similarity

Similarity is an equivalence relation:

Reflexive: $A \sim A$ (take $P = I$ )
Symmetric: If $B = P^{-1} A P$ , then $A = (P^{-1})^{-1} B P^{-1}$
Transitive: If $B = P^{-1} A P$ and $C = Q^{-1} B Q$ , then $C = (PQ)^{-1} A (PQ)$

Theorem 4.3: Similar Matrices Share Properties

If $A \sim B$ , then:

$\det(A) = \det(B)$
$\text{rank}(A) = \text{rank}(B)$
$\text{tr}(A) = \text{tr}(B)$ (trace)
$A$ and $B$ have the same characteristic polynomial and eigenvalues

Example 4.1: Change of Basis Example

Let $T: \mathbb{R}^2 \to \mathbb{R}^2$ be $T(x, y) = (x + y, x - y)$ .

In standard basis $\mathcal{E}$ : $[T]_\mathcal{E} = \begin{pmatrix} 1 & 1 \\ 1 & -1 \end{pmatrix}$

In basis $\mathcal{B} = \{(1, 1), (1, -1)\}$ :

Change of basis matrix: $P = \begin{pmatrix} 1 & 1 \\ 1 & -1 \end{pmatrix}$

Then $[T]_\mathcal{B} = P^{-1} [T]_\mathcal{E} P = \begin{pmatrix} 2 & 0 \\ 0 & -2 \end{pmatrix}$ (diagonal!)

5. Matrix Powers and Polynomials

We can raise matrices to powers and evaluate polynomials at matrices. These operations are fundamental for solving matrix equations and understanding matrix functions.

Definition 5.1: Matrix Powers

For a square matrix $A$ and positive integer $k$ :

A^k = \underbrace{A \cdot A \cdots A}_{k \text{ times}}

We define $A^0 = I$ (identity matrix).

Theorem 5.1: Properties of Matrix Powers

For square matrices $A$ and integers $m, n \geq 0$ :

$A^m A^n = A^{m+n}$
$(A^m)^n = A^{mn}$
If $A$ is invertible, $(A^{-1})^k = (A^k)^{-1} = A^{-k}$

Definition 5.2: Matrix Polynomial

For a polynomial $p(x) = a_0 + a_1 x + \cdots + a_n x^n$ and a square matrix $A$ , define:

p(A) = a_0 I + a_1 A + a_2 A^2 + \cdots + a_n A^n

Example 5.1: Matrix Polynomial Example

If $p(x) = x^2 - 3x + 2$ and $A = \begin{pmatrix} 1 & 2 \\ 0 & 1 \end{pmatrix}$ , then:

p(A) = A^2 - 3A + 2I = \begin{pmatrix} 1 & 4 \\ 0 & 1 \end{pmatrix} - 3\begin{pmatrix} 1 & 2 \\ 0 & 1 \end{pmatrix} + 2\begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} = \begin{pmatrix} 0 & -2 \\ 0 & 0 \end{pmatrix}

Theorem 5.2: Polynomial Evaluation Preserves Operations

For polynomials $p$ and $q$ , and square matrix $A$ :

$(p + q)(A) = p(A) + q(A)$
$(pq)(A) = p(A) q(A)$
If $A = P^{-1} B P$ , then $p(A) = P^{-1} p(B) P$

Example 5.2: Computing High Powers

If $A = P^{-1} D P$ with $D$ diagonal, then:

A^k = (P^{-1} D P)^k = P^{-1} D^k P

Since $D^k$ is just the diagonal matrix with entries raised to the $k$ -th power, this makes computing $A^k$ much easier.

6. Block Matrices

Block matrices partition a matrix into submatrices (blocks). This perspective simplifies many computations and reveals structural properties.

Definition 6.1: Block Matrix

A block matrix (or partitioned matrix) is a matrix written as:

A = \begin{pmatrix} A_{11} & A_{12} & \cdots & A_{1n} \\ A_{21} & A_{22} & \cdots & A_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ A_{m1} & A_{m2} & \cdots & A_{mn} \end{pmatrix}

where each $A_{ij}$ is a submatrix (block) of appropriate size.

Theorem 6.1: Block Matrix Multiplication

If $A$ and $B$ are block matrices with compatible block sizes, then $AB$ can be computed block-wise:

(AB)_{ij} = \sum_k A_{ik} B_{kj}

provided the block dimensions are compatible for multiplication.

Example 6.1: Block Multiplication

For $A = \begin{pmatrix} A_1 & A_2 \\ A_3 & A_4 \end{pmatrix}$ and $B = \begin{pmatrix} B_1 & B_2 \\ B_3 & B_4 \end{pmatrix}$ :

AB = \begin{pmatrix} A_1 B_1 + A_2 B_3 & A_1 B_2 + A_2 B_4 \\ A_3 B_1 + A_4 B_3 & A_3 B_2 + A_4 B_4 \end{pmatrix}

Definition 6.2: Block Diagonal Matrix

A block diagonal matrix has the form:

\text{diag}(A_1, A_2, \ldots, A_k) = \begin{pmatrix} A_1 & 0 & \cdots & 0 \\ 0 & A_2 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & A_k \end{pmatrix}

where $A_i$ are square matrices and off-diagonal blocks are zero.

Theorem 6.2: Determinant of Block Diagonal Matrix

If $A = \text{diag}(A_1, \ldots, A_k)$ is block diagonal, then:

\det(A) = \prod_{i=1}^k \det(A_i)

Example 6.2: Block Upper Triangular

A block upper triangular matrix has the form:

\begin{pmatrix} A & B \\ 0 & C \end{pmatrix}

where $A$ and $C$ are square blocks and the lower-left block is zero.

Its determinant is $\det(A) \det(C)$ .

Remark 6.1: Applications of Block Matrices

Block matrices are useful for:

Partitioning large matrices for parallel computation
Understanding direct sum decompositions
Simplifying determinant and inverse computations
Analyzing structured matrices (e.g., block tridiagonal)

Frequently Asked Questions

Why does matrix multiplication work the way it does?

It's designed so that [S ∘ T] = [S][T]—the matrix of a composition is the product of matrices. The row-column dot product formula follows directly from how coordinates transform under composition of linear maps.

Why isn't matrix multiplication commutative?

Geometrically: rotate then reflect ≠ reflect then rotate. Algebraically: the composition of linear maps isn't commutative, and matrix multiplication represents composition. Even when both AB and BA are defined, they usually give different results.

What is the relationship between similar matrices?

Similar matrices represent the same linear operator in different bases. They share many properties: determinant, trace, eigenvalues, characteristic polynomial, and rank. If B = P⁻¹AP, then A and B are similar, connected by the change of basis matrix P.

Why are special matrices important?

Special matrices have structure that simplifies computations and reveals properties. Diagonal matrices are easy to invert and power, triangular matrices make solving systems efficient, and symmetric matrices have real eigenvalues with orthogonal eigenvectors.

What's the geometric meaning of an orthogonal matrix?

Orthogonal matrices represent isometries—transformations that preserve lengths and angles. In 2D/3D, these are rotations (det = 1) and reflections (det = -1). They preserve the dot product: ⟨Ax, Ay⟩ = ⟨x, y⟩.

Matrix Theory Practice

10

Questions

0

Correct

0%

Accuracy

1

If

A

is

3 \times 4

and

B

is

4 \times 2

, what size is

AB

?

Easy

Not attempted

2

Is matrix multiplication commutative?

Easy

Not attempted

3

(AB)^T = ?

Medium

Not attempted

4

If

A

is

m \times n

, what size is

A^T

?

Easy

Not attempted

5

A matrix

A

is symmetric if and only if:

Easy

Not attempted

6

If

A

is orthogonal, then

A^{-1} = ?

Easy

Not attempted

7

The product of two upper triangular matrices is:

Medium

Not attempted

8

If

T: \mathbb{R}^2 \to \mathbb{R}^2

is represented by

A

in the standard basis, what represents

T

in a new basis

B

?

Medium

Not attempted

9

What is the

(i,j)

entry of

AB

?

Medium

Not attempted

10

The matrix of the identity map

I: V \to V

in any basis is:

Easy

Not attempted