Matrix Operations & Linear Transformations

Table of Contents

1. Learning Objectives
2. Prerequisites
3. Notation
4. Core Intuition
5. Linear Transformations
6. Matrix Representation of Linear Maps
7. Matrix Multiplication as Composition
8. The Four Fundamental Subspaces
9. The Rank-Nullity Theorem
10. Determinants and Invertibility
11. Systems of Linear Equations
12. Transpose, Symmetry, and Orthogonality
13. Affine Transformations and Homogeneous Coordinates
14. Special Matrix Structures in Deep Learning
15. Attention as Composed Linear Maps
16. Common Pitfalls and Misconceptions
17. Research Perspective
18. Summary of Takeaways
19. Exercises

Learning Objectives

After reading this chapter, you should be able to:

1. Define linear transformations and prove basic properties (e.g., $T(\mathbf{0}) = \mathbf{0}$).
2. Construct the matrix representation of a linear map relative to the standard basis and prove uniqueness.
3. Derive matrix multiplication from composition of linear maps and interpret products via column, row, and outer-product expansions.
4. State and prove the rank-nullity theorem, and apply it to analyze information loss in neural network layers.
5. Define the determinant, prove its multiplicative property, and characterize invertibility.
6. Solve linear systems $\mathbf{A}\mathbf{x} = \mathbf{b}$ and derive the least-squares normal equations.
7. Distinguish linear from affine maps and explain how homogeneous coordinates restore linearity.
8. Decompose attention into a sequence of matrix operations.

Prerequisites

• Vectors, Spans & Linear Independence: vector spaces, span, linear independence, basis, dimension, inner products.

Notation

• $\mathbf{A} \in \mathbb{R}^{m \times n}$ — Matrix with $m$ rows and $n$ columns
• $\mathcal{C}(\mathbf{A}), \mathcal{N}(\mathbf{A})$ — Column space and null space
• $\text{rank}(\mathbf{A})$ — Column rank
• $\mathbf{T}: V \to W$ — Linear transformation between vector spaces
• $\mathbf{I}_n$ — $n \times n$ identity matrix
• $\mathbf{A}^{-1}, \mathbf{A}^T$ — Inverse and transpose

Core Intuition

A matrix is not merely a table of numbers — it is a concrete representation of a linear transformation. When a neural network layer computes $\mathbf{h} = \mathbf{W}\mathbf{x}$, the matrix $\mathbf{W}$ specifies how each input coordinate contributes to each output coordinate. Composing layers corresponds to multiplying matrices. Understanding which outputs are reachable (column space), which inputs are collapsed to zero (null space), and how much information is preserved (rank) is essential for analyzing model capacity, bottlenecks, and low-rank adaptation.

This chapter develops matrix algebra from the transformation viewpoint, proves the rank-nullity theorem in full, introduces determinants as volume-scaling factors, and connects the theory to attention mechanisms and affine neural network layers.

Series context. This is Part II of Chapter 1 (Linear Algebra). We build on Part I and prepare for Eigenvalues & Eigenvectors and the SVD.

Linear Transformations

Definition 1 (Linear Transformation). A function $T: V \to W$ between vector spaces over $\mathbb{F}$ is a linear transformation (linear map) if for all $\mathbf{u}, \mathbf{v} \in V$ and $\alpha, \beta \in \mathbb{F}$:

Equivalently, $T$ satisfies additivity $T(\mathbf{u}+\mathbf{v}) = T(\mathbf{u}) + T(\mathbf{v})$ and homogeneity $T(\alpha\mathbf{v}) = \alpha T(\mathbf{v})$.

Proposition 1. Every linear transformation maps the zero vector to the zero vector: $T(\mathbf{0}) = \mathbf{0}$.

Proof. $T(\mathbf{0}) = T(0 \cdot \mathbf{v}) = 0 \cdot T(\mathbf{v}) = \mathbf{0}$. $\blacksquare$

Corollary 1. An affine map $f(\mathbf{x}) = \mathbf{W}\mathbf{x} + \mathbf{b}$ with $\mathbf{b} \neq \mathbf{0}$ is not linear.

Proof. $f(\mathbf{0}) = \mathbf{b} \neq \mathbf{0}$, contradicting Proposition 1. $\blacksquare$

Neural network layers apply affine maps followed by nonlinear activations: $\mathbf{h} = \sigma(\mathbf{W}\mathbf{x} + \mathbf{b})$. The linear part is $\mathbf{W}$; the bias $\mathbf{b}$ shifts the origin.

Definition 2 (Kernel and Image). For $T: V \to W$:

Both are subspaces of $V$ and $W$ respectively.

Matrix Representation of Linear Maps

Theorem 1 (Matrix Representation). Every linear transformation $T: \mathbb{R}^n \to \mathbb{R}^m$ can be represented as multiplication by a unique matrix $\mathbf{A} \in \mathbb{R}^{m \times n}$:

Proof. Let $\{\mathbf{e}_1, \ldots, \mathbf{e}_n\}$ be the standard basis of $\mathbb{R}^n$. Any $\mathbf{x} = \sum_{j=1}^n x_j \mathbf{e}_j$. By linearity:

Define $\mathbf{A} = [T(\mathbf{e}_1) \mid T(\mathbf{e}_2) \mid \cdots \mid T(\mathbf{e}_n)] \in \mathbb{R}^{m \times n}$, so the $j$-th column is the image of the $j$-th basis vector. Then (4) becomes $T(\mathbf{x}) = \mathbf{A}\mathbf{x}$.

For uniqueness, suppose $\mathbf{A}\mathbf{x} = \mathbf{B}\mathbf{x}$ for all $\mathbf{x}$. Taking $\mathbf{x} = \mathbf{e}_j$ gives the $j$-th columns of $\mathbf{A}$ and $\mathbf{B}$ equal for all $j$, so $\mathbf{A} = \mathbf{B}$. $\blacksquare$

Important equation. Equation (3) identifies the matrix column $j$ with $T(\mathbf{e}_j)$. In a neural network, column $j$ of $\mathbf{W}$ tells us how input feature $j$ contributes to every output dimension.

Proposition 2. Under the identification $T \leftrightarrow \mathbf{A}$, $\text{ker}(T) = \text{null}(\mathbf{A})$ and $\text{im}(T) = \text{col}(\mathbf{A})$.

Matrix Multiplication as Composition

Let $T_1: \mathbb{R}^n \to \mathbb{R}^m$ with matrix $\mathbf{A} \in \mathbb{R}^{m \times n}$ and $T_2: \mathbb{R}^m \to \mathbb{R}^p$ with matrix $\mathbf{B} \in \mathbb{R}^{p \times m}$.

Definition 3 (Composition). The composition $T_2 \circ T_1: \mathbb{R}^n \to \mathbb{R}^p$ satisfies

Thus composition corresponds to matrix multiplication, defined by $(\mathbf{B}\mathbf{A})_{ij} = \sum_{k=1}^m B_{ik} A_{kj}$.

Theorem 2 (Four Interpretations of $\mathbf{C} = \mathbf{A}\mathbf{B}$). Let $\mathbf{A} \in \mathbb{R}^{m \times n}$, $\mathbf{B} \in \mathbb{R}^{n \times p}$, $\mathbf{C} = \mathbf{A}\mathbf{B} \in \mathbb{R}^{m \times p}$.

1. Entrywise (dot product): $C_{ij} = \mathbf{a}_{i*} \cdot \mathbf{b}_{*j}$ (row $i$ of $\mathbf{A}$ dotted with column $j$ of $\mathbf{B}$).
2. Column view: $\mathbf{c}_{*j} = \mathbf{A}\mathbf{b}_{*j} = \sum_{k=1}^n B_{kj}\, \mathbf{a}_{*k}$.
3. Row view: $\mathbf{c}_{i*} = \mathbf{a}_{i*} \mathbf{B} = \sum_{k=1}^n A_{ik}\, \mathbf{b}_{k*}$.
4. Outer-product (rank-1 sum): $\mathbf{C} = \sum_{k=1}^n \mathbf{a}_{*k}\, \mathbf{b}_{k*}^T$.

Proof of (4). $(\mathbf{a}_{*k}\mathbf{b}_{k*}^T)_{ij} = A_{ik} B_{kj}$, and summing over $k$ gives $C_{ij}$. $\blacksquare$

The outer-product view is foundational for low-rank matrix factorization and LoRA fine-tuning: a rank-$r$ update is a sum of $r$ rank-1 terms.

Proposition 3 (Non-commutativity). Matrix multiplication is associative: $(\mathbf{A}\mathbf{B})\mathbf{C} = \mathbf{A}(\mathbf{B}\mathbf{C})$, but not commutative in general.

Deep networks compute $\mathbf{W}_L \cdots \mathbf{W}_2 \mathbf{W}_1 \mathbf{x}$ — rightmost factor applied first.

The Four Fundamental Subspaces

For $\mathbf{A} \in \mathbb{R}^{m \times n}$, define:

Theorem 3 (Orthogonality Relations).

Proof of first relation. Let $\mathbf{x} \in \text{null}(\mathbf{A})$ and $\mathbf{r} \in \text{row}(\mathbf{A})$. Then $\mathbf{r}^T \mathbf{x} = 0$ because each row of $\mathbf{A}$ is orthogonal to $\mathbf{x}$ (since $\mathbf{A}\mathbf{x} = \mathbf{0}$). Any vector in $\text{row}(\mathbf{A})$ is a linear combination of rows, hence also orthogonal to $\mathbf{x}$. $\blacksquare$

Visual description. In $\mathbb{R}^n$, the row space and null space are complementary orthogonal subspaces: every vector decomposes uniquely into a component in $\text{row}(\mathbf{A})$ and a component in $\text{null}(\mathbf{A})$. The column space describes reachable outputs; the null space describes input directions "invisible" to the transformation.

Definition 4 (Rank). $\text{rank}(\mathbf{A}) = \dim(\text{col}(\mathbf{A})) = \dim(\text{row}(\mathbf{A}))$.

The Rank-Nullity Theorem

Theorem 4 (Rank-Nullity). For any $\mathbf{A} \in \mathbb{R}^{m \times n}$:

where $\text{nullity}(\mathbf{A}) = \dim(\text{null}(\mathbf{A}))$.

Proof. Let $r = \text{rank}(\mathbf{A})$ and $k = \text{nullity}(\mathbf{A})$. Choose a basis $\{\mathbf{u}_1, \ldots, \mathbf{u}_k\}$ for $\text{null}(\mathbf{A})$ and extend to a basis $\{\mathbf{u}_1, \ldots, \mathbf{u}_k, \mathbf{w}_1, \ldots, \mathbf{w}_r\}$ for $\mathbb{R}^n$ (possible since $\text{row}(\mathbf{A}) \oplus \text{null}(\mathbf{A}) = \mathbb{R}^n$ and $\dim(\text{row}(\mathbf{A})) = r$).

Claim: $\{\mathbf{A}\mathbf{w}_1, \ldots, \mathbf{A}\mathbf{w}_r\}$ is a basis for $\text{col}(\mathbf{A})$.

Spanning: Any $\mathbf{A}\mathbf{x}$ with $\mathbf{x} = \sum_i \alpha_i \mathbf{u}_i + \sum_j \beta_j \mathbf{w}_j$ equals $\sum_j \beta_j \mathbf{A}\mathbf{w}_j$ since $\mathbf{A}\mathbf{u}_i = \mathbf{0}$.

Independence: If $\sum_j \beta_j \mathbf{A}\mathbf{w}_j = \mathbf{0}$, then $\mathbf{A}(\sum_j \beta_j \mathbf{w}_j) = \mathbf{0}$, so $\sum_j \beta_j \mathbf{w}_j \in \text{null}(\mathbf{A})$. But $\sum_j \beta_j \mathbf{w}_j$ is also in $\text{span}(\mathbf{w}_j)$, which intersects $\text{null}(\mathbf{A})$ only at $\mathbf{0}$ (by construction of the extended basis). Hence all $\beta_j = 0$. $\blacksquare$

Important equation. Equation (8) quantifies the trade-off between information preserved (rank) and information destroyed (nullity). A layer with $\text{rank}(\mathbf{W}) < d_{\text{in}}$ has a nontrivial null space — distinct inputs can produce identical outputs.

Corollary 2. For $\mathbf{A} \in \mathbb{R}^{n \times n}$, $\mathbf{A}$ is invertible if and only if $\text{rank}(\mathbf{A}) = n$ if and only if $\text{null}(\mathbf{A}) = \{\mathbf{0}\}$.

Determinants and Invertibility

Definition 5 (Determinant — axiomatic). The determinant is the unique function $\det: \mathbb{R}^{n \times n} \to \mathbb{R}$ satisfying:

1. Multilinearity in rows (or columns).
2. Alternating: swapping two rows changes sign.
3. Normalization: $\det(\mathbf{I}) = 1$.

Theorem 5 (Multiplicativity). For $\mathbf{A}, \mathbf{B} \in \mathbb{R}^{n \times n}$:

Theorem 6. $\mathbf{A} \in \mathbb{R}^{n \times n}$ is invertible if and only if $\det(\mathbf{A}) \neq 0$.

Geometric interpretation. $|\det(\mathbf{A})|$ is the factor by which $\mathbf{A}$ scales $n$-dimensional volume. $\det(\mathbf{A}) = 0$ means the transformation collapses space to lower dimension.

Proposition 4 (Characteristic connection). $\det(\mathbf{A}) = \prod_{i=1}^n \lambda_i$ where $\lambda_i$ are eigenvalues (with multiplicity).

This links determinants to eigendecomposition, developed in Eigenvalues & Eigenvectors.

2×2 formula. For $\mathbf{A} = \begin{bmatrix} a & b \\ c & d \end{bmatrix}$:

Systems of Linear Equations

Problem. Given $\mathbf{A} \in \mathbb{R}^{m \times n}$ and $\mathbf{b} \in \mathbb{R}^m$, find $\mathbf{x} \in \mathbb{R}^n$ such that

Theorem 7 (Existence and Uniqueness).

• Existence: $\mathbf{A}\mathbf{x} = \mathbf{b}$ has a solution iff $\mathbf{b} \in \text{col}(\mathbf{A})$.
• Uniqueness: If a solution exists, it is unique iff $\text{null}(\mathbf{A}) = \{\mathbf{0}\}$.

Proof. Existence: $\mathbf{A}\mathbf{x} = \mathbf{b}$ means $\mathbf{b}$ is a linear combination of columns of $\mathbf{A}$. Uniqueness: if $\mathbf{x}_1, \mathbf{x}_2$ are solutions, then $\mathbf{A}(\mathbf{x}_1 - \mathbf{x}_2) = \mathbf{0}$, so $\mathbf{x}_1 - \mathbf{x}_2 \in \text{null}(\mathbf{A})$. $\blacksquare$

Least squares. When $\mathbf{b} \notin \text{col}(\mathbf{A})$ (overdetermined system), the least-squares solution minimizes $\|\mathbf{A}\mathbf{x} - \mathbf{b}\|^2$.

Theorem 8 (Normal Equations). If $\mathbf{A}^T\mathbf{A}$ is invertible, the unique least-squares solution satisfies

Derivation. Define $L(\mathbf{x}) = \|\mathbf{A}\mathbf{x} - \mathbf{b}\|^2 = (\mathbf{A}\mathbf{x} - \mathbf{b})^T(\mathbf{A}\mathbf{x} - \mathbf{b})$. Expanding:

Setting $\nabla_{\mathbf{x}} L = \mathbf{0}$:

This is the foundation of linear regression, treated in Volume II, Chapter 4.

Connection to independence. $\mathbf{A}^T\mathbf{A}$ is invertible iff columns of $\mathbf{A}$ are linearly independent (Proposition 5, Part I). Dependent columns make the normal equations singular.

Transpose, Symmetry, and Orthogonality

Definition 6 (Transpose). $(\mathbf{A}^T)_{ij} = A_{ji}$.

Proposition 5.

Proof of reversal. $((\mathbf{AB})^T)_{ij} = (\mathbf{AB})_{ji} = \sum_k A_{jk} B_{ki} = \sum_k (\mathbf{B}^T)_{ik} (\mathbf{A}^T)_{kj} = (\mathbf{B}^T \mathbf{A}^T)_{ij}$. $\blacksquare$

Definition 7. $\mathbf{A}$ is symmetric if $\mathbf{A} = \mathbf{A}^T$. It is orthogonal if $\mathbf{A}^T \mathbf{A} = \mathbf{I}$.

Orthogonal matrices preserve Euclidean norms: $\|\mathbf{Q}\mathbf{x}\| = \|\mathbf{x}\|$. They appear in SVD, QR decomposition, and orthogonal initialization.

Definition 8 (Gram matrix). $\mathbf{G} = \mathbf{A}^T \mathbf{A}$ encodes inner products between columns of $\mathbf{A}$: $G_{ij} = \langle \mathbf{a}_i, \mathbf{a}_j \rangle$.

Affine Transformations and Homogeneous Coordinates

An affine transformation is $f(\mathbf{x}) = \mathbf{A}\mathbf{x} + \mathbf{b}$.

Homogeneous coordinates. Augment $\mathbf{x}$ with a 1:

Then $\tilde{\mathbf{A}}\tilde{\mathbf{x}} = \begin{bmatrix} \mathbf{A}\mathbf{x} + \mathbf{b} \\ 1 \end{bmatrix}$, and composition of affine maps becomes matrix multiplication in the augmented space.

This trick allows us to treat biases as part of a larger linear system — useful in backpropagation (Chapter 8) and in projective geometry for computer vision.

Special Matrix Structures in Deep Learning

Diagonal matrices

$\mathbf{D} = \text{diag}(d_1, \ldots, d_n)$ scales coordinate $i$ by $d_i$. Layer normalization applies a learned diagonal scaling after centering.

Block-diagonal matrices

Multi-head attention uses block structure:

Each block operates on a head-specific subspace.

Low-rank matrices

$\mathbf{A} = \mathbf{U}\mathbf{V}^T$ with $\mathbf{U} \in \mathbb{R}^{m \times r}$, $\mathbf{V} \in \mathbb{R}^{n \times r}$, $r \ll \min(m,n)$. Storage $O(r(m+n))$ vs $O(mn)$. LoRA fine-tuning learns $\Delta \mathbf{W} = \mathbf{B}\mathbf{A}$ with small rank $r$.

Toeplitz and convolution

A 1D convolution corresponds to multiplication by a Toeplitz matrix — each row is a shifted copy of the filter. CNN theory (Chapter 9) builds on this equivalence.

Attention as Composed Linear Maps

Self-attention on input $\mathbf{X} \in \mathbb{R}^{T \times d}$ ( $T$ tokens, dimension $d$) proceeds as:

Step 1 — Linear projections:

where $\mathbf{W}_Q, \mathbf{W}_K, \mathbf{W}_V \in \mathbb{R}^{d \times d_k}$.

Step 2 — Similarity matrix:

Entry $S_{ij} = \langle \mathbf{q}_i, \mathbf{k}_j \rangle / \sqrt{d_k}$ is a scaled inner product (cosine similarity when vectors are normalized).

Step 3 — Softmax (row-wise, nonlinear):

Step 4 — Weighted combination of values:

Steps 1, 2, and 4 are purely linear (matrix multiplications). Softmax introduces nonlinearity. The overall map is not linear, but each linear sub-step inherits the subspace structure developed in this chapter: $\text{col}(\mathbf{V})$ determines reachable outputs; rank constraints on $\mathbf{W}_Q, \mathbf{W}_K$ limit expressivity.

Visual description. Think of $\mathbf{Q}\mathbf{K}^T$ as a $T \times T$ table of pairwise affinities between tokens. Softmax converts each row into a probability vector. Multiplying by $\mathbf{V}$ blends value vectors according to these probabilities — a data-dependent linear combination.

Common Pitfalls and Misconceptions

Pitfall 1: Treating matrix multiplication as element-wise multiplication. The Hadamard product $\mathbf{A} \odot \mathbf{B}$ is different from $\mathbf{A}\mathbf{B}$. Composition vs local gating are distinct operations.

Pitfall 2: Assuming $\mathbf{AB} = \mathbf{BA}$. Order of transformations matters. Reversing layer order generally changes the network function.

Pitfall 3: Confusing rank with number of nonzero entries. A sparse matrix can be full rank; a dense matrix can be rank-deficient.

Pitfall 4: Ignoring the null space in dimensionality reduction. Projecting onto $\text{col}(\mathbf{A})$ loses all information in $\text{null}(\mathbf{A})$. PCA (Chapter 1, Part V) chooses directions to preserve variance, not arbitrary projections.

Pitfall 5: Inverting nearly singular matrices. When $\det(\mathbf{A}) \approx 0$, $\mathbf{A}^{-1}$ is numerically unstable. The condition number $\kappa(\mathbf{A}) = \|\mathbf{A}\|\|\mathbf{A}^{-1}\|$ measures this sensitivity.

Research Perspective

The matrix formulation of linear algebra crystallized in the nineteenth century (Cayley, Sylvester). The four-subspace framework was popularized by Gilbert Strang and provides the language for modern numerical linear algebra.

In machine learning, matrix factorization underpins recommender systems (Koren et al., 2009), where low-rank structure in rating matrices is exploited via SVD-like methods. The rank-nullity theorem explains bottleneck layers in autoencoders: an encoder with rank $r$ can pass at most $r$ independent features to the decoder. Low-rank adaptation (LoRA, Hu et al., 2021) exploits the empirical observation that fine-tuning updates often have low intrinsic rank.

Attention's formulation as matrix multiplication (Vaswani et al., 2017) enabled GPU-optimized implementations and subsequent algorithmic improvements (FlashAttention, linear attention). The linear-algebraic decomposition in equations (18)–(21) remains the reference point for all variants.

Summary of Takeaways

• Linear map — $T(\alpha\mathbf{u}+\beta\mathbf{v}) = \alpha T(\mathbf{u}) + \beta T(\mathbf{v})$ — Layers without bias
• Matrix columns — $j$-th column = $T(\mathbf{e}_j)$ — Weight interpretation
• Composition — $(T_2 \circ T_1) \leftrightarrow \mathbf{B}\mathbf{A}$ — Deep network depth
• Column space — $\text{col}(\mathbf{A})$ — Reachable outputs
• Null space — $\text{null}(\mathbf{A})$ — Lost information
• Rank-nullity — $\text{rank} + \text{nullity} = n$ — Capacity vs compression
• Normal equations — $\mathbf{A}^T\mathbf{A}\hat{\mathbf{x}} = \mathbf{A}^T\mathbf{b}$ — Linear regression
• Attention — $\text{softmax}(\mathbf{QK}^T/\sqrt{d_k})\mathbf{V}$ — Transformer core

Next article: Eigenvalues & Eigenvectors → — intrinsic directions of linear transformations and spectral analysis of symmetric matrices.

Exercises

Exercise 1. Prove that $\text{rank}(\mathbf{A}) = \text{rank}(\mathbf{A}^T)$ using the rank-nullity theorem applied to $\mathbf{A}$ and $\mathbf{A}^T$ and the dimension identity $n = \dim(\text{row}) + \dim(\text{null})$.

Exercise 2. Let $\mathbf{A} \in \mathbb{R}^{m \times n}$ with linearly independent columns. Derive $\hat{\mathbf{x}} = (\mathbf{A}^T\mathbf{A})^{-1}\mathbf{A}^T\mathbf{b}$ from the normal equations (12) and interpret $\mathbf{P} = \mathbf{A}(\mathbf{A}^T\mathbf{A})^{-1}\mathbf{A}^T$ as an orthogonal projection onto $\text{col}(\mathbf{A})$.

Exercise 3. For $\mathbf{A} \in \mathbb{R}^{2 \times 2}$, prove $\det(\mathbf{A}) \neq 0$ iff columns of $\mathbf{A}$ are linearly independent.

Exercise 4. Show that orthogonal matrices satisfy $\|\mathbf{Q}\mathbf{x}\| = \|\mathbf{x}\|$ and that eigenvalues of $\mathbf{Q}$ have modulus 1.

Exercise 5 (Attention). For $T = 3$, $d_k = 2$, suppose $\mathbf{S}$ (before softmax) has rows with equal entries. Prove that $\mathbf{A} = \text{softmax}(\mathbf{S})$ has $A_{ij} = 1/3$ for all $i, j$, and hence the attention output for each token is the arithmetic mean of all value vectors.

Exercise 6 (Composition). A network has layers $\mathbf{W}_1 \in \mathbb{R}^{d_2 \times d_1}$, $\mathbf{W}_2 \in \mathbb{R}^{d_3 \times d_2}$. Express $\text{rank}(\mathbf{W}_2 \mathbf{W}_1)$ in terms of ranks of $\mathbf{W}_1, \mathbf{W}_2$. Give an example where $\text{rank}(\mathbf{W}_2 \mathbf{W}_1) < \min(\text{rank}(\mathbf{W}_1), \text{rank}(\mathbf{W}_2))$.

Exercise 7 (Affine maps). Prove that the composition of two affine maps $f(\mathbf{x}) = \mathbf{A}\mathbf{x} + \mathbf{a}$ and $g(\mathbf{x}) = \mathbf{B}\mathbf{x} + \mathbf{b}$ is affine, and compute its homogeneous matrix (16).