Volume VII
Large Language Models
Tokenization, pretraining, alignment, fine-tuning, inference, and retrieval-augmented generation.
Foundations
Emergent Abilities & In-Context Learning
Abilities that appear at scale: in-context learning, chain-of-thought reasoning, instruction following — theoretical explanations, phase transitions, and the debate on whether emergence is real or a measurement artifact.
Tokenization
Volume III, Chapter 11 — Part I. Subword tokenization theory: BPE algorithm, vocabulary construction, byte-level encoding, compression tradeoffs, and the mathematical structure of discrete-to-continuous mapping in LLMs.
Architecture Deep Dive
Context Length Extension
Extending transformer context beyond training length: RoPE scaling (NTK-aware, YaRN), position interpolation, sliding window + sink tokens, ring attention, and the theory of length generalization.
Rotary Position Embeddings (RoPE)
Volume III, Chapter 11 — Part II. Complete derivation of RoPE: rotation matrices, relative position in attention scores, frequency spectrum, length extrapolation, and NTK-aware scaling.
Pre-training
Pre-training Objectives & Data
Next-token prediction, masked language modeling, span corruption, UL2, data curation, deduplication, quality filtering, the Chinchilla scaling law, and compute-optimal training.
Scaling Laws for Language Models
Volume III, Chapter 12 — Part I. Power-law scaling of loss with compute, parameters, and data: Chinchilla optimal allocation, Kaplan vs. Hoffmann laws, and theoretical implications for LLM training.
Alignment & RLHF
RLHF: Reinforcement Learning from Human Feedback
The full RLHF pipeline: reward modeling from comparisons, PPO optimization, KL-constrained policy updates, the Bradley-Terry model, and InstructGPT methodology.
Direct Preference Optimization (DPO)
Volume III, Chapter 13 — Part I. Deriving DPO from the RLHF objective: Bradley-Terry preferences, closed-form optimal policy, implicit reward modeling, and the DPO loss without reinforcement learning.
Fine-Tuning Methods
LoRA: Low-Rank Adaptation
Volume III, Chapter 14 — Part I. Low-rank decomposition of weight updates: theoretical motivation from intrinsic dimension, LoRA formulation, merge at inference, and connection to the SVD.
Supervised Fine-Tuning & Instruction Tuning
Full fine-tuning vs parameter-efficient methods, instruction formatting, chat templates, data quality vs quantity, the role of SFT in the RLHF pipeline, and loss masking strategies.
Inference & Serving
KV Cache Explained
Volume III, Chapter 15 — Part I. Autoregressive inference optimization: prefill vs. decode phases, KV cache memory analysis, MQA/GQA, and complexity reduction from quadratic to linear per generated token.
Decoding Strategies & Sampling
Greedy, beam search, top-k, top-p (nucleus), temperature scaling, repetition penalty, contrastive decoding, and the theory of why sampling produces better text than search.