Flow Models

Change of Variables & Jacobian Determinants

The mathematical backbone of all flow models: the change of variables formula, Jacobian matrices, determinant computation tricks, and when exact likelihood is tractable.

Normalizing Flows: Foundations

Invertible transformations for density estimation: the change of variables formula, coupling layers, autoregressive flows, and the tradeoff between expressiveness and computational cost.

Coupling Flows: NICE, RealNVP & Glow

The coupling layer family: NICE's additive coupling, RealNVP's affine coupling, Glow's 1x1 convolutions, multi-scale architecture, and why these form the backbone of practical discrete normalizing flows.

Autoregressive Flows: MAF & IAF

Maximum expressiveness with triangular Jacobians: Masked Autoregressive Flow, Inverse Autoregressive Flow, neural spline flows, and the fundamental density-vs-sampling speed tradeoff.

Residual Flows & iResNet

Free-form flows via Lipschitz-constrained residual connections: invertible ResNets, spectral normalization for invertibility, unbiased log-likelihood estimation, and Russian roulette estimators.

Continuous Normalizing Flows & Flow Matching

From discrete layers to continuous dynamics: neural ODEs for generation, the instantaneous change of variables, flow matching training, rectified flows, and optimal transport connections.

Neural ODEs & FFJORD

Continuous-depth networks via neural ordinary differential equations: the adjoint method for memory-efficient training, FFJORD's trace estimator for free-form flows, and the connection between depth and continuous dynamics.

Flow Matching: Theory & Training

Simulation-free training of continuous normalizing flows: the flow matching objective, conditional flow matching, probability paths, Gaussian paths, and why flow matching dominates modern generative modeling.

Rectified Flows & Reflow

Straightening flow trajectories for one-step generation: the rectification procedure, reflow iterations, distillation to one step, consistency distillation connection, and achieving near-diffusion quality in a single forward pass.

Stochastic Interpolants & Generalized Paths

Unifying diffusion and flow matching through stochastic interpolants: general interpolation frameworks, time-dependent noise injection, bridge processes, and the connection between SDE and ODE generation.

Optimal Transport for Generative Models

The theory of optimal transport applied to flow models: Wasserstein distances, Monge and Kantorovich formulations, Brenier's theorem, Sinkhorn algorithm, and how OT coupling improves flow matching.

Flow Models for Image & Video Generation

Applying flow models to visual generation: Stable Diffusion 3, Flux, DALL-E 3 architecture choices, CFG with flows, text-to-image conditioning, video generation via temporal flows, and the flow vs diffusion debate.

Flow Models for Audio & Speech Synthesis

Applying normalizing flows and flow matching to audio generation: WaveGlow, Glow-TTS, VoiceFlow, Matcha-TTS, and the advantages of flows for real-time speech synthesis with exact likelihood.