Table of Contents
Learning Objectives Prerequisites Notation Core Intuition The Reverse Process Tractable Posterior The Variational Lower Bound KL Between Gaussians and Mean Matching The ε-Prediction Parameterization The Simplified DDPM Loss Connection to Score Matching Alternative Parameterizations Sampling the Reverse Process Worked Examples Connection to the Broader Curriculum Common Pitfalls and Misconceptions Research Perspective Summary of Takeaways Exercises
Learning Objectives
After reading this chapter, you should be able to:
Define the learned reverse process p θ ( x t − 1 ∣ x t ) p_\theta(\mathbf{x}_{t-1} \mid \mathbf{x}_t) p θ ( x t − 1 ∣ x t ) q ( x t − 1 ∣ x t ) q(\mathbf{x}_{t-1} \mid \mathbf{x}_t) q ( x t − 1 ∣ x t )
Derive the tractable posterior q ( x t − 1 ∣ x t , x 0 ) q(\mathbf{x}_{t-1} \mid \mathbf{x}_t, \mathbf{x}_0) q ( x t − 1 ∣ x t , x 0 ) μ ~ t \tilde{\boldsymbol{\mu}}_t μ ~ t β ~ t \tilde{\beta}_t β ~ t
Decompose the variational lower bound (VLB) into KL terms L T , L t − 1 , L 0 L_T, L_{t-1}, L_0 L T , L t − 1 , L 0
Show that training reduces to matching posterior means, then to ϵ \boldsymbol{\epsilon} ϵ
Connect ϵ θ \boldsymbol{\epsilon}_\theta ϵ θ ∇ x t log q ( x t ) \nabla_{\mathbf{x}_t} \log q(\mathbf{x}_t) ∇ x t log q ( x t )
State the simplified DDPM loss and sampling update.
Prerequisites
Notation
p θ ( x t − 1 ∣ x t ) p_\theta(\mathbf{x}_{t-1} \mid \mathbf{x}_t) p θ ( x t − 1 ∣ x t ) ϵ θ ( x t , t ) \boldsymbol{\epsilon}_\theta(\mathbf{x}_t, t) ϵ θ ( x t , t ) x 0 , x T \mathbf{x}_0, \mathbf{x}_T x 0 , x T ∇ x t log q ( x t ) \nabla_{\mathbf{x}_t} \log q(\mathbf{x}_t) ∇ x t log q ( x t ) L s i m p l e \mathcal{L}_{\mathrm{simple}} L simple
Core Intuition
Forward Diffusion destroys structure by adding noise. Generation requires inverting this process: starting from x T ∼ N ( 0 , I ) \mathbf{x}_T \sim \mathcal{N}(\mathbf{0}, \mathbf{I}) x T ∼ N ( 0 , I ) x 0 \mathbf{x}_0 x 0
The true reverse q ( x t − 1 ∣ x t ) q(\mathbf{x}_{t-1} \mid \mathbf{x}_t) q ( x t − 1 ∣ x t ) x 0 \mathbf{x}_0 x 0 tractable Gaussian posterior q ( x t − 1 ∣ x t , x 0 ) q(\mathbf{x}_{t-1} \mid \mathbf{x}_t, \mathbf{x}_0) q ( x t − 1 ∣ x t , x 0 ) p θ p_\theta p θ
The remarkable result: the VLB reduces to predicting the noise ϵ \boldsymbol{\epsilon} ϵ
Series context. Volume III, Chapter 10, Part II.
Reverse Denoising Process
Explore: Reverse diffusion learns a score function pointing toward data. As t decreases, noisy samples follow these arrows to recover structured clusters.
The Reverse Process
Definition 1 (Learned Reverse).
p θ ( x t − 1 ∣ x t ) = N ( x t − 1 ; μ θ ( x t , t ) , σ t 2 I ) , (1) p_\theta(\mathbf{x}_{t-1} \mid \mathbf{x}_t) = \mathcal{N}(\mathbf{x}_{t-1}; \boldsymbol{\mu}_\theta(\mathbf{x}_t, t), \sigma_t^2 \mathbf{I}),
\tag{1} p θ ( x t − 1 ∣ x t ) = N ( x t − 1 ; μ θ ( x t , t ) , σ t 2 I ) , ( 1 ) where μ θ \boldsymbol{\mu}_\theta μ θ σ t 2 \sigma_t^2 σ t 2 β ~ t \tilde{\beta}_t β ~ t
Proposition 1 (Anderson, 1982). For infinitesimal forward steps, the reverse-time SDE is a diffusion with drift depending on the score ∇ x log q ( x t ) \nabla_{\mathbf{x}} \log q(\mathbf{x}_t) ∇ x log q ( x t )
Proposition 2 (Intractability). q ( x t − 1 ∣ x t ) = ∫ q ( x t − 1 , x 0 ∣ x t ) d x 0 q(\mathbf{x}_{t-1} \mid \mathbf{x}_t) = \int q(\mathbf{x}_{t-1}, \mathbf{x}_0 \mid \mathbf{x}_t)\, d\mathbf{x}_0 q ( x t − 1 ∣ x t ) = ∫ q ( x t − 1 , x 0 ∣ x t ) d x 0
Tractable Posterior
Theorem 1 (Posterior). By Bayes' rule on the Markov forward chain, the conditional q ( x t − 1 ∣ x t , x 0 ) q(\mathbf{x}_{t-1} \mid \mathbf{x}_t, \mathbf{x}_0) q ( x t − 1 ∣ x t , x 0 )
q ( x t − 1 ∣ x t , x 0 ) = N ( x t − 1 ; μ ~ t ( x t , x 0 ) , β ~ t I ) , (2) q(\mathbf{x}_{t-1} \mid \mathbf{x}_t, \mathbf{x}_0) = \mathcal{N}(\mathbf{x}_{t-1}; \tilde{\boldsymbol{\mu}}_t(\mathbf{x}_t, \mathbf{x}_0), \tilde{\beta}_t \mathbf{I}),
\tag{2} q ( x t − 1 ∣ x t , x 0 ) = N ( x t − 1 ; μ ~ t ( x t , x 0 ) , β ~ t I ) , ( 2 ) where
μ ~ t = α ˉ t − 1 β t 1 − α ˉ t x 0 + α t ( 1 − α ˉ t − 1 ) 1 − α ˉ t x t , (3) \tilde{\boldsymbol{\mu}}_t = \frac{\sqrt{\bar{\alpha}_{t-1}}\,\beta_t}{1-\bar{\alpha}_t}\,\mathbf{x}_0 + \frac{\sqrt{\alpha_t}(1-\bar{\alpha}_{t-1})}{1-\bar{\alpha}_t}\,\mathbf{x}_t,
\tag{3} μ ~ t = 1 − α ˉ t α ˉ t − 1 β t x 0 + 1 − α ˉ t α t ( 1 − α ˉ t − 1 ) x t , ( 3 ) β ~ t = 1 − α ˉ t − 1 1 − α ˉ t β t . (4) \tilde{\beta}_t = \frac{1-\bar{\alpha}_{t-1}}{1-\bar{\alpha}_t}\,\beta_t.
\tag{4} β ~ t = 1 − α ˉ t 1 − α ˉ t − 1 β t . ( 4 ) Proof sketch. Product of Gaussians q ( x t ∣ x t − 1 ) ⋅ q ( x t − 1 ∣ x 0 ) q(\mathbf{x}_t \mid \mathbf{x}_{t-1}) \cdot q(\mathbf{x}_{t-1} \mid \mathbf{x}_0) q ( x t ∣ x t − 1 ) ⋅ q ( x t − 1 ∣ x 0 ) x t − 1 \mathbf{x}_{t-1} x t − 1
1 β ~ t = α t β t + 1 1 − α ˉ t − 1 = 1 − α ˉ t β t ( 1 − α ˉ t − 1 ) . (5) \frac{1}{\tilde{\beta}_t} = \frac{\alpha_t}{\beta_t} + \frac{1}{1-\bar{\alpha}_{t-1}} = \frac{1-\bar{\alpha}_t}{\beta_t(1-\bar{\alpha}_{t-1})}.
\tag{5} β ~ t 1 = β t α t + 1 − α ˉ t − 1 1 = β t ( 1 − α ˉ t − 1 ) 1 − α ˉ t . ( 5 ) Mean from linear terms. ■ \blacksquare ■
The Variational Lower Bound
Theorem 2 (VLB).
log p θ ( x 0 ) ≥ E q [ log p θ ( x 0 : T ) q ( x 1 : T ∣ x 0 ) ] = − L VLB , (6) \log p_\theta(\mathbf{x}_0) \geq \mathbb{E}_{q}\left[\log \frac{p_\theta(\mathbf{x}_{0:T})}{q(\mathbf{x}_{1:T} \mid \mathbf{x}_0)}\right] = -\mathcal{L}_{\text{VLB}},
\tag{6} log p θ ( x 0 ) ≥ E q [ log q ( x 1 : T ∣ x 0 ) p θ ( x 0 : T ) ] = − L VLB , ( 6 ) L VLB = D KL ( q ( x T ∣ x 0 ) ∥ p ( x T ) ) + ∑ t = 2 T D KL ( q ( x t − 1 ∣ x t , x 0 ) ∥ p θ ( x t − 1 ∣ x t ) ) − E [ log p θ ( x 0 ∣ x 1 ) ] . (7) \mathcal{L}_{\text{VLB}} = D_{\text{KL}}(q(\mathbf{x}_T \mid \mathbf{x}_0) \| p(\mathbf{x}_T)) + \sum_{t=2}^{T} D_{\text{KL}}(q(\mathbf{x}_{t-1} \mid \mathbf{x}_t, \mathbf{x}_0) \| p_\theta(\mathbf{x}_{t-1} \mid \mathbf{x}_t)) - \mathbb{E}[\log p_\theta(\mathbf{x}_0 \mid \mathbf{x}_1)].
\tag{7} L VLB = D KL ( q ( x T ∣ x 0 ) ∥ p ( x T )) + t = 2 ∑ T D KL ( q ( x t − 1 ∣ x t , x 0 ) ∥ p θ ( x t − 1 ∣ x t )) − E [ log p θ ( x 0 ∣ x 1 )] . ( 7 ) Interpretation:
L T L_T L T q ( x T ) ≈ N ( 0 , I ) q(\mathbf{x}_T) \approx \mathcal{N}(\mathbf{0}, \mathbf{I}) q ( x T ) ≈ N ( 0 , I ) L t − 1 L_{t-1} L t − 1 L 0 L_0 L 0
KL Between Gaussians and Mean Matching
With σ t 2 = β ~ t \sigma_t^2 = \tilde{\beta}_t σ t 2 = β ~ t
Proposition 3.
D KL ( q ∥ p θ ) = 1 2 β ~ t ∥ μ ~ t ( x t , x 0 ) − μ θ ( x t , t ) ∥ 2 . (8) D_{\text{KL}}(q \| p_\theta) = \frac{1}{2\tilde{\beta}_t}\|\tilde{\boldsymbol{\mu}}_t(\mathbf{x}_t, \mathbf{x}_0) - \boldsymbol{\mu}_\theta(\mathbf{x}_t, t)\|^2.
\tag{8} D KL ( q ∥ p θ ) = 2 β ~ t 1 ∥ μ ~ t ( x t , x 0 ) − μ θ ( x t , t ) ∥ 2 . ( 8 ) Training = match posterior mean .
The ε-Prediction Parameterization
From Forward Diffusion :
x t = α ˉ t x 0 + 1 − α ˉ t ϵ . (9) \mathbf{x}_t = \sqrt{\bar{\alpha}_t}\,\mathbf{x}_0 + \sqrt{1-\bar{\alpha}_t}\,\boldsymbol{\epsilon}.
\tag{9} x t = α ˉ t x 0 + 1 − α ˉ t ϵ . ( 9 ) Solving for x 0 \mathbf{x}_0 x 0
μ ~ t = 1 α t ( x t − β t 1 − α ˉ t ϵ ) . (10) \tilde{\boldsymbol{\mu}}_t = \frac{1}{\sqrt{\alpha_t}}\left(\mathbf{x}_t - \frac{\beta_t}{\sqrt{1-\bar{\alpha}_t}}\,\boldsymbol{\epsilon}\right).
\tag{10} μ ~ t = α t 1 ( x t − 1 − α ˉ t β t ϵ ) . ( 10 ) Definition 2 (ϵ \boldsymbol{\epsilon} ϵ
μ θ ( x t , t ) = 1 α t ( x t − β t 1 − α ˉ t ϵ θ ( x t , t ) ) . (11) \boldsymbol{\mu}_\theta(\mathbf{x}_t, t) = \frac{1}{\sqrt{\alpha_t}}\left(\mathbf{x}_t - \frac{\beta_t}{\sqrt{1-\bar{\alpha}_t}}\,\boldsymbol{\epsilon}_\theta(\mathbf{x}_t, t)\right).
\tag{11} μ θ ( x t , t ) = α t 1 ( x t − 1 − α ˉ t β t ϵ θ ( x t , t ) ) . ( 11 ) Proposition 4.
L t − 1 ∝ ∥ ϵ − ϵ θ ( x t , t ) ∥ 2 . (12) L_{t-1} \propto \|\boldsymbol{\epsilon} - \boldsymbol{\epsilon}_\theta(\mathbf{x}_t, t)\|^2.
\tag{12} L t − 1 ∝ ∥ ϵ − ϵ θ ( x t , t ) ∥ 2 . ( 12 ) The Simplified DDPM Loss
Definition 3 (DDPM Simple Loss, Ho et al.).
L simple = E t , x 0 , ϵ [ ∥ ϵ − ϵ θ ( α ˉ t x 0 + 1 − α ˉ t ϵ , t ) ∥ 2 ] . (13) \mathcal{L}_{\text{simple}} = \mathbb{E}_{t, \mathbf{x}_0, \boldsymbol{\epsilon}}\left[\|\boldsymbol{\epsilon} - \boldsymbol{\epsilon}_\theta(\sqrt{\bar{\alpha}_t}\,\mathbf{x}_0 + \sqrt{1-\bar{\alpha}_t}\,\boldsymbol{\epsilon}, t)\|^2\right].
\tag{13} L simple = E t , x 0 , ϵ [ ∥ ϵ − ϵ θ ( α ˉ t x 0 + 1 − α ˉ t ϵ , t ) ∥ 2 ] . ( 13 ) Dropping time-dependent weights improves sample quality empirically.
Training algorithm:
Sample x 0 \mathbf{x}_0 x 0 t t t ϵ \boldsymbol{\epsilon} ϵ
Compute x t \mathbf{x}_t x t
Minimize ∥ ϵ − ϵ θ ( x t , t ) ∥ 2 \|\boldsymbol{\epsilon} - \boldsymbol{\epsilon}_\theta(\mathbf{x}_t, t)\|^2 ∥ ϵ − ϵ θ ( x t , t ) ∥ 2
Connection to Score Matching
Theorem 3 (Score–Noise Identity).
∇ x t log q ( x t ∣ x 0 ) = − ϵ 1 − α ˉ t . (14) \nabla_{\mathbf{x}_t} \log q(\mathbf{x}_t \mid \mathbf{x}_0) = -\frac{\boldsymbol{\epsilon}}{\sqrt{1-\bar{\alpha}_t}}.
\tag{14} ∇ x t log q ( x t ∣ x 0 ) = − 1 − α ˉ t ϵ . ( 14 ) Corollary 2. ϵ θ \boldsymbol{\epsilon}_\theta ϵ θ
ϵ θ ( x t , t ) = − 1 − α ˉ t s θ ( x t , t ) . (15) \boldsymbol{\epsilon}_\theta(\mathbf{x}_t, t) = -\sqrt{1-\bar{\alpha}_t}\,\mathbf{s}_\theta(\mathbf{x}_t, t).
\tag{15} ϵ θ ( x t , t ) = − 1 − α ˉ t s θ ( x t , t ) . ( 15 ) Connects DDPM to score-based models (Song & Ermon, 2019).
Alternative Parameterizations
ϵ \boldsymbol{\epsilon} ϵ x 0 \mathbf{x}_0 x 0 v \mathbf{v} v α ˉ t ϵ − 1 − α ˉ t x 0 \sqrt{\bar{\alpha}_t}\boldsymbol{\epsilon} - \sqrt{1-\bar{\alpha}_t}\mathbf{x}_0 α ˉ t ϵ − 1 − α ˉ t x 0
All related by linear transforms of ( x t , t ) (\mathbf{x}_t, t) ( x t , t )
Sampling the Reverse Process
Definition 4 (DDPM Sampling Step). Given x t \mathbf{x}_t x t ϵ θ \boldsymbol{\epsilon}_\theta ϵ θ
x t − 1 = 1 α t ( x t − β t 1 − α ˉ t ϵ θ ) + β ~ t z , z ∼ N ( 0 , I ) . (16) \mathbf{x}_{t-1} = \frac{1}{\sqrt{\alpha_t}}\left(\mathbf{x}_t - \frac{\beta_t}{\sqrt{1-\bar{\alpha}_t}}\,\boldsymbol{\epsilon}_\theta\right) + \sqrt{\tilde{\beta}_t}\,\mathbf{z}, \quad \mathbf{z} \sim \mathcal{N}(\mathbf{0}, \mathbf{I}).
\tag{16} x t − 1 = α t 1 ( x t − 1 − α ˉ t β t ϵ θ ) + β ~ t z , z ∼ N ( 0 , I ) . ( 16 ) Start from x T ∼ N ( 0 , I ) \mathbf{x}_T \sim \mathcal{N}(\mathbf{0}, \mathbf{I}) x T ∼ N ( 0 , I ) t = T , … , 1 t = T, \ldots, 1 t = T , … , 1
Worked Examples
Example 1: Posterior Mean at High Noise
When α ˉ t ≈ 0 \bar{\alpha}_t \approx 0 α ˉ t ≈ 0 μ ~ t ≈ x t / α t \tilde{\boldsymbol{\mu}}_t \approx \mathbf{x}_t / \sqrt{\alpha_t} μ ~ t ≈ x t / α t
Example 2: Score from Noise
If ϵ θ = ϵ \boldsymbol{\epsilon}_\theta = \boldsymbol{\epsilon} ϵ θ = ϵ q ( x t ∣ x 0 ) q(\mathbf{x}_t \mid \mathbf{x}_0) q ( x t ∣ x 0 )
Connection to the Broader Curriculum
Common Pitfalls and Misconceptions
Pitfall 1: Confusing q ( x t − 1 ∣ x t ) q(\mathbf{x}_{t-1} \mid \mathbf{x}_t) q ( x t − 1 ∣ x t ) q ( x t − 1 ∣ x t , x 0 ) q(\mathbf{x}_{t-1} \mid \mathbf{x}_t, \mathbf{x}_0) q ( x t − 1 ∣ x t , x 0 )
Pitfall 2: Using wrong variance σ t 2 \sigma_t^2 σ t 2
Pitfall 3: Assuming L simple \mathcal{L}_{\text{simple}} L simple
Research Perspective
DDPM (Ho et al., 2020). Score SDE (Song et al., 2021). DDIM deterministic sampling. Consistency models. Flow matching as alternative.
Summary of Takeaways
Posterior — Gaussian with μ ~ t \tilde{\boldsymbol{\mu}}_t μ ~ t β ~ t \tilde{\beta}_t β ~ t VLB — Sum of KL mean-matching termsTraining — — ϵ − ϵ θ — 2 \ — \boldsymbol{\epsilon} - \boldsymbol{\epsilon}_\theta\ — ^2 — ϵ − ϵ θ — 2 Score — ϵ θ ∝ − ∇ log q \boldsymbol{\epsilon}_\theta \propto -\nabla \log q ϵ θ ∝ − ∇ log q
Next: Classifier-Free Guidance
Exercises
Exercise 1. Derive (3)–(4) from product of Gaussians.
Exercise 2. Derive (10) from (3) and (9).
Exercise 3. Prove (14).
Exercise 4. Show (11) and (12) are equivalent objectives.
Exercise 5. Derive sampling step (16).
Exercise 6. Why is L T ≈ 0 L_T \approx 0 L T ≈ 0
Exercise 7. Compare ϵ \boldsymbol{\epsilon} ϵ x 0 \mathbf{x}_0 x 0
Exercise 8. Connect to Gradient Descent on (13).