Noise Schedules & Signal-to-Noise Ratio

Table of Contents

1. Learning Objectives
2. Notation
3. Core Intuition
4. Linear Schedule
5. Cosine Schedule
6. Signal-to-Noise Ratio Analysis
7. Continuous-Time Formulation
8. Learned and Adaptive Schedules
9. Schedule for Latent Diffusion
10. Common Pitfalls
11. Summary
12. Exercises

Learning Objectives

1. Derive the linear and cosine schedules and compute their SNR curves.
2. Explain why cosine schedule preserves information longer than linear.
3. Connect discrete schedules to continuous-time SDEs.
4. Analyze how schedule choice affects which timesteps the model prioritizes.
5. Explain schedule rescaling for latent diffusion models.

Notation

• $\beta_t$ — noise added at step $t$
• $\bar{\alpha}_t = \prod_{s=1}^t(1-\beta_s)$ — cumulative signal retention
• $\text{SNR}(t) = \bar{\alpha}_t/(1-\bar{\alpha}_t)$ — signal-to-noise ratio
• $\lambda_t = \log\text{SNR}(t)$ — log-SNR

Core Intuition

The noise schedule controls how quickly information is destroyed during the forward process. Too fast → model sees only noise at most timesteps (can't learn structure). Too slow → many timesteps are redundant (inefficient). The ideal schedule distributes "learning difficulty" evenly across timesteps.

Linear Schedule

DDPM (Ho et al., 2020): $\beta_t$ increases linearly from $\beta_1 = 10^{-4}$ to $\beta_T = 0.02$:

Cumulative:

Problem: Signal decays too quickly in early steps. By $t=T/2$, $\bar{\alpha}_t$ is already very small — most timesteps see nearly pure noise. The model spends most training time on "easy" pure-noise timesteps.

Cosine Schedule

Improved DDPM (Nichol & Dhariwal, 2021): Define $\bar{\alpha}_t$ directly via a cosine function:

Then $\beta_t = 1 - \bar{\alpha}_t/\bar{\alpha}_{t-1}$, clipped to $[0, 0.999]$.

Properties:

• Signal decays gradually (cosine-shaped).
• More timesteps with intermediate SNR (the "interesting" regime).
• Avoids the abrupt destruction of the linear schedule.

Signal-to-Noise Ratio Analysis

SNR interpretation:

• SNR $\gg 1$: signal dominates, image is clear.
• SNR $\approx 1$: equal parts signal and noise.
• SNR $\ll 1$: noise dominates, image is unrecognizable.

Uniform log-SNR schedule: Ideally, $\lambda_t$ should be approximately uniformly spaced in $[\lambda_T, \lambda_0]$, ensuring equal "difficulty" per timestep.

Linear schedule: Log-SNR drops quickly then plateaus → most steps are "easy" (pure noise).

Cosine schedule: Log-SNR decreases more uniformly → better distribution of difficulty.

Continuous-Time Formulation

In the continuous limit ($T \to \infty$), the forward process becomes an SDE:

where $\beta(t)$ is the continuous noise rate.

Variance-preserving (VP) SDE: $\bar{\alpha}(t) = e^{-\int_0^t\beta(s)ds}$.

Variance-exploding (VE) SDE: $d\mathbf{x} = \sqrt{\frac{d[\sigma^2(t)]}{dt}}\,d\mathbf{w}$ (noise increases without signal decay).

The choice of SDE determines the noise schedule and the structure of the generative process.

Learned and Adaptive Schedules

Learned schedule: Parameterize $\beta_t$ (or $\bar{\alpha}_t$) as a neural network output and optimize jointly with the diffusion model. Constraint: must be monotonically decreasing in SNR.

Resolution-dependent scheduling: Higher-resolution images need faster initial noise addition (large-scale structure is determined early) and slower fine-detail noise.

EDM schedule (Karras et al., 2022): Parameterize via noise level $\sigma(t)$ directly, with:

Train with log-uniform sampling of $\sigma$, which naturally weights all SNR levels equally.

Schedule for Latent Diffusion

Problem: Latent diffusion operates on VAE latents $\mathbf{z} \in \mathbb{R}^{h \times w \times c}$ with different statistics than pixel space.

Rescaling: Latents are typically zero-centered with standard deviation $\sigma_{\text{data}}$. The effective SNR must account for this:

Stable Diffusion: Uses a scaled linear schedule where $\beta$ values are adjusted for the latent space statistics. The "zero terminal SNR" condition ($\text{SNR}(T) = 0$) is enforced for proper generation.

Common Pitfalls

Pitfall 1. Using a linear schedule for high-resolution images. Signal is destroyed too quickly; the model never learns fine details.

Pitfall 2. Not ensuring terminal SNR ≈ 0. If $\text{SNR}(T) > 0$, the model cannot generate from pure noise — there's residual signal information that was never removed.

Pitfall 3. Changing the schedule without adjusting the loss weighting. The optimal loss weights $w(t)$ depend on the schedule; a mismatch leads to poor training.

Summary

• Linear: Simple, fast decay, biased toward high-noise timesteps.
• Cosine: Gradual decay, more uniform log-SNR distribution, better sample quality.
• SNR connects all schedules: ideal schedule distributes log-SNR uniformly.
• Continuous-time: VP-SDE and VE-SDE formulations unify discrete schedules.
• Latent diffusion: Requires schedule rescaling for VAE latent statistics.

Exercises

Exercise 1. Compute $\bar{\alpha}_t$ and SNR$(t)$ for the linear schedule at $t = 0, 250, 500, 750, 1000$.

Exercise 2. Derive $\beta_t$ from the cosine schedule definition and verify it's monotonically increasing.

Exercise 3. For the VP-SDE, show that $\bar{\alpha}(t) = e^{-\int_0^t\beta(s)ds}$ by solving the mean of the forward process.

Exercise 4. Design a noise schedule where log-SNR decreases linearly from $\lambda_0 = 10$ to $\lambda_T = -10$. Derive the corresponding $\bar{\alpha}_t$.

Exercise 5. Explain why "zero terminal SNR" ($\bar{\alpha}_T = 0$) is important for unconditional generation.