Model Training Methods

Pre-training Objectives: CLM, MLM & Beyond

The fundamental pre-training tasks: causal language modeling, masked language modeling, prefix LM, denoising objectives (T5), UL2, and how the pre-training objective determines downstream capability.

Curriculum Learning & Data Scheduling

Ordering training data for better learning: curriculum strategies, data mixing schedules, the effect of data order on convergence, anti-curriculum, and dynamic data selection.

Scaling Laws & Compute-Optimal Training

The science of scaling: Kaplan scaling laws, Chinchilla optimal ratios, compute-performance prediction, over-training for inference efficiency, and planning large-scale training runs.

Full Fine-tuning & Instruction Tuning

Adapting pre-trained models to downstream tasks: full parameter fine-tuning, instruction tuning (FLAN, Alpaca), supervised fine-tuning (SFT), learning rate schedules, catastrophic forgetting, and multi-task fine-tuning.

Adapter Methods & Parameter-Efficient Fine-Tuning

LoRA, QLoRA, AdaLoRA, prefix tuning, prompt tuning, (IA)3, and the theory of low-rank adaptation: why PEFT works, rank selection, and when to use which method.

RLHF: Reinforcement Learning from Human Feedback

The complete RLHF pipeline: reward model training from preferences, PPO optimization against the reward, KL regularization, practical challenges (reward hacking, mode collapse), and InstructGPT's three-stage approach.

DPO, KTO & Direct Alignment Methods

Aligning LLMs without RL: Direct Preference Optimization, Kahneman-Tversky Optimization, IPO, ORPO, SimPO — the mathematics of preference-based training without reward models or PPO.

Continual Learning & Catastrophic Forgetting

Training models on sequential tasks without forgetting: elastic weight consolidation (EWC), progressive neural networks, experience replay, PackNet, and modern continual pre-training strategies for LLMs.

Transfer Learning & Domain Adaptation

Leveraging pre-trained representations: feature extraction vs fine-tuning, domain shift theory, unsupervised domain adaptation, few-shot learning via transfer, and the foundation model paradigm.

Optimizers: Adam, AdamW, LION & Beyond

The mathematics of neural network optimization: Adam's moment estimation, weight decay vs L2 regularization, LION's sign-based updates, Sophia's second-order information, learning rate scheduling, and optimizer selection.

Training Stability: Loss Spikes, Divergence & Recovery

Diagnosing and fixing training instabilities: loss spikes, gradient explosions, learning rate warmup theory, Z-loss, QK-norm, $\mu$P parameterization, and recovery strategies for large-scale training.

Distributed Training: Data, Tensor & Pipeline Parallelism

Training across multiple GPUs and nodes: data parallelism, ZeRO stages, tensor parallelism for large layers, pipeline parallelism for long models, 3D parallelism configuration, and communication optimization.

Data Engineering for LLM Training

Building training datasets at scale: web crawling and filtering, deduplication (MinHash, exact), quality filtering (perplexity, classifiers), data mixing, tokenization choices, and the data quality vs quantity frontier.