Chapter 7: Model Adaptation & Finetuning
+---------------------------------------------------------------------------------+
| THE ALIGNMENT PIPELINE |
+---------------------------------------------------------------------------------+
| Pre-training: Predict next token on 15T tokens ($10M+ compute) |
| | |
| v |
| Supervised Finetuning (SFT): Teach conversation format on 10k-100k pairs |
| | |
| v |
| Preference Alignment (DPO / RLHF): Steer model toward safety & high quality |
+---------------------------------------------------------------------------------+
Why Finetuning is an Efficiency & Specialization Lever
In Chapter 7, Chip emphasizes that finetuning is primarily an optimization of form, efficiency, and steerability:
- Prompt Shortening: Removing 15 few-shot examples from every prompt drops inference latency and token costs by 80%.
- Edge Deployment: Training a 3B or 8B model to match the accuracy of a 70B model on a single narrow enterprise task (e.g. generating dbt SQL).
Key Hyperparameters for SFT
| Hyperparameter | Typical Value | Practical Impact |
|---|---|---|
| Learning Rate | 1e-5 to 2e-4 |
Smaller than pre-training; too high causes catastrophic forgetting. |
| LoRA Rank ($r$) | 8 to 64 |
Higher rank = more capacity, but larger adapter size. |
| LoRA Alpha ($lpha$) | 16 to 64 |
Typically set to $1 imes r$ or $2 imes r$ to scale adapter weight updates. |
| Batch Size & Accumulation | 16 to 128 |
Use gradient accumulation steps to simulate large batches on small VRAM. |
| Epochs | 1 to 3 |
LLMs overfit rapidly on small instruction datasets; rarely train past 3 epochs. |
Find this content helpful?
☕ Buy me a coffee