Phase 5: The Frontier Era

Goal: Explore cutting-edge techniques.

Modules

Module	Status	Description
State Space Models	🔲	Mamba, S6
Mixture of Experts	🔲	Sparse routing
LoRA	🔲	Parameter-efficient fine-tuning
Quantization	🔲	INT8, FP8, PTQ, QAT
Speculative Decoding	🔲	Draft models, parallel generation
RLHF	🔲	Conceptual overview

🎯 Capstone

Apply LoRA to fine-tune a small transformer. Implement INT8 quantization and measure speedup.