The applied chapter of a from-scratch project: after building tokenization, attention, gradient descent, a tiny GPT, and LoRA by hand, I ran a real QLoRA fine-tune — teaching Llama-3.2-1B to call tools on a MacBook, then measuring honestly what changed and what the adapter costs at inference. A 2.8M-parameter adapter (0.23% of the model) clearly helped on a small test; the debugging taught me the most.

Jul 7, 2026