2 posts tagged “diffusion”
Three ways to make an LLM read its weights less often on a Mac — and why each one backfires
Single-stream decoding on Apple Silicon is bottlenecked by reading the model's weights out of memory, not by the math. Three techniques attack that directly — speculative decoding, diffusion generation, and self-speculative layer-skipping. I measured all three on a 16 GB M3. Each is right in theory and backfires in its own way: the bottleneck just moves one step further from the arithmetic.
I expected a diffusion LLM to be fast on my Mac. It tied the best model on quality instead — and lost on speed.
LLaDA2.0-mini, a diffusion language model, runs on a 16 GB M3 and ties Qwen3-4B for the best answer-quality score I've measured (20/21). But it's slower than the fastest autoregressive model and uses 4× the memory of the lightest — and the exact reason I expected it to be fast on bandwidth-bound hardware turned out to be why it isn't. A measured look at where the bottleneck actually moved.