I expected a diffusion LLM to be fast on my Mac. It tied the best model on quality instead — and lost on speed.

Jun 13, 2026

A diffusion language model running on a 16 GB M3 matched the best autoregressive model I have on answer quality — a genuine tie at the top — but it was slower than the fastest autoregressive model and used four times the memory of the lightest. I went in expecting the opposite: diffusion models generate many tokens per forward pass, and on a Mac, where the bottleneck is reading weights out of memory rather than the arithmetic itself, fewer-but-heavier passes should be the favorable trade. The quality result held up under every check I could throw at it. The speed intuition did not. Here’s the whole thing, including the part where I caught my own benchmark flattering the diffusion model and had to walk a headline back.

The model you’d want doesn’t fit

The diffusion LLM that made news was DiffusionGemma-26B-A4B, with day-zero MLX conversions. It doesn’t fit a 16 GB Mac at any quantization that exists — the 4-bit weights alone are 15.7 GB, before a token of context or the OS’s own footprint. Instant out-of-memory. The headline model is, for most Mac owners, unrunnable.

What fits is LLaDA2.0-mini — a 16-billion-parameter mixture-of-experts diffusion model (only a fraction of the experts fire per token), 9.2 GB at 4-bit, peaking at 8.6 GB under load. That’s the model this is about.

One rough edge before the first token

The official mlx-community LLaDA2 checkpoints fail to load through the standard path with 596 unmatched parameters. The cause is a convention seam: the checkpoint was converted with mlx-lm, which stamps the file format: mlx but leaves the weight keys in their original Hugging Face layout; mlx-vlm’s loader reads format: mlx as “already in my layout” and skips the key-remapping step the model actually needs. So the one model that needs the remap is the one that never gets it. I worked around it by forcing the remap — and I’ve since confirmed generation itself works fine through the normal API once the weights are in. This isn’t unique to LLaDA: while I was testing, someone independently hit the identical failure on a different diffusion checkpoint (a Nemotron model), so it’s a known issue affecting any checkpoint converted this way — not something specific to this model.

A note on process, since it’s the point of this whole exercise: my first draft of this post claimed two bugs — the second being that the diffusion generation path was dead code with no callers. That was wrong. I’d grepped too narrowly; the path is wired into both the high-level stream_generate and the server. I caught it before publishing by actually running the public API instead of trusting my own earlier read. One real rough edge, not two.

The quality result (this part held up)

I ran LLaDA2.0-mini through the same 21-task verifiable suite I use for every model: 8 coding tasks (the code is executed and its output checked), 8 math (the answer parsed and compared), 5 factual. Same checkers across models, so the totals are comparable. Every model is measured in its fair configuration — Qwen3-4B with thinking off, LLaDA and Llama as-is, and LFM2.5 with enough token budget to finish reasoning (more on that below).

20 out of 21 — a dead tie with Qwen3-4B, my best score on this suite. I didn’t take it on trust. The coding 8/8 is real: I read the executed standard output of every task — fizzbuzz printed the exact correct 15-element list, fib(10) returned 55, the flattener produced [1, 2, 3, 4, 5, 6, 7], the reverser turned “OpenSource” into “ecruoSnepO”. The math 8/8 I re-ran a second time to confirm determinism and watched the parsed answers land: compound interest 1610.51, 97 is prime, 42, series sum 5050. The single miss is almost funny — asked for “Apple’s first in-house Mac chip, 2020,” it answered “Apple1” (the 1976 hobby computer), the same class of miss Qwen3-4B made on that exact question.

¹ LFM2.5 has no thinking-off switch, and at the 512-token budget the others used, it spent the whole budget reasoning and never emitted code — a false 1/8 on coding that would have flattered everyone above it. I caught that (it’s the same artifact that once gave me a bogus Gemma-4 coding score), gave it room to finish at 2048 tokens, and its real score is 16/21. Worth stating plainly because the artifact pointed the wrong way: it made the diffusion model look better by comparison.

The speed result (this part didn’t go as I expected)

Here’s where the intuition broke — twice. My first measurement had the diffusion model as the fastest thing on the machine. That was an artifact of run order: I’d measured all the diffusion configs first, while the chip was cool, then the autoregressive models after it heated up, and a fanless M3 throttles hard as it warms. So I re-ran everything interleaved — diffusion and each competitor alternating back to back, flipping the leader each round, so both saw the same thermal conditions. That killed the diffusion “win.” But it also wasn’t the whole story, because I’d only measured one prompt. When I ran three prompts of different answer lengths, the speed didn’t just rank differently — it stopped being a single number at all.

The diffusion model’s throughput swung roughly 30× depending on the prompt — 0.6 tok/s on a short answer, 17.6 on a code answer that filled the canvas, 4.9 on long prose — while the autoregressive MoE (LFM2.5) sat at ~22 tok/s for all three. The reason is structural: LLaDA denoises a fixed-size canvas every generation, so a two-word answer costs nearly as much as a full one, and harder text needs more denoising passes to converge (the long-prose run took 211 forward passes for ~245 tokens, versus 58 passes for the 256-token code answer). It never beat the AR MoE on any prompt, beat dense Qwen3-4B only on the canvas-filling code case, and was an order of magnitude slower than everything on the short answer. So “competitive on speed” — which the single code prompt suggested — is wrong; the honest statement is diffusion’s speed is dominated by a fixed per-generation cost, so it loses across the board and loses worst when answers are short.

(I’m deliberately not leaning on absolute tokens/second. The same model and prompt swung from ~41 to ~15 tok/s between sessions purely on thermal state and background load — on a fanless chip the absolute number measures the room as much as the model. Every comparison above is interleaved under shared conditions, which is the only speed claim I trust.)

Why the obvious trade didn’t pay off

This is the part I find genuinely interesting, and the reason the result is worth a post even though my prediction was wrong.

What I measured, not reasoned: on the code answer, LLaDA2.0-mini produced 256 tokens in 58 forward passes — about 4.4 tokens per pass, where an autoregressive model runs one pass per token by definition. I counted this directly by instrumenting the forward call. So on its best prompt the diffusion model really does read the network far fewer times per token, which on a memory-bandwidth-bound machine sounds like exactly the right trade — each forward pass pays to haul weights out of unified memory, and diffusion makes fewer trips.

The trade has two halves, and the second one is why it loses. First, that 4.4-tokens-per-pass figure isn’t a constant: on long prose the same model took 211 passes for ~245 tokens — about 1.16 tokens per pass, barely better than autoregressive, because harder text needs far more denoising iterations to converge. The “fewer trips” advantage evaporates exactly when the text gets interesting. Second, every one of those passes processes the entire canvas of a 16B-parameter model at once, not one token’s worth of work — so each trip is far more expensive than a 1-billion-active-parameter AR MoE’s per-token pass. Diffusion swaps many cheap passes for few expensive ones, and on an M3 the expensive ones cost more than the trips they save.

The bottleneck-shifting story is still real and still the interesting part: diffusion genuinely moves work off the memory-bandwidth axis and onto the compute axis. But “moves work onto compute” only wins if compute is the idle resource and the saved memory traffic outweighs the added arithmetic — and here it doesn’t. I’ll be precise about what’s shown versus inferred: the forward-pass counts are measured; the “where the cost lands” reading is the standard model of on-device decode, consistent with the data but not isolated. The clean test would be to run the same comparison on a higher-bandwidth machine and watch whether diffusion gains ground as bandwidth stops being the wall. I have only the one 16 GB M3 — so the counts are fact, the mechanism is the best-supported reading, and the prediction it was meant to license, “diffusion will be fast on bandwidth-bound hardware,” is the thing the data refuted.

What it adds up to

Set against the model most people should run on a 16 GB Mac — Qwen3-4B, which ties this exact quality score at 2.2 GB — the diffusion model uses nearly 4× the memory (8.6 GB), runs slower than the fastest autoregressive option, and matches but doesn’t exceed the best quality. As a daily driver it’s dominated.

What changed isn’t the recommendation; it’s the premise. Diffusion language models were a research curiosity I assumed couldn’t run on a 16 GB Mac and wouldn’t be competitive if they did. One of them runs, and ties the best model I have on the single axis hardest to fake — answer quality. It’s not fast and it’s not lean, and the specific reason I expected it to be fast turned out to be the reason it isn’t. That’s a more useful result than the win I went looking for: on this hardware, generating tokens in parallel is a quality story, not yet a speed one — and the bottleneck just moved one more step away from the arithmetic.

I couldn’t find other published quality-and-speed numbers for LLaDA2 on a 16 GB M3, so if you’ve run it — especially a speed sweep across more than one prompt, or the same comparison on an M-series Max where bandwidth is less of a wall — I’d like to compare notes.