DFlash-MLX on Apple Silicon: where speculative decoding gives 3.5x and where context kills it

Around August 19 an Apache 2.0 project called dflash-mlx passed 760 GitHub stars with lossless speculative decoding using DFlash, a block-diffusion drafter. A roughly 1B drafter proposes 16 tokens, the target model verifies them in a single forward pass, and only the greedy-argmax tokens are emitted, so the output is identical to running the target alone. The numbers going around are real. The framing, 4.6x everywhere, is not. Here is where the speedup lives and where context length kills it.

The published numbers

All measured on an M5 Max with 64 GB, MLX 0.31.1.

The pattern is obvious. The speedup is largest on small dense models with short contexts, shrinks as the model grows, shrinks again with quantization, and collapses as context grows. The peak 4.37x on the 9B model happens at 1024 tokens of context. The same drafter on the 27B model at 8192 tokens gives 1.34x, which is barely worth the extra memory.

Why context kills speculation

Speculative decoding wins when verification is cheap relative to generation. On Apple Silicon decode is memory-bandwidth-bound: each token requires streaming the full weight set through unified memory, so verifying 16 tokens in one pass costs roughly one weight read instead of sixteen. That is the 3-4x on the small models.

Two things erode it. As the model gets bigger and more quantized, the target's per-token cost is dominated more by dequantization and attention than by weight streaming, so the one-pass-for-sixteen saving shrinks. And as context grows, the KV cache read per token grows with it, for the drafter as well as the target. At 8192 tokens the attention over the cache is a bigger share of each step than the weight read, and speculation cannot amortize it. This is the break-even curve from speculative decoding break-even, now with MLX-native points to plot on it.

Speculative decoding buys back weight bandwidth, and once your KV cache is bigger than your weights there is nothing left to buy.

The MoE case is different again. Qwen 35B-A3B already decodes at 141 tok/s because only about 3B parameters are active per token. The target is already cheap, so the drafter has less to save, and the result is around 1.8x.

What it is, and what it is not

• Apple Silicon and MLX only. There is no CUDA path.
• Narrow architecture support for now: Qwen3.5 and 3.6 with GatedDeltaNet, and Gemma 4.
• A different mechanism from the llama.cpp MTP head. DFlash uses a separate block-diffusion drafter rather than an extra prediction head inside the model, so acceptance rates are not comparable.
• Lossless by construction. Greedy-argmax verification means the tokens are the target's tokens. The quality-adjusted token speed is just the raw speed here, which is rare.
• An OpenAI-compatible server on port 8000, so it drops into an agent loop that already talks to a local endpoint.

How to measure it on your Mac

The viral numbers are single runs at chosen context lengths. To know what you will get, fix the prompt, run the median of three with a cooldown between runs, and sweep context at 512, 2048, 8192 and whatever your agent actually carries. Record prefill and decode separately, because DFlash touches only decode and a long-prompt workload is mostly prefill. Then do the same sweep against the Metal versus MLX baseline you already trust.

The honest gap

These are the project's own benchmarks on one machine, an M5 Max. I have not reproduced them yet, and the 8192-token result is the only long-context data point published, so the curve between 1024 and 8192 is interpolation. The DFlash paper, arXiv 2602.06036, is where to check the acceptance-rate methodology; the 87.7% figure is one model at one context and should not be assumed for anything else.