MLX beats llama.cpp until about 40K tokens of context

Ollama v0.32.0 shipped on July 11 and it runs on MLX on Apple Silicon, not on the llama.cpp Metal backend it used for years. The switch itself happened back in v0.19, and Ollama's own number for the move was decode going from roughly 58 to 112 tok/s. So the practical question for anyone running models on a Mac is no longer "MLX or llama.cpp" in the abstract. It is: where exactly does MLX stop winning? The July benchmarks give a surprisingly precise answer, and it is a context length, not a model size.

The 2026 numbers

Independent comparisons published around July 19 line up on the same picture. Below 14B parameters MLX generates 20-87% faster than llama.cpp. At 14B and above the gap narrows to 10-20%, with unified memory doing most of the work. MLX-Swift specifically wins decode by 1.4-1.8x over llama.cpp on the Qwen and Gemma kernels. Those are wide ranges because they cover different quantisations and machines, but the sign never flips: on short and medium prompts, MLX is ahead.

Then the arXiv study 2511.05502 and several follow-up write-ups add the caveat that matters: the MLX advantage disappears somewhere above 40K tokens of context. Past that boundary llama.cpp holds its throughput and MLX does not. If your workload is chat with a few thousand tokens of history, you never see this. If it is a long-context RAG pass or an agent loop that has accumulated a fat transcript, you live at that boundary all day.

MLX wins the decode race and llama.cpp wins the endurance race, and most local pipelines quietly run both kinds of workload.

Why the boundary exists

I wrote about the kernel-level differences in Metal vs MLX, and the 40K cliff is consistent with that. MLX's decode advantage comes from lazy evaluation and unified-memory friendly kernels that shine when the KV cache is small relative to the weights. As context grows, attention over the cache dominates, and the more mature long-context paths in llama.cpp catch up and pass. Nobody has published a per-layer breakdown I would call definitive, so treat that as my reading of the numbers rather than a proven mechanism.

There is a second, less comfortable detail from the same week. An open llama.cpp issue reports a roughly 13% throughput regression on M3 Ultra between the March builds and current builds, with Metal kernel commits as the suspect. That is the kind of thing that silently eats your margin if you upgrade a serving box without re-measuring.

The stack I recommend for 2026

The consensus recommendation from the July write-ups matches what I run: Ollama for daily use, MLX when performance matters, llama.cpp when you need control or portability. In practice:

• Ollama (now on MLX) for the default path. Interactive agent mode arrived in v0.32.0, the MLX cache leak is fixed, and speculative decoding has been available on Apple Silicon since mlx-lm 0.21 in May. See Ollama in practice for the operational side.
• MLX or mlx-lm directly for perf-sensitive inference under 40K tokens. Short-prompt agent tools, code completion, anything where decode tok/s is the metric.
• llama.cpp for long context and for anything that must run off-Mac too. GGUF is the portable artifact; the same file runs on a Linux box with CUDA.
• Measure tok/s per commit on the machine you actually ship on. The M3 Ultra regression is the argument. Fix the prompt, fix the quant, take the median of three runs, and log it against the build hash.

Sizing still matters more than the runtime. If the model does not fit, no kernel saves you, and unified memory sizing covers that maths.

What these numbers do not tell you

Every figure above is a throughput number on somebody else's hardware and prompt. The 40K boundary comes from one arXiv study and a handful of community benchmarks, not from a controlled sweep across M-series generations, quantisations and context lengths, and I have not seen the same test run on an M5 with its Neural Accelerators. Ranges like 20-87% are honest about variance but useless for a capacity plan. Before you commit a client to one runtime, run their real prompts at their real context length on their real Mac. The boundary is real; its exact position on your machine is not something you can read off a blog, including this one.