M6 and M5 Ultra: 1.2 TB/s and what actually fits in unified memory

Apple announced the M6 Mac mini and the M5 Ultra Mac Studio on August 25, with preorders the same day and shipping on September 22. The M5 Ultra is the one that matters for local inference: a quad-die package, up to 36 CPU cores and an 80-core GPU, and 1.2 TB/s of unified memory bandwidth, a 50% jump over M3 Ultra. Apple also claims up to 4.3x peak AI compute versus M3 Ultra. Pricing starts at $5,499; the 512 GB configurations arrive at the end of October. The M6 mini brings the first 2nm node, a 12-core CPU and dual Neural Engines from $899. Apple Newsroom, MacRumors and Forbes carried the details.

Bandwidth is the tokens-per-second ceiling

Local LLM decode is a memory-bound loop: every generated token streams the active weights through the memory system once. That makes tokens per second, to a first approximation, bandwidth divided by bytes per token. At 1.2 TB/s the ceiling for a 70B model at 4-bit, roughly 40 GB of weights, is around 30 tokens per second before any overhead; a 32B at 4-bit is nearer 65; a 7B is well into the hundreds and scheduling overhead dominates instead. M3 Ultra at the implied 800 GB/s sits at two thirds of those numbers. The 50% figure is the one that translates directly into speed. The 4.3x does not.

That is why I keep repeating the sizing rule from unified memory Mac sizing: buy the memory for the model you want to fit, then look at bandwidth for how fast it will run. They are separate axes, and the M5 Ultra moves both.

On unified memory, the spec that predicts your tokens per second is bandwidth; the AI compute multiplier predicts the keynote.

Which of the 4.3x survives production

Peak AI compute is a matmul throughput number, and matmul throughput is what prefill and training use. Decode barely touches it. So the 4.3x will show up in prompt processing on long contexts, in image and audio models, and in anything batched. For a single user chatting with a 70B model it will show up as almost nothing, because the bottleneck never left the memory bus. Where the number came from matters too: Apple's launch figures are Apple's, measured on preproduction hardware with workloads of Apple's choosing, and independent numbers do not exist until units ship on September 22. The fraction of 4.3x that is real for your workload is the fraction of your workload that is compute-bound, and for most local inference that is the prefill of the first prompt and not much else.

What fits, and at what speed

With 512 GB coming in October, the M5 Ultra fits things a single H100 cannot. Working from bandwidth and 4-bit weights, these are rough ceilings, not measurements:

• 70B dense at 4-bit: ~40 GB, ~30 tokens/s ceiling, comfortable on 128 GB.
• 120B-class dense at 4-bit: ~65 GB, ~18 tokens/s ceiling; wants 192 GB once KV cache is counted.
• 235B-class MoE at 4-bit: fits in 256 GB, decode governed by active parameters not total, so it can outrun the 120B dense.
• 400B-plus at 4-bit: the 512 GB configuration's reason to exist; single digits to low teens tokens per second.

Prefill on long contexts is where the MLX and Core ML stack and the new compute headroom earn their keep; decode is where the bandwidth does. Keep the KV cache math next to you, because at 128K context the cache eats the memory you thought was spare.

Local versus a shared H100

A $5,499 base Mac Studio, or considerably more with 256 GB, is a single-user machine with excellent capacity and modest throughput. A shared H100 is roughly the inverse: less memory, far more bandwidth and compute, and it serves a team. The break-even is concurrency. One developer running large models all day is cheaper on the Mac inside a year; five developers hammering the same model are cheaper on the shared GPU. Sizing that honestly is the whole decision.

The honest limitation

Every number above is either Apple's launch claim or my back-of-envelope from bandwidth. No independent benchmarks exist yet, MLX will need time to exploit the new silicon fully, and the 4.3x is exactly the kind of figure that shrinks when someone outside Cupertino measures it. I will update when I have a unit on the desk.