Cerebras CS-4: 4,400 tokens per second per user, and how not to misread it

Cerebras announced the CS-4 on August 18. It is the first implementation of its Nexus architecture, with three WSE-3 Turbo wafers in a rack, and the headline figures are large: 750 PFLOPS of compute and 129.6 PB/s of aggregate memory bandwidth per rack. The number everyone quoted, though, is the small one. On GPT-OSS-120B Cerebras reports more than 4,400 tokens per second for a single user, which it says is up to 30x faster than the GPU configurations it tested. On models above 10 trillion parameters it claims more than 1,000 tokens per second, and 10x the throughput per watt of the CS-3.

Where the speed comes from

The detail that makes this interesting is what did not change. There is no new silicon generation. The WSE-3 Turbo is the WSE-3 with tighter power delivery: roughly twice the power into the wafer, which lets it run at higher clocks. Everything else in the decode path, the on-wafer SRAM and the fabric between cores, was already there. Cerebras got its record single-user decode latency by feeding the same chip more current.

Decode is not compute-bound. Generating one token means streaming the model's weights and the KV cache past the arithmetic units once; the arithmetic is trivial, the movement is not. On a GPU the weights live in HBM and the token rate is capped by HBM bandwidth divided by bytes per token. On a wafer the weights live in SRAM spread across the die, the bandwidth number has a P in front of it, and the cap moves to how fast you can clock the fabric, which is a power question.

Decode latency is a bandwidth and power problem wearing a FLOPS costume; the CS-4 just took the costume off.

Three ways to misread 4,400

The figure is tokens per second per user, on one model, with identical prompts. Each of those qualifiers matters.

• Per user is not aggregate. A rack serving one stream at 4,400 tokens per second and a rack serving 400 streams at 11 each have the same aggregate throughput and completely different economics. Cerebras is quoting the first. If your workload is batch summarisation of a million documents, cost per token at full batch is the number you need and it is not in the announcement.
• Decode is not prefill. 4,400 per second describes generation. Prefill, the pass over your input before the first token appears, is compute-heavy and benefits from FLOPS, not clock. Time to first token is a separate benchmark and it is not the one being advertised.
• 30x is against tested GPU configurations. Which GPUs, which serving stack, which batch size and which quantisation are not stated. A GPU serving a 120B model at batch 1 is a deliberately bad comparison for the GPU; nobody runs it that way. Even Hopper against Blackwell can swing decode rates several times depending on setup, as I went through in Hopper vs Blackwell.

Narrower and still impressive: for a single interactive stream on a 120B model, wafer-scale SRAM beats HBM by a wide margin, and that margin grew this generation through power, not transistors.

Where Apple unified memory sits in this picture

The same physics runs on a Mac, at a smaller scale. Unified memory means the whole model is one hop from the GPU, so a local decode rate is set by memory bandwidth divided by bytes per token, and the ceiling is fixed by the chip you bought. That is why quantisation matters so much locally: fewer bytes per token is more tokens per second on the same bandwidth, which is the framework I use in quality-adjusted token speed. The CS-4 is the same equation with a bandwidth figure several orders of magnitude higher and a power budget to match.

The honest gap

Every number here comes from Cerebras. The 4,400 tokens per second, the 30x and the 10x per watt are vendor-run on a vendor-chosen model with an undisclosed GPU baseline, and there is no cost per token, no price per rack and no independent measurement in the announcement. Until someone outside Cerebras runs the same model at the same batch size on a current Blackwell rack and publishes both prefill and decode, treat the ratio as a claim and the physics as the thing worth remembering.