Recursive Flint: 125 tok/s on an M4 Pro, measured where it matters least
On October 21, 2026, Recursive released Flint, its first model. It's a coding-agent MoE with 39.8B total parameters and about 2.5B active per token, licensed Apache 2.0, with BF16, NVFP4 and MLX INT4 weights on Hugging Face from day one. The headline: more than 125 tokens per second on an M4 Pro MacBook Pro.
That combination is new for laptops: a permissively licensed model that fits in 48 GB and claims 74.2% on SWE-bench Verified. Whether it changes daily coding depends on what the headline leaves out: prefill and long context.
What actually shipped
From the model card:
| Item | Detail |
|---|---|
| Size | 39.8B total and ~2.5B active, counting the output head. The active body alone is ~2.2B. "40B-A2.5B" is the rounded name. |
| Architecture | 49 layers, 2 dense and then 47 MoE. Each MoE layer has 256 routed experts, picks 6 per token, and adds an always-on shared expert. 35 layers use 513-token sliding-window attention and 14 use full attention with no positional encoding. GQA uses 32 query heads and 4 KV heads, with a head dimension of 128. |
| Context | The card documents 262,144 tokens, shared between prompt and output. Recursive's blog says "up to 512k" but recommends staying within 256K. |
| Weights | BF16 is ~80 GB and needs one B300 or H200 per replica, via an SGLang kit. There is no Transformers modeling code yet. MLX INT4 is ~22.6 GB (group-64 affine INT4, INT8 embeddings). NVFP4 GGUF is 22.9 GB, and Recursive recommends at least 48 GB of GPU or unified memory for it. |
| Local engine | rec, Apache 2.0. It requires an M4 Pro, M5 Pro or M5 Max with at least 48 GB, and launches the pi or OpenCode harnesses. |
| Hosted | A free OpenRouter endpoint (262K context, ~104 tok/s measured). |
Numbers that matter
Self-reported benchmarks (temperature 1.0, top-p 0.95, top-k 20; average pass@1 over ≥5 attempts):
| Benchmark | BF16 | NVFP4 |
|---|---|---|
| SWE-bench Verified (mini-swe-agent) | 74.2% | 74.1% |
| Terminal-Bench 2.1 (Terminus 2) | 54.9% | 53.7% |
| LiveCodeBench v6 | 80.3% | 80.8% |
| SWE-bench Pro v2 (Recursive-measured) | 89.7% | 88.5% |
The 4-bit build stays within ~1.2 points of BF16 on every row, which matters more than the absolute scores. MarkTechPost lists developer-reported comparisons: Qwen3.6-35B-A3B at 73.4% on SWE-bench Verified, and dense Qwen 3.8 at 73.0% on Terminal-Bench 2.1, well above Flint.
Memory footprint (illustrative):
| Format | Weights | + KV at 262K | Fits |
|---|---|---|---|
| INT4 / NVFP4 (official) | 22.6 to 22.9 GB | ~7.5 GB | 48 GB Mac, with room for an IDE |
| Q8 (my estimate, ~1 byte/param) | ~40 GB | ~7.5 GB | 64 GB+ machine, not 48 GB |
| BF16 | ~80 GB | ~7.5 GB | One datacenter GPU |
KV arithmetic, from the card's specs: each full-attention layer stores 4 heads × 128 dims × 2 (K and V) × 2 bytes (FP16, as in the llama-server example) = 2,048 bytes per token. 14 such layers give ~28 KB per token, and 262,144 tokens × 28 KB ≈ 7.5 GB. The 35 sliding-window layers add only ~37 MB, because each caps at 513 tokens. This hybrid design is why the full context fits on a laptop at all.
125 tok/s is a decode number, single stream, short context, on Recursive's own engine, with prefill excluded.
What 125 tok/s does and doesn't mean
The blog states "more than 125 tokens per second" on an M4 Pro "at short contexts". The rec repo's chart setup reads: M4 Pro (48 GB), 4-bit weights, 1K to 252K-token prompts, 1,024 output tokens, five runs, prefill excluded. So it's a decode number, single stream, short context, on Recursive's own engine. The 1.4 to 2.6× speedup over llama.cpp is measured on the same basis.
A sanity check on the ceiling: the M4 Pro has 273 GB/s of memory bandwidth. Each decode step reads ~2.5B active params × ~0.56 bytes ≈ 1.4 GB, so the theoretical limit is 273 / 1.4 ≈ 195 tok/s. 125 tok/s is ~64% of that. Plausible, and impressive.
Long context is where it drops. Each decode step also reads the whole full-attention KV cache. At 200K tokens that's 200,000 × 28 KB ≈ 5.7 GB per token on top of the weights, which caps decode near 273 / (1.4 + 5.7) ≈ 38 tok/s. That's my arithmetic, not a measurement. And prefill, the part where an agent swallows 50K tokens of repo before answering, isn't published at all. On Apple GPUs, prefill is compute-bound, and for agentic coding it often dominates wall-clock time.
Does it change local vs API?
Not on cost. An illustrative heavy coding day on DeepSeek's deepseek-flash API at off-peak rates: 4.5M cache-hit input × $0.003/M + 0.5M cache-miss × $0.15/M + 0.2M output × $0.60/M ≈ $0.01 + $0.08 + $0.12 = ~$0.21 a day. Electricity alone won't beat that by much, and the API model is far larger.
Flint wins elsewhere. First, code that can't leave the machine: it's Apache 2.0 with no usage thresholds, and runs offline. Second, latency on short turns (completions, small edits, test scaffolding), where 125 tok/s with no network round trip feels instant. Third, pinning: your model doesn't change under a stable API name.
My read: Flint makes a 48 GB Mac a credible first-line coding agent for bounded tasks. It doesn't replace a frontier model for long, multi-file agent runs, where Terminal-Bench 2.1 at 54.9% and long-context prefill both work against it.
What I'd do
Use the official INT4 or NVFP4 builds, not community quants; Recursive publishes the BF16-vs-4-bit delta only for its own quantization. Benchmark prefill yourself: time to first token on a real 30 to 60K-token agent prompt in your repo is the number to watch. Keep -c at what you actually use, because the full 262K context costs ~7.5 GB of KV and slows decode as it fills.
Route by task: Flint locally for short loops, an API model for long-horizon runs. See Mac sizing, and compare with Devstral if you already run a local agent. On 48 GB, don't bother with Q8. It won't fit alongside a 262K KV cache.
What I couldn't verify
All benchmarks are self-reported, and SWE-bench Pro v2 is Recursive's own variant that can't be compared with public SWE-bench Pro. Prefill throughput is not published anywhere I could find. On context, the card says 262K and the blog says 512K; I've used the card. The 150 tok/s RTX Spark figure and the "5× faster than dense 25 to 40B models" claim come from vendor marketing without a published method, so I left them out of the analysis. Ollama, LM Studio and Unsloth support is reported by MarkTechPost; I didn't verify the Ollama listing myself. For background, see Ollama's MLX move.
I'll revisit when independent prefill and long-context decode measurements on M4/M5 Pro appear.
Sources
Primary: RecursiveSI/flint model card (Hugging Face); RecursiveSI/flint-int4 (MLX) model card; RecursiveSI/flint-nvfp4-gguf model card; Recursive: Introducing Recursive Flint; recursive-oss/rec (GitHub); OpenRouter: Recursive Flint (free). Corroboration and context: MarkTechPost: Recursive Flint local coding model; MacRumors: Apple announces M4 Pro chip; DeepSeek API: Models & Pricing.