Perplexity's Lily beats MLX by 1.35x by doing one thing: single-model Rust/Metal engines

On September 1 Perplexity shipped Hybrid Compute for its Mac app and, alongside it, open-sourced Lily: a Rust and Metal inference engine built to run one model, Qwen3.6-35B-A3B, on one hardware family, Apple Silicon. Perplexity claims 1.35x the throughput of MLX on that model. The claim is the vendor's, but the design decision behind it is the interesting part. MLX is a general framework that has to run everything. Lily runs one thing, and everything a general framework leaves on the table for generality is headroom a single-model engine can spend.

What shipped

Hybrid Compute splits each task between frontier models in the cloud and a local model on the Mac. Private files, PII and on-device actions stay local; the rest goes up. The local side launches with Gemma 4 E4B, Qwen3.6 35B-A3B and a Perplexity model, with PPLX Qwen 3.8 27B available in one click. It runs on any Apple Silicon Mac with macOS 15 or later and 24 GB or more of unified memory, and tokens generated locally are not billed. That last line is the business model: local speed is product margin.

Lily is what makes that floor fast. Rust for the host side, hand-written Metal for the kernels, and no attempt to be portable across models or chips.

Why one model on one chip is faster

A general framework like MLX pays for flexibility in a dozen small places. Kernels are written for a family of shapes rather than the exact shapes of one model. Memory layouts are chosen to work for dense and MoE alike. Dispatch has to handle any graph, so it cannot fuse the specific sequence of operations one architecture always runs. Each cost is small; together they are the 35% Perplexity is claiming back.

A single-model engine can hardcode the expert routing for a 35B-A3B MoE, pick tile sizes for exactly the matrix shapes in that model, fuse the attention and normalization steps that always occur together, and lay out weights in whatever order the Metal kernels read fastest. None of that is novel. It is what every game engine does, applied to a transformer.

A general framework is a bet that your workload will change, and a single-model engine is a bet that it will not; both bets are correct for different people.

The trend beyond MLX

The September 12 radar listed three more native-Metal runtimes, all trend rather than shipped product.

• BaseRT: C++, tiled GEMM on simdgroup matrix intrinsics, with a C API meant to be called from Python, Node, Rust and Swift.
• Lattice: pure Rust plus NEON, so the CPU side is first-class rather than a fallback.
• PMetal: Rust, FlashAttention with linear memory in sequence length, and fused LoRA so adapters do not cost a second pass.

Each answers the same question: what do you get if you skip MLX's abstraction layer and write to Metal directly. The trade is obvious. You get speed on the shapes you optimized for and nothing on the shapes you did not. For a product on a fixed model that is a fine trade. For someone changing models monthly it is a maintenance sink, and MLX with CoreML alongside remains the sane default.

How to test the 1.35x claim

The claim is throughput, and throughput on Apple Silicon is memory bandwidth, so a fair reproduction needs the same model, quantization, context and machine for both engines. Measure decode tok/s and time to first token separately; a fused prefill path can win TTFT while losing steady-state decode, or the reverse. Run the median of three with a cooldown between runs. Then compare against the numbers in Metal versus MLX for local models. If Lily's advantage survives at 8K and 32K context, the specialization thesis is real. If it shows up only at short context, it is a prefill trick.

The honest gap

The 1.35x figure is Perplexity's own, on their choice of model, quantization and context, and I have not reproduced it. Lily runs one model, so it cannot be compared to MLX on anything else, and BaseRT, Lattice and PMetal are trend signals from a radar entry, not projects I have benchmarked. The architectural argument for specialization is sound. Whether these particular engines cash it in is something to measure, not assume.