The LLM coding benchmark

Methodology & caveats. Figures are indicative, compiled from public reports and vendor disclosures as of June 2026, and they move constantly — treat this as a shape-of-the-field snapshot, not a live leaderboard. SWE-bench Verified measures the share of real GitHub issues a model resolves with a passing patch; scores depend heavily on the agent scaffold around the model, so cross-vendor numbers are directional, not head-to-head. Prices are list API rates per million tokens and ignore caching, batching, and volume discounts — per-task cost depends on how many tokens your loop actually burns. Self-host models have no per-token price but a real hardware-and-ops cost (see running models locally). Claude Fable 5 is back on the board: it was withdrawn globally on 2026-06-12 under a US national-security order, but the US lifted the ban on 2026-06-27 and Anthropic is restoring access (vetted US orgs first, broader rollout in the coming weeks — the fallout). Its safeguards-lifted sibling Mythos 5 is listed too, but it stays a restricted rollout with no separate published score, so its SWE-bench shows n/a. The newly previewed GPT-5.6 family (Sol / Terra / Luna) is listed with OpenAI's announced prices but no SWE-bench number yet — OpenAI published only Terminal-Bench, so SWE-bench Verified shows n/a until an independent figure exists; context is inherited from the GPT-5.5 line pending confirmation. GLM-5.2's figure is an early, indicative estimate — Z.ai shipped it without published benchmarks, so treat it as provisional until independently verified. Earlier models' scores are reconstructed from public reports at their time and use representative pricing. The SWE-bench Pro column is the harder, independently-run multi-file variant; it is published for only a handful of frontier models, so most rows show n/a. Several recent rows and figures (prices, context, SWE-bench Verified) are sourced from vibecoding.cz. Always verify against primary sources before a buying decision: SWE-bench, Aider polyglot, LMArena, and each vendor's pricing page.