Claude designed protein binders and a wet lab confirmed them: anatomy of a verified agent loop
Around August 20 Anthropic published something most agent announcements never include: a wet-lab verification. Its Claude Science agent designed de novo protein binders against 14 of 15 clinically relevant targets, a list that includes PD-L1, TREM2, TNF-alpha and EGFR. Independent labs, Adaptyv Bio and Twist Bioscience, then synthesized and tested the designs. Of 1,320 designs, 354 were functional binders, a hit rate of 22-35% depending on the target, against the 10-15% Anthropic cites as typical for the field. In a head-to-head, the best Claude design bound at about 3.9 nM, versus about 45 nM for the winner of a public design competition in which 245 teams managed a 3.7% hit rate.
Why this is a different kind of result
The interesting part of this story for an engineer is not the binders. It is the loop. The pipeline has three stages: the agent proposes designs, an in-silico filter scores and prunes them, and a physical lab measures whether the survivors actually bind. Only the third stage counts. The 354 is not a model's opinion of its own work; it is a number produced by a third party with an assay. That is the definition of ground truth, and it is what separates this from most of the agent demos that reach enterprise decks. The AGI distance check argued that the gap to watch is between generation and verified outcome. This is a data point where the gap was measured, and the agent cleared a baseline set by humans.
An agent is only as trustworthy as the cheapest thing in its loop that can say no without asking the model.
Anatomy of a verified agent loop
- A generator with a wide search. 1,320 designs is a lot of candidates. The agent was allowed to explore, not asked for one answer.
- A cheap filter that is not the model. The in-silico stage prunes on physics, not on the model's confidence. It is the first place a bad design dies, and it costs nothing compared to a lab run.
- An expensive, independent verifier. Two external companies ran the assays. The agent could not influence the measurement, and the measurement is the only score that gets reported.
- A published baseline. The 10-15% typical hit rate and the 45 nM competition winner turn 'it worked' into 'it worked better than the field'. Without a baseline, a hit rate is just a number.
- A closed loop. Verified results feed the next round of design. That is what makes it an agent process rather than a one-shot generation.
What to steal for your own agents
- Find your assay. For a coding agent it is the test suite and the type checker; for a data agent it is a query against a known answer; for a document agent it is a schema validator. Whatever it is, it has to be something the model cannot talk its way past. Guardrails and output validation covers the mechanics.
- Put the cheap filter before the expensive one. Lint before you run the integration tests; validate the plan before you call the tool. The in-silico step is the reason 1,320 designs did not cost 1,320 lab runs.
- Report the verified rate, not the generated count. '354 functional of 1,320' is the honest metric. 'Generated 1,320 designs' is the vanity metric, and it is the one most agent products publish.
- Establish the human baseline first. If you do not know what your team's hit rate is on the same task, you cannot claim the agent improved it.
The honest limitation
Every number here comes from Anthropic's own post, and the brief I am working from flags exactly that: verify against primary sources before you repeat them. The 22-35% range hides which targets sat at 22 and which at 35, the 'typical 10-15%' baseline is Anthropic's framing of the field rather than a controlled comparison, and the head-to-head is one design against one competition winner. Adaptyv and Twist did the assays, which is the strongest part of the claim, but the design of the study, the selection of the 15 targets, and the choice of what to publish were all the vendor's. Read it as a well-instrumented case study, not as a benchmark.