Over-proactivity is the new hallucination: HALO and the November 2 agent papers

Twenty experts read 8,000 responses from ten models acting as proactive assistants, deciding for each one whether a human would accept it. When a response was rejected, the most common reason was not a wrong fact. In 61.2% of rejections the agent stepped in too early or did too much. That is the central result of HALO, one of the agent papers on Hugging Face's November 2 daily list, and it reframes the rest of that list for me. Most of this week's papers make agents better at acting. HALO measures what happens when they act and should not have.

HALO: measuring restraint

HALO-Arena drops the agent into 50 personal environments across 10 everyday domains and gives it four kinds of proactive need, named Dependency, Blockage, Risk and Opportunity. Before it decides whether to help at all and how far to go, it has to dig the evidence out of structured databases or unstructured documents. The project repository lists four possible responses: observe, notify, prepare or execute an action. Picture a flight change in your inbox that collides with a meeting. Observing notes it, notifying tells you, preparing drafts the reschedule, and executing sends it to everyone before you have even seen the flight email. That ladder is exactly what I want in a production assistant, because each rung up costs more to undo.

The dataset, HALO-8K, holds 8,000 responses from 10 models over 400 scenarios. Experts labelled the action level, whether the response was acceptable, and for rejections, what went wrong: timing, intensity or something else. The best proprietary model reached 72.88% acceptability and the best open model 56.63%. Even the best system gets roughly one in four proactive moves wrong, and most of those wrong moves are overreach.

Hallucination is a model asserting something it has no evidence for. Over-proactivity is an agent doing something it has no mandate for.

I think that framing will hold up. With chatbots the cost of a hallucination is a wrong sentence a human can ignore. With agents that hold tools, the expensive failure is an unasked email, a calendar change, a file rewritten "to help". Hallucination evals asked whether the content was true. Proactive agents need evals that ask whether the action was wanted, and HALO is the first benchmark I have seen that grades that with humans at the granularity of timing and intensity.

R²Orch: orchestrators that teach themselves

The most upvoted agent paper of the day, R²Orch from CUHK Shenzhen, removes the person who writes the training curriculum for an orchestrator. Today, when an orchestrator fails at a task type, someone writes a skill for it and the skill list only grows. R²Orch closes that loop itself: an inner self-play loop has a Proposer pick training tasks, an outer loop improves how the inner loop learns, and a mechanism called Skill Withdrawal deletes a skill once controlled self-play shows the orchestrator does just as well without it. Failures become skills, skills become parameters, and each withdrawal exposes the next failure. The payoff the authors report, in their own summary, is a 9B orchestrator deployed with zero skills that beats the strongest baseline on all twelve benchmarks, lifting the in-distribution average from 86.0 to 89.8 and AIME 2026 from 74.0 to 81.3. Those are author numbers I have not reproduced.

CEPO and GEPA-Curator: optimizing the prompts around agents

CEPO splits a multi-agent system's prompts into one shared coordinator prompt and a separate executor prompt per agent, then lets existing prompt optimizers search inside that structure. The authors report consistent gains for four optimizers across 11 benchmarks in four domains and better scaling with the number of agents, though the abstract gives no headline numbers. GEPA-Curator from UC Berkeley starts from a point I agree with completely: agent failures come from bad prompts, harness limits, random variance or noisy judges, and only the first is worth learning from. It filters out tasks likely to give unreliable signal and retrieves earlier successful trajectories as context for the optimizer, reporting gains of up to 20.7% over GEPA and 21.2% over ACE on GoBrowse, ALFWorld, AppWorld and tau-bench.

The gap between these papers

Put the four side by side and a pattern shows. R²Orch, CEPO and GEPA-Curator all optimize for task success. Their reward is "did the agent complete the thing". None of them rewards an agent for declining to act, waiting, or stopping at notify instead of execute. A self-improvement loop trained only on completion will learn to complete more, and HALO's data says completing more, sooner, is exactly how proactive agents get rejected. I expect the first self-improving orchestrators deployed into inboxes and calendars to be measurably more over-proactive than the hand-tuned ones they replace.

The limitation on my side: HALO's scenarios are personal assistant settings, and its preprint does not yet tell me how acceptability moves with reasoning effort or retrieval depth beyond the fact that the authors log both. Enterprise agents with explicit tickets have a clearer mandate. Still, most of the agents I see being built for internal tools are moving toward the proactive end, triggered by events rather than requests.

What I am changing this week

In my own agent evals I am adding negative cases: scenarios where the correct response is to observe or notify, scored as failures if the agent executes. I am logging the action level of every agent step next to its outcome, so I can see the rate of execute decisions over time, and I am gating execute behind a confidence check that must cite the evidence it retrieved. If you run any self-improvement or prompt optimization loop, put restraint into its reward before you let it near a real inbox.