CUA-Master scores 80.2 on OSWorld-Verified with 27B open weights. Is it time to retire RPA?

An open 27B model now clears 80 on OSWorld-Verified. CUA-Master, posted to arXiv on 2 November by a team the project page lists as MMLab at CUHK, Tsinghua, HKUST (Guangzhou), Explore Academy and JD, reports 80.2 on the benchmark buyers use as shorthand for "can an agent drive a desktop". The weights are on Hugging Face. Anyone running UiPath or Power Automate bots will ask whether this is the moment to start replacing them. My answer after reading the numbers: not for your stable flows, yes for the long tail you never automated.

How it got to 80.2

The interesting part is the data engine. The model card says CUA-Master-27B is built on Qwen-3.5-27B, so the gain comes from what the authors fed it. Their framework is a nested double loop. The outer loop is a world, data and model flywheel: watch where the current model fails, then aim the next round of data construction at those weaknesses. The inner loop seeds tasks by mining workflows from instructional videos on YouTube, TikTok and Bilibili, reconstructs the trajectories, then breaks them into reusable ability primitives that are recombined into new verifiable tasks, from single-app to cross-app.

The engineering detail I like most is the snapshot-rollback retry. When a trajectory fails halfway, the environment rolls back to the last valid VM snapshot instead of restarting from zero, so the valid prefix of a long task survives. The project page describes the scale behind it: 8 domains, 650+ applications, 100+ environments and 2,000+ concurrent VMs. Several rounds of SFT widen the abilities and a final RL stage pushes past the plateau. The training data itself is not released.

Read the whole scoreboard, not one number

Here is what the model card reports, with its own average time per task:

The drop from 80.2 to 36.9 tells me the verified set is close to saturated, and longer, harder tasks are where things still break. On Agents' Last Exam a full pass happens one time in four. The cross-platform results are mid-range: 56.2 on WindowsAgentArena, 43.0 on MacOSWorld and 59.9 on MacAgentBench. RPA estates are mostly Windows, so 56.2 is the figure I would quote to a client, not 80.2.

On the project page's own comparison table, CUA-Master still trails Claude-Opus-5 (86.3), Kimi-K3 (84.2), Qwen-CUA-397B-A17B (84.0) and GPT-5.6-Sol (83.6) on OSWorld-Verified, and beats UI-Mate-27B (77.6). The authors also claim 24 to 36 percent less runtime and fewer steps than comparable models. For a 27B model you can host yourself, that is a strong result. It is not the best agent on the board.

When it replaces RPA, and when it does not

RPA wins on stable, high-volume flows: the same SAP screen, the same invoice layout, ten thousand times a month. A script runs in seconds and fails the same way every time, which is what auditors want. An agent that needs eight minutes per task and gets one in five wrong on the easier benchmark is a bad trade there.

Computer-use agents will not replace your bots first. They will replace the backlog of processes nobody scripted because the screens kept changing.

The agent wins on the long tail: the hundred low-volume processes nobody built a bot for, the legacy app whose UI shifts every release, the task that needs reading an email before clicking. There, minutes per task and occasional failure beat a person doing it by hand, as long as someone reviews the result. I would deploy it with a disposable VM per task, a snapshot before every irreversible step, and a hard rule that it never holds production credentials. The model card itself warns that actions may modify files or run commands and should run in a sandbox. My notes on sandboxing coding agents apply almost unchanged to desktop agents.

Two caveats before anyone puts this in a proposal. The Hugging Face card does not state a license, so commercial use is unclear until the authors publish one. And every number above is self-reported on benchmarks with a fixed setup; your desktops have pop-ups, SSO prompts and VPN timeouts the benchmark never saw. The usual production failure modes do not disappear because the model got better at clicking.

What I would do this week: pick five processes from the RPA backlog that never got automated, run CUA-Master on them inside disposable VMs with snapshot rollback, and log success rate and minutes per task. If it clears 80 percent on your own tasks, you have a business case. If it does not, you have a cheap measurement instead of a vendor slide.