When to Let AI Act Alone: What Anthropic Checks

The question that comes up most in AI projects isn't which model to use. It's how far to let the system act without anyone watching. In September, Anthropic published how it answers that within its own walls, with around thirty thousand agents running at once — and the answer isn't "review everything that matters." It's a simpler rule, one that fits any company.

The published numbers

The publication brings together three internal measurements from Anthropic, covering August 2026:

• How much of the work AI drives. Claude "leads" 26% of the company's AI research and development work. In February it was less than 1%. "Leading," on Anthropic's own scale, means completing most of the task from a general request, with a person supervising — not working unattended.
• How much gets monitored. On the most widely used internal platform, around thirty thousand agents work simultaneously. In August there were over a billion decisions, and every one passed through a monitor before being executed, usually within seconds. 0.002% of them were blocked — one in every 47,000.
• How much is spent on security. In a one-week survey in July, about 6% of research computing went to security work. In the portion of research conducted by AI itself, that share doubles to about 12%.

The third number is the one almost no one comments on: the more work AI does on its own, the larger the share of the budget that goes toward checking that work.

The rule behind it: speed and reversibility

Anthropic doesn't sort agent actions by importance. It sorts them by two questions, and each combination leads to a different type of review.

Is the damage fast and irreversible? Then review comes beforehand. That's the role of the real-time monitor, built to stop what happens in seconds and can't be undone — the example the company itself gives is an agent copying the model's files outside its systems. The action only runs after passing the filter, and people review everything that was blocked within a week.

Is the damage slow and reversible? Then review comes afterward. A second monitor reads through all activity already executed, looking for behavior that develops gradually. What it finds is used to adjust the filters, and only a serious, imminent problem would make the company shut down part of the system.

Notice what this solves: no one needs to approve a billion decisions. A person steps in where a mistake can't be fixed, and everything else gets audited.

The funnel that brings only the essentials to a person

The second monitor flags around a hundred thousand agent conversations a week. They pass through several automated classifiers, which discard false alarms and resolve most of them. About fifty a week reach a human.

This design matters more than the scale. A process that sends everything to human review isn't safer — it's a process where the person learns to click "approve" without reading. Safety comes from human review being rare enough to be taken seriously.

Translating this for a company without thirty thousand agents

The rule works with a single agent. List everything the system can do — reply, schedule, charge, publish, delete — and run each action through the two questions:

1. Fast and irreversible — approval beforehand. Paying, refunding, deleting customer data, sending a mass message, publishing under the brand's name, signing or accepting a business term. The agent prepares; a person releases it.
2. Fast, but reversible — the agent acts and reports. Rescheduling an appointment, changing an order's status, answering a question that has a ready-made answer. It needs a log and an easy way to undo it.
3. Slow and reversible — runs on its own, with sampling. Classifying support tickets, summarizing conversations, suggesting copy. Someone reads a sample every week, not every item.
4. Slow and irreversible — the trickiest case. Wrong information repeated for weeks until it becomes policy, or a discount the agent starts offering on its own. There's no single moment to block; it takes someone measuring the pattern, not the isolated action.

The fourth category is the one that usually ends up with no owner. And it's exactly the one the after-the-fact monitor was built to catch.

Three common mistakes when letting AI act

• Giving the agent the permissions of the person who configured it. If whoever set up the system can delete the customer database, so can the agent. Permissions should match the task, not the owner.
• Blocking without anyone reviewing the block. A filter that stops something and no one looks at it becomes a stalled process. Anthropic set a deadline for this: one week.
• Measuring only success. What needs measuring is how often the system was blocked, and why. That number says more about risk than any success rate — and when it drops to zero, that's not necessarily good news.

The case of two agents that start talking to each other, and agree on things neither should, is covered in the piece about what happens when you have more than one AI agent. And the case of the agent that pays on its own is in the piece about autonomous agents in commerce.

What Anthropic itself says these numbers don't show

The publication includes its own caveats, and they apply to any company that copies this model:

• It's hard to be certain the monitoring covers every possible type of behavior.
• The system has been running for months, not years.
• Computing is an imperfect gauge of security effort, because much of that work is people thinking, not machines running — and the estimates were deliberately made on the conservative side.

It's also a snapshot of those who build AI, not those who use it. But the two-question rule doesn't depend on scale: it depends on you knowing what each action in your system can and can't undo. For the other end of the comparison, in September OpenAI published how much work its agents really do, and what that costs per day.

How we do it in practice

In AI implementation, the list of actions comes before the agent. Each item gets marked with the same question — if it goes wrong, can it be undone? — and that's what determines what runs unattended, what waits for a person, and what gets checked at the end of the week. Changing this after the system is already serving customers costs far more than deciding it upfront.

Sources

Anthropic Institute, "Measurements for understanding the pace of AI development inside frontier labs", by Marina Favaro and Phillie Wright, with August 2026 data on research automation, agent oversight, and computing dedicated to security.