Perspective

How to think about approvals in an autonomous system

Full autonomy sounds appealing until an agent does something expensive or irreversible. Here's a framework for deciding what actually needs a human in the loop.

Jordan Kim

The pitch for agent autonomy is simple: let the agent handle everything end to end, without a human slowing it down. The reality, once agents are handling real customer data or real money, is more complicated. Full autonomy is fine right up until an agent does something expensive, irreversible, or reputationally damaging — and then the absence of any human checkpoint becomes the whole story, the thing everyone remembers about the incident.


Why "autonomous or not" is the wrong question

Most teams frame this as a binary choice: either the agent runs fully autonomously, or a human reviews every single action, which defeats much of the point of having an agent in the first place. Neither extreme is actually right for most real systems. The useful question isn't "autonomous or not" — it's which specific actions warrant a pause, and which don't.


Why the binary framing persists anyway

The binary framing is appealing because it's simple to communicate and simple to implement. "Fully autonomous" and "fully reviewed" are both single settings. A more nuanced, per-action policy requires actually thinking through which actions matter and why — which is more work up front, even though it produces a far better system in practice.


A framework for deciding what to gate

We've found three criteria that reliably separate actions worth gating from actions that aren't.


Cost

Does this action involve spending money above some meaningful threshold? A five-dollar refund and a five-thousand-dollar refund are not the same risk, even if they're the same type of action. Cost-based gating means the threshold does the work — small actions run freely, large ones pause for review.


Reversibility

Can this action be easily undone if it turns out to be wrong? Sending a draft for internal review is trivially reversible. Deleting a customer's data is not. The less reversible an action, the stronger the case for a human checkpoint before it executes, regardless of cost.


Blast radius

Does this affect one customer or many, one record or an entire dataset? An action that could go wrong for a single user is a very different risk than one that could go wrong for every user simultaneously. Bulk operations deserve more scrutiny than single-record operations, even when the per-unit action looks identical.


Applying the framework

Actions that score high on any of these three — a large refund, a bulk data deletion, a production deployment — are worth gating, even if they individually seem routine to a human doing the same task. Actions that score low on all three — drafting a response, looking up a record, summarizing a document — usually aren't worth slowing down for, because the cost of a mistake is low and easily corrected.

This produces a naturally uneven policy, and that's the point. Most of what an agent does should run without a pause. A small number of specific, well-chosen actions should always pause. Getting this split right is what makes an agent feel both fast and safe, rather than either reckless or bottlenecked.


What "gating" should actually look like in practice

A gate shouldn't feel like submitting a ticket into a queue that might get looked at eventually. It should feel like a fast, specific decision: here's exactly what the agent wants to do, here's why, approve or don't. The faster and more specific that review experience is, the less a gate feels like friction and the more it feels like exactly the safety net it's meant to be.


Routing matters as much as the gate itself

This also means gates need to route to the right person automatically — a refund approval shouldn't land in a generic queue that anyone might eventually check; it should go directly to whoever owns that decision, so the pause is measured in minutes, not days. A well-designed gate with poor routing is barely better than no gate at all, because the delay it introduces is the same either way.


The goal isn't more caution — it's better-placed caution

It's tempting to respond to any agent incident by adding more approval steps everywhere, as a blanket response to risk. This is usually the wrong instinct. Over-gating an agent doesn't make it safer in any meaningful sense — it just makes it slow, and slow-and-safe agents tend to get their approval requirements quietly relaxed over time by frustrated users, which puts you back where you started, minus the trust you had in the system.

The better instinct is to add approval steps exactly where the cost of a bad autonomous decision genuinely outweighs the cost of a brief pause for review — and to trust the agent everywhere else. That's a harder line to draw than "approve everything" or "approve nothing," but it's the only one that actually holds up once real usage starts.

Teal gradient background

TRUSTED BY 200+ AI TEAMS

Ship your agents to production.

Book a 30-minute demo, or start in the docs. No infrastructure required.

Teal gradient background

TRUSTED BY 200+ AI TEAMS

Ship your agents to production.

Book a 30-minute demo, or start in the docs. No infrastructure required.

Teal gradient background

TRUSTED BY 200+ AI TEAMS

Ship your agents to production.

Book a 30-minute demo, or start in the docs. No infrastructure required.

Create a free website with Framer, the website builder loved by startups, designers and agencies.