Autonomy is earned: how to give an AI agent more control, safely.
The question we hear most from buyers this autumn is not “can an agent do this?” It is “how much should we let it do on its own?” The honest answer: less than the demo suggests at first, and more than your risk team expects later — if you earn it with numbers.
Shadow — the agent works alongside your team; nothing it produces is used. Suggest — staff see its draft and accept or change it. Approve — it acts, but a person approves each action. Exceptions only — it acts alone on routine cases and routes the rest to people.
Why skipping rungs fails
An agent that goes straight to acting alone fails in one of two ways. Either it makes a visible mistake in week two and loses the team’s trust for a year, or it is so fenced in that it never saves anyone time. The rungs exist so that trust is built on evidence rather than enthusiasm.
The numbers that unlock each step
Shadow to suggest when the agent’s output matches your team’s on a labelled set of your own cases, at a bar written down before the build starts.
Suggest to approve when staff accept its drafts unchanged most of the time, over several weeks. Acceptance rate is the most honest metric there is — people vote with their edits.
Approve to exceptions only for the categories where approvals are near-universal and a mistake is small and reversible. Everything else stays with people — permanently, if that is the right answer.
What has to be true at every rung
Every action logged with its evidence. A named person accountable. A way to stop it. And an evaluation that keeps running after go-live, because the data your agent sees in March is not the data it saw in your pilot.
How this shows up in our work
Our standard sprint puts an agent beside your team in 21 days — the shadow and suggest rungs — and into production by week twelve, with the success metric agreed in writing. If the metric is missed, we keep working at no extra fee for up to 60 days. Autonomy beyond that is a decision you make with the numbers in front of you, not one we make for you.
A worked example: prior authorisation
In weeks one to three, a prior-authorisation agent drafts packets that staff never send — it is scored against what staff actually submitted. From week four, staff see its packet first and edit it; the acceptance rate is tracked by payer and procedure. By week ten, for the three payers and five procedure codes where staff accept nearly every packet unchanged, the agent submits on its own and staff review a daily sample. Everything else still gets a human. Autonomy grew where the evidence was strongest, and nowhere else.
Common mistakes
Setting the bar after the fact. If the threshold for more autonomy is decided once results are in, it will be decided by mood. Write it down first.
Granting autonomy to the whole workflow at once. Promote categories, not agents. Some cases will never be suitable, and that is fine.
Forgetting to step back down. If acceptance rates fall — a payer changes its policy, a new document type appears — autonomy should shrink automatically until the agent catches up.
Every agent on our healthcare and legal pages shows exactly where a person stays in the loop — and what changes, in numbers, when the agent does the rest.
Frequently asked questions
What does human-in-the-loop mean for AI agents?
A person reviews, approves or can intervene in what the agent does. It ranges from approving every action to reviewing only exceptions and samples, depending on risk and evidence.
How long before an agent can act on its own?
It depends on the workflow. In our sprints an agent works beside the team from day 21; categories usually earn independent action over the following weeks, only where acceptance rates support it.
What is the best metric for trusting an agent?
The rate at which experienced staff accept its output unchanged, measured on real work over time. It reflects judgment that a test set cannot fully capture.