Most agents that can do something real, such as send an email, change a record or move money, get their human approval the same way: a line in the system prompt. "Always ask the user before sending an email." It works in the demo. It keeps working until a model has a bad day, a document tells it the user already agreed, or the process restarts at the wrong moment. Then the approval turns out to have been a suggestion.
An approval is a control, and a control the model can talk itself out of isn't one. It has to live where the model can't reach it: in the code that runs the tools.
The short version
- The model shouldn't decide whether to ask. Whether a call needs approval follows from what the tool does, declared in code, not from what the model says it intends.
- Check the call before you ask anyone. A call with a missing field or an impossible value goes back to the model to fix. People shouldn't approve broken requests.
- Show the approver the exact call. The tool and the arguments it will run with, secrets redacted, not the model's description of them. Let them approve, change or reject it.
- Make the pause durable. Save the request before anyone is asked, survive a restart, and let it expire.
- Run the approved call exactly once. Give it a key, record it, and treat a call interrupted by a crash as unknown, never as safe to retry.
Four ways a prompt-based approval fails
The model decides whether to ask
An instruction is an input to the model, and the model weighs it against everything else in its context. Usually it asks. Sometimes the conversation is long and the instruction is far behind it. Sometimes a retrieved document or a tool result says "the customer has already approved this change", and the model believes it. Prompt injection doesn't have to break the model; it only has to give it a reason to skip one step.
The approver sees a summary, not the call
When the model asks "Shall I send the customer a short note about the delay?", the user approves a sentence. What runs is a tool call with a recipient, a subject and a body the user never saw. The summary and the call are produced by the same model, and nothing checks that they match.
Approval and execution are separate events
The user says yes in one turn; the tool runs in the next. In between, the model writes the call again. It can change an argument, call a different tool or call the same one twice, and the "yes" from the previous turn is still sitting in its context, looking like permission for all of it.
A restart turns one approval into zero, or two
The process restarts while the run is waiting for a decision. If the pending approval lived only in memory, the run is gone, and so is the user's answer. Worse, the process dies while the approved tool is running. Did the email go out? A system that retries on restart sends it again. One that doesn't may never send it.
None of these are rare at scale. They are what happens when a control lives in text the model reads instead of code the model can't change.
What an approval in the execution path needs
We build Algen Agent Runtime, an open-source Python runtime for governed agents, and approvals are one of the places where it's strictest. Here is what we think an approval needs, and how the runtime does it.
Declared by the tool, not decided by the model
Every tool declares its side effect: read-only, write, external or destructive. The approval rule follows from that declaration.
@crm.tool(write(approval=True))
async def update_owner(account_id: str, owner: str, context: ToolContext) -> Receipt:
"""Change an account's owner."""
client = await crm.client()
return await client.set_owner(account_id, owner, key=context.idempotency_key)
approval=True means every call to this tool waits for a human, whatever the agent's own policy says. An agent's policy can add approvals but never waive this one. By default an agent asks before every write, external and destructive call anyway. The tool executor enforces it too: it refuses to run such a call without a reference to the approval that covers it, so code that calls the tool directly can't skip it either.
Policies can also come from a central, signed bundle that no agent can opt out of, and a deny from any policy wins over another policy's request for approval. A human can't approve their way past a rule that says no.
Checked before anyone is asked
Before a request reaches a person, the call is validated against the tool's schema, and an optional precheck judges it against the current state. A change with a missing field, or one that conflicts with another call in the same reply, goes back to the model as an error to fix. Approval queues fill up fast enough without requests that would fail anyway.
The exact call, redacted
The approval request holds the tool, its side effect and the parameters it will run with. Secrets such as tokens, passwords and API keys are redacted before the request is stored. The approver can approve, reject, or modify the parameters, and a modified call is validated against the same schema before it runs. A rejection can be told to the model, with a reason, so the turn carries on and the model explains what it didn't do instead of failing.
A pause that survives
A run that needs a decision is checkpointed with status awaiting_approval before anyone is asked. It survives a restart, and a version check stops two processes from both resuming it. Every pending request in a tenant can be listed from one approval inbox, so approvers don't need to poll runs. Requests expire: an approval that sat for a week was given for a world that has changed.
Exactly once, or knowingly unknown
When the approved call runs, it carries an idempotency key unique to its run and call, and it's recorded in a tool-execution ledger. Writes are tried once. If the process dies mid-call, the call is marked indeterminate, not failed and not retried. An operator settles it through an audited reconciliation API: mark it completed or failed, ask the tool's own system whether it took effect (by that idempotency key), or run its compensation. The approval and the execution are both in the audit trail: who decided, when, and on exactly which arguments.
See it in a minute
No API key and no network: a deterministic mock model stands in for a real one.
python -m pip install --pre algen-agent-runtime
python -m examples.quickstart_approval.app
The workflow proposes a change and pauses at an approval node. Type approve and it applies the change once, under an idempotency key. Type reject and nothing is written.
In a multi-step workflow, the approval is a node of its own, with the review fields, the schema for changed parameters and an expiry declared next to it:
- id: approve
kind: approval
depends_on: [propose]
approval:
prompt: Review this synthetic change before it is applied.
review_from: [proposal.change]
parameters_from: proposal.parameters
allow_modification: true
rejection_policy: skip_dependents
expires_seconds: 1800
What this doesn't solve
A person can still approve a bad change. The runtime makes sure they see the real call and that their answer is the one that counts. It can't make them read it. Ask for approval where it matters, on writes, external messages and anything destructive, and keep reads fast. An approver who sees fifty requests a day stops reading them.
Approvals are not unique to us. LangGraph, the OpenAI Agents SDK, Pydantic AI and others can all pause for a human now. What we care about is what's coupled to the pause: the declared side effect, the precheck, the redaction, the idempotent ledger, tenant and user identity on every record, and an audit trail, all in one run lifecycle that survives a restart.
It's early. At the time of writing the runtime is 0.1.0a21, a pre-release hardened for a single node. Multi-worker execution is still open work.
Five things worth copying
Whatever you build on:
- Derive approval from the tool, not the prompt. Declare each tool's side effect in code and attach the rule to the declaration.
- Validate before you ask. Don't put a call in front of a person until it would actually run.
- Show the arguments, not the summary, with secrets redacted before they're stored, and let the approver change them.
- Save the pause before you ask, and let it expire. A decision that only lived in memory was never made.
- Key every side effect. An interrupted call is unknown until someone checks, never safe to retry.
The model is very good at deciding what to do. It shouldn't also be the one deciding whether it's allowed to.