What Should an AI Agent Be Allowed to Do?

Suppose a bank allows its service team to waive one fee of up to £50 per customer in a rolling twelve-month period. An AI agent receives a request to waive a £40 charge. Its connected tool can post the credit.

The customer had another fee waived eight months ago. Under the standing policy, the agent can explain the limit or prepare a case for review. It cannot post a second credit.

If the earlier waiver was fourteen months ago, the same action might be permitted. A fraud hold or a change to the account would alter the answer again. Permission depends on this customer, this fee, the amount, the applicable policy and the state of the account at the time of action.

A role that can call the credit API does not establish whether this particular credit is authorized. The bank needs a decision when the agent attempts to act, based on records the agent cannot invent, with a trace of the rule and facts used.

That is the design problem for enterprise agents: turn delegated business authority into conditions the system can enforce while work is happening.

An agent’s job description is not its authority

A job description explains the outcome someone is expected to produce. Authority defines the decisions and actions they may take to produce it.

The distinction is easy to lose in an AI project. Teams describe an agent as “handling onboarding,” “supporting claims,” or “qualifying accounts.” Those phrases combine many different actions with very different consequences.

An onboarding agent might retrieve a document, classify it, decide that a file is complete, create a customer record, approve an exception or notify an external party. Giving the agent authority for one of those actions does not imply authority for the others.

The useful unit is not the agent. It is the action under a defined set of conditions.

This ladder avoids an unhelpful choice between “copilot” and “autonomous.” An agent may be autonomous when gathering evidence, constrained when recommending a decision and unable to execute without approval. Authority can vary inside the same workflow.

Define authority as an operating contract

An agent’s authority should be recorded in a form that an operating owner can understand and a technical system can enforce. It should answer six questions.

For the fee request, the authorization check receives the agent identity, customer and fee IDs, proposed amount, prior-waiver record, account status and policy version. It returns one of three outcomes: permit this action, require a named approver, or deny it under the current mandate.

A permit applies to that fee and amount for a short period. The credit tool checks it at execution. If the account changes between the check and the posting, the tool evaluates the conditions again. It records allowed and rejected attempts, because a request that was denied, one that was approved but never posted, and one that posted twice require different responses.

The World Economic Forum’s Agent Capability and Authorization Profile makes a similar distinction between an agent’s capabilities and the authority delegated to a particular deployment. That separation matters because many agents may share the same underlying model while operating under different mandates.

Evidence should open the gate, not merely support the explanation

Enterprise AI systems often treat evidence as something attached after a recommendation has been produced. The sources make the answer easier to review, but they do not determine whether the agent was permitted to act.

For consequential workflows, evidence should be part of the authorization decision.

A banking agent preparing a customer review might need verified identity, current ownership records, jurisdiction-specific documents and an unresolved-exception check. A revenue agent creating an outbound task might need a confirmed account match, a recent signal, an assigned owner and evidence that the contact is eligible for outreach.

The action gate should fail closed when required evidence is missing, stale or contradictory.

This changes the meaning of explainability. The system does not only show why the agent reached a conclusion. It proves that the conditions allowing the action were satisfied at that time.

Tool access is not permission to use the tool

Connecting an agent to a system often grants it a credential with broad technical access. The team then relies on instructions to prevent unwanted use.

That reverses the control model. The agent can perform the action technically and is being asked not to.

NIST has highlighted this problem in its work on software and AI agent identity and authorization. Its more recent discussion of identity foundations for agentic AI warns that shared credentials and long-lived API keys create accountability gaps and often provide access that is too broad for the task.

An enterprise agent should have its own identity. Its permissions should be bound to the workflow, the user or service delegating the work, the intended resource and the duration of the task.

Consider a customer-service agent that can read account information and issue credits.

The tool boundary must reject a disallowed action even when a prompt or plan asks for it.

IBM Research describes this as governance by construction: controls intervene before planning, around tool use, at approval points and when structuring outputs. The important idea is that governance operates throughout execution rather than reviewing the final result after the agent has already acted.

Human approval belongs at decision boundaries

Adding a person to every action sounds safe. It can also produce a system in which people approve requests too frequently, with too little context, to exercise meaningful judgement.

The relevant question is not whether a human is “in the loop.” It is what the person is being asked to decide.

A useful approval should present:

• the proposed action and its consequence;

• the evidence that satisfied the action gate;

• missing, conflicting or unusually weak evidence;

• the policy, threshold and authority being applied;

• the changes that will occur if approval is granted;

• the available alternative, including escalation or rejection.

Approval can then sit at the points where authority changes: when the agent moves from preparation to execution, crosses a value threshold, requests new access, encounters an exception or attempts an irreversible action.

Routine actions that already fall inside a signed-off mandate should not require repeated clicks. Excessive approval prompts train users to accept without reviewing. NIST compares this risk to consent fatigue: a control can remain visible while losing its practical effect.

This also clarifies accountability. The approver is not accepting responsibility for every internal step the agent performed. They are authorizing a defined business action based on the evidence and controls presented.

Reversibility should determine the control, not confidence alone

Teams often set automation thresholds using model confidence. Confidence matters, but it is not a measure of consequence.

An agent can be highly confident and still apply the wrong policy version, act on the wrong entity or use a tool outside the intended process. A moderately uncertain draft may be harmless if it is easy to inspect and discard.

Actions should therefore be classified by both uncertainty and reversibility.

The OWASP AI Agent Security Cheat Sheet recommends minimum tool access, per-tool permission scopes and explicit authorization for sensitive operations. Its related guidance also treats reversibility as a property of the action that should be declared in advance, not inferred by the agent during a run.

The distinction matters in enterprise GTM as well. Creating an internal account-research note is reversible. Enrolling a contact in an external sequence is not equivalent: the communication reaches a person, carries the company’s reputation and may trigger legal restrictions. The second action needs a different authority even when both originate from the same research.

Plan for partial failure before granting execution rights

An agent rarely performs only one operation. It may retrieve evidence, update a record, generate a document, send a notification and create a downstream task.

If the fourth step fails, the first three may already have changed the business state.

A retry can make the problem worse. The agent may issue a duplicate request, create another payment instruction or overwrite a manual correction made after the first attempt.

Execution authority should therefore include a recovery design:

Rollback is not always possible. A customer may have read the message. A payment may have left the institution. In those cases, the control is a compensating process with a clear owner, not a technical undo button.

This is where the boundary between agent design and operations disappears. Authority cannot be granted safely without knowing how the operating team will detect, own and resolve incomplete work.

The audit record should reconstruct authority, not expose hidden reasoning

Enterprises do not need an unlimited transcript of internal model reasoning. They need a reliable record of the facts, rules and actions that determined the outcome.

For each consequential run, retain:

• the identity and version of the agent deployment;

• the user, service or event that delegated the task;

• the mandate and policy version in force;

• the evidence used, with entity, source and timestamp;

• the checks performed and their results;

• the proposed and completed actions;

• approval, override and escalation events;

• tool responses, failures and recovery actions;

• the final business state.

NIST’s work on evaluation probes for agentic AI focuses on visibility into tool usage and supporting evidence, including structured audit trails that connect decisions to source material. That record is useful beyond compliance. It allows an operations team to determine why exception volume changed, whether a policy update affected outcomes and which control prevented or permitted a disputed action.

The audit record should be readable as an operational event history. A reviewer should be able to answer: who delegated this work, what authority applied, which evidence opened the gate, what changed, and who accepted any exception?

Govern the agent estate, not only individual deployments

A well-controlled agent can still create enterprise risk when nobody knows it exists, who owns it or which other agents depend on it.

APRA’s 2026 review of banks, insurers and superannuation trustees found that governance, assurance and operational resilience practices were not keeping pace with adoption. Its expectations include inventories of AI tools and use cases, accountable ownership across the lifecycle, human involvement in high-risk decisions and monitoring of third-party dependencies. APRA, Letter to Industry on Artificial Intelligence.

An agent register should contain operational facts, not only a project name and risk rating.

Authority should expire. A pilot’s permission should not quietly become a permanent production mandate. A change in model, tool, data source, business policy or downstream action may require the authorization profile to be reviewed.

This matters when agents delegate work to other agents. The receiving agent should not assume that a request is authorized because it came from another system. The chain of delegation needs to preserve identity, scope and limits, with authority becoming narrower rather than broader as work is passed on.

How the framework changes across functions

The structure is reusable, but the authority should remain specific to the business process.

The enterprise decides where the agent may proceed, encodes that decision and retains evidence that the boundary was respected.

A practical authorization review

Before an agent receives production access, ask the operating owner, risk owner and technical owner to agree the following.

If the answer depends on a careful user noticing something in free text, the control is not finished.

Put one action under a clear mandate

Start with a transaction the operating owner can describe precisely. For the waiver example, write the eligibility rule, name who may change it, specify the records the tool must check and decide how a mistaken credit is corrected. Test both sides of each boundary, including a prior waiver within twelve months and a hold added after approval.

Then grant the agent access to that one action. The test of readiness is whether the bank can show why each fee was or was not waived, which rule applied and where an exception went. The same discipline can govern a claim decision, an account review or an outbound message.

Questions that come up before deployment

Is a system prompt part of the authorization model?

It can describe the intended behaviour, but it should not be the only control over a consequential action. Scope, evidence gates, permissions, thresholds and approvals should be enforced outside the model wherever possible.

Should every agent have its own identity?

Each production deployment that can access data or take action should be attributable as a distinct non-human actor. That identity should also preserve the human or service that delegated the work, without simply sharing their full credentials.

How much autonomy is appropriate for a first deployment?

Begin with the most autonomous evidence-gathering and preparation that can be safely reversed. Keep consequential execution behind explicit approval or narrow pre-authorized limits until the operating team has measured performance, exception demand and recovery.

Who owns an agent’s authority?

The business owner should own the mandate and outcome. Technology owns the enforcement and operation of the system. Risk, security and compliance define or challenge the applicable controls. None of those roles should be replaced by the vendor supplying the model or platform.

What should trigger a new authorization review?

A material change in purpose, model, tools, data sources, permissions, policy, thresholds, user population, downstream action or vendor dependency. Incident patterns and repeated overrides should also reopen the mandate.