Governing AI agents: permissions, approvals and logs
A chatbot that answers questions can be wrong. An agent that sends emails, changes records and calls APIs can be wrong and act on it. Governing agents is less about the model and more about what it is allowed to touch.
In short
OWASP names the core risk excessive agency and traces it to three roots: too much functionality, too many permissions, too much autonomy. The controls follow directly: give the agent narrow tools instead of a shell or open web access; let it act in the user's context with the least privilege; enforce authorization in the target system, not in the model's instructions; require human approval for actions with high impact; and keep logs detailed enough to reconstruct what happened. Logs and rate limits reduce damage; they do not prevent it.
Three roots of the problem
| Root | What it looks like | Control |
|---|
| Excessive functionality | The agent has a tool that can do far more than the task requires: a shell, a general HTTP client, write access to an entire mailbox | Narrow, purpose-built tools. One tool for each action the task requires |
| Excessive permissions | The tool runs under a service account that can read and change everything | Least privilege; act in the context of the specific user; separate identity for each agent |
| Excessive autonomy | High-impact actions run without anyone confirming them | Human approval for actions that delete, pay, send externally or change access |
The breakdown comes from OWASP's entry on excessive agency in its Top 10 for LLM applications, and it is the most useful place to start an agent design review.
Authorization belongs in the target system
The most common design error is putting the rules in the prompt: do not delete records, only read the customer's own data. A prompt is a request, not a control. Anything that can reach the model, including text inside a document or an email the agent reads, can try to change its behavior.
Rule of thumb: assume the agent will eventually try to do whatever its tools allow. If that outcome is unacceptable, the tool or its permissions are wrong, whatever the instructions say.
So the target system checks every request on its own terms: who the user is, what they may do, whether this record belongs to them. The agent's identity has only the rights the task needs. This is the same principle as server-side authorization in any application, applied to a new kind of client.
Which actions need a human
| Action | Default |
|---|
| Read data the user may already see | Agent acts alone |
| Draft a message, a record or a change | Agent acts alone; a person sends or saves |
| Send anything outside the organization | Human approval |
| Delete or overwrite data | Human approval, with a way to undo |
| Payments, refunds, credit changes | Human approval, and limits enforced by the payment system |
| Change permissions, configuration or deployments | Human approval; ideally not available to the agent at all |
The approval step has to show what will actually happen, not the agent's summary of it. A person approving a paraphrase approves the paraphrase.
Logs you can reconstruct
- Every tool call with its inputs and outputs, the identity it ran under and the time.
- The user and the conversation that led to the action.
- Approvals: who approved what, and what they were shown.
- Failures and refusals, not only successes.
The test: after an incident, can you answer what the agent did, on whose behalf, with which permissions and who allowed it? If not, the logs are decoration.
A governance checklist for each agent
- Written purpose and a list of actions the agent may take.
- Tools mapped one to one to those actions; nothing more.
- Identity and permissions scoped to the purpose, separate from people's accounts.
- Authorization enforced by the systems it calls.
- Approval rules for high-impact actions.
- Logs that meet the reconstruction test.
- An owner who can switch the agent off, and a tested way to do it.
Frequently asked questions
What is excessive agency in AI agents?
OWASP's term for the risk that an LLM-based agent can perform damaging actions because it has too much functionality, too many permissions or too much autonomy. It is listed in the OWASP Top 10 for LLM applications.
Can a system prompt keep an AI agent safe?
No. Instructions in the prompt are requests, and content the agent reads can try to override them. Safety comes from narrow tools, least privilege and authorization enforced by the target system.
Which agent actions need human approval?
Anything with high impact: sending outside the organization, deleting or overwriting data, payments and refunds, and changes to permissions or configuration. The person approving should see the actual action, not a summary.
Should an AI agent use a shared service account?
No. Each agent should have its own identity with the permissions its purpose requires, and where possible act in the context of the user it serves.
Are logs enough to govern an agent?
No. Logs and rate limits reduce and reveal damage but do not prevent it. They have to be combined with least privilege, narrow tools and approval for high-impact actions.