An AI Agent Should Not Inherit Every Permission Its User Holds

The next operating question for AI agents is how much authority each task actually requires, and how a mistaken action can be contained.

Authority is part of the product

An assistant that suggests a sentence and an agent that changes a business record create different operational questions. The second system can affect the world through the permissions attached to it. Evaluating the quality of its answers is therefore only part of the assessment. A business also needs to examine what the system can do when an answer is wrong or a request is misunderstood.

NIST's Generative AI Profile, published in July 2024, is a companion to its voluntary AI Risk Management Framework. It provides a risk-management resource, not a guarantee that any particular deployment is safe. This analysis applies that broader risk perspective to task permissions. It does not describe these recommendations as newly enacted law or as a verbatim NIST implementation checklist.

Begin with the smallest useful action

A task should be described in terms of the operation required, not merely the job title of the person asking. Finding an invoice, preparing a correction and approving a payment are distinct actions. An agent helping with one of them does not automatically need authority over the others.

For illustration, a system that prepares a draft customer response may need access to selected order details. It need not necessarily be able to change a shipping address or issue a refund. Separating these capabilities makes the design more precise. It also gives the organisation a way to learn from a bounded use case before connecting actions with larger consequences.

Treat read access as a consequential choice

Restrictions should not focus exclusively on writing or deleting information. Access to sensitive records can matter even when the system cannot modify them. A useful design considers which information is necessary for the task and whether the output could reveal information to someone who should not receive it.

Imagine an internal request to summarise a client relationship. The employee may be entitled to see ordinary account activity without having a legitimate need for unrelated personal information in another record. The agent should not treat the convenience of searching everything as proof that every result belongs in its response. Information boundaries should follow the authorised purpose, not simply the technical reach of the connection.

Put consequential transitions in a visible place

An operating workflow becomes easier to govern when it clearly distinguishes preparation from execution. A draft change can be examined before it alters a live record. A proposed external communication can show the destination and relevant content before it is sent. The boundary should be designed around consequences rather than an arbitrary number of clicks.

This does not mean demanding the same review for every action. An unnecessary approval requirement can encourage habitual confirmation without attention. The aim is to place scrutiny where it can still prevent a material mistake. The person reviewing the action needs enough context to understand what will change, not merely an alert that a system is ready to continue.

Design a meaningful stop

An organisation should know how to suspend an agent's ability to act without relying on the system to interpret a new instruction correctly. The operational team needs a clear route to remove access, stop a workflow and identify actions already performed. A stop mechanism is useful only if responsible people can find and use it under pressure.

A rehearsal can begin with a fictional error and follow the response. Who receives the report? Which authority can disable the relevant connection? What remains queued? What information is available for investigating the event? These questions make the design testable. They also reveal whether the apparent simplicity of an agent interface hides an unclear chain of responsibility behind it.

Record actions in language the business understands

A technical log can establish that a request occurred while failing to explain its commercial meaning. For consequential work, the organisation should be able to connect an action with the initiating request, the relevant authority and the record affected. The amount of detail retained should respect privacy and the purpose of the process.

The objective is reconstruction, not indiscriminate collection. If a mistaken change appears, someone should be able to identify its scope and decide how to respond. A business that cannot distinguish a recommendation from an executed action will struggle to investigate problems accurately. Clear records support learning and accountability without pretending that recording an action makes it appropriate.

Test the permission boundary itself

A useful rehearsal includes a request the agent should decline or escalate because it exceeds the authorised task. The team can then observe whether the boundary exists in practice rather than only in a description. A system that performs permitted work well but also completes a consequential unauthorised action has not satisfied the intended design. Record the expected refusal or escalation alongside the successful cases. This makes restraint part of the acceptance criterion, instead of treating any additional capability as an automatic improvement.

Review permissions when the task changes

A successful pilot may gradually acquire new features, data sources and responsibilities. The original access decision can become outdated even when no single addition appears dramatic. Permission review should therefore accompany meaningful changes in scope rather than occur only during the first setup.

My conclusion is that agent usefulness depends partly on disciplined limits. A system can be valuable while being unable to perform many actions its user could perform personally. The relevant standard is sufficient authority for a defined task, with understandable transitions and a workable response to error. Capability becomes easier to trust when the organisation can explain both its purpose and its boundary.

Gabriel Nicolaev — CyGuru

Sources & further reading

NIST — Generative AI Profile, published 26 July 2024 ↗

Editorial photograph for illustration.

FOLLOW THE CONNECTION

Another angle on this subject

Continue through related ideas, decisions and consequences.

The CyGuru Column

Build a Life That Does Not Require Constant Urgency

When every decision feels immediate, the deeper question is which commitments keep manufacturing the next emergency.

Money & Business

Services Exporters Need a Margin Map, Not Just a Demand Forecast

The WTO’s split outlook for goods and services raises a practical question: which contracts still create value after travel, delivery and operating costs?

Human Behaviour

An Apology Becomes Credible When It Changes the Next Encounter

Repair requires a different future arrangement, not simply a persuasive explanation of what went wrong.