AI Agents Do Not Need Better Policies. They Need an Action Inventory.
What This Is About
A new arXiv working paper, AI Agents Under EU Law, makes a useful argument for anyone building agentic systems in Europe: the compliance question is not "How autonomous is the model?" It is "What can the agent actually do in the outside world?"
That sounds subtle. It is not.
It changes where governance has to live.
For the last year, many organizations have treated AI governance as a layer around model usage: acceptable-use policies, model lists, prompt guidelines, human review statements, maybe a risk register if the organization is particularly awake. That was already thin for copilots. For agents, it is nowhere near enough.
An agent is not just a model that answers questions. It plans, calls tools, reads and writes data, touches other systems, triggers workflows, and sometimes changes external state. It sends the email. It updates the CRM. It modifies a ticket. It deploys code. It approves a refund. It ranks a candidate. It deletes the file you really hoped it would not delete.
At that point, the governance object is no longer the model alone.
It is the action surface.
The Paper's Core Claim
The paper argues that AI agents sit inside a broader EU regulatory stack. The EU AI Act matters, but it does not operate in isolation. Depending on what the agent does, the relevant obligations may also come from GDPR, the Cyber Resilience Act, the Digital Services Act, the Data Act, the Data Governance Act, NIS2, product liability rules, medical device law, financial regulation, employment law, and sector-specific frameworks.
That sounds like legal soup because, frankly, it is legal soup. But the practical takeaway is clean:
The regulatory profile of an AI agent is determined by its external actions, data flows, connected systems, affected persons, and decision context.
Not by whether the agent uses GPT-5, Claude, Gemini, a local model, a graph planner, a browser tool, or a swarm architecture.
The same technical architecture can be low-risk in one workflow and high-risk in another. A tool-calling model that summarizes meeting notes is one thing. A tool-calling model that screens job applicants, assesses creditworthiness, triages patients, or operates IT systems for critical infrastructure is a very different thing.
Same pattern. Different legal universe.
That is the part many enterprise AI roadmaps still understate.
The Wrong Mental Model: "We Govern the Model"
Most AI governance programs start from a model-centric view:
- Which models are approved?
- Which vendors are allowed?
- Where is data processed?
- What are the prompt rules?
- Who signs off on use cases?
Those questions still matter. But for agents, they are only the beginning.
The paper makes an important distinction: if an agent is built on top of a third-party general-purpose AI model, there are two regulatory layers. There is the model layer, with obligations for the GPAI model provider. And there is the system layer, where the agent provider or deployer turns that model into a concrete AI system with tools, purpose, interfaces, data access, and external effects.
That second layer is where a lot of real-world risk appears.
If your agent can access HR records, call internal APIs, write to production systems, send customer communications, or make recommendations that humans routinely rubber-stamp, the important governance question is not just "Which model is underneath?"
It is:
What exactly is this agent allowed to do, under which conditions, with which permissions, and with what evidence trail?
That question is architectural.
Prompt Rules Are Not Controls
One of the most useful practical points in the paper is about cybersecurity. Telling an agent "do not delete files" in a system prompt is not a security control.
It is a hope wearing a policy costume.
A real control lives outside the model:
- API-level permission scopes
- least-privilege credentials
- per-action authorization
- separation between read and write tools
- user confirmation for high-impact actions
- audit logs that distinguish user actions from agent actions
- non-human identity governance
- environment-based restrictions depending on input trust
This matters because agent failures are not only hallucination failures. They can become authorization failures, data-governance failures, logging failures, change-management failures, and incident-response failures.
If an agent can invoke tools, then tool permissions are part of the safety system. If it can store memory, then memory is part of the system state. If it can discover tools dynamically, then the tool catalogue is part of the conformity baseline. If it can adapt behavior after deployment, then drift monitoring becomes more than a nice-to-have dashboard.
The governance surface expands from "model plus prompt" to "runtime plus tools plus state plus permissions plus evidence."
That is the real engineering shift.
The Action Inventory
The most useful artifact implied by the paper is something many organizations do not yet maintain:
an Agent Action Inventory.
Not a list of AI use cases. Not a list of models. Not a spreadsheet of vendors.
A real action inventory should describe what the agent can do at the level where compliance and security actually attach:
- action name
- business process
- connected system
- read/write capability
- data categories
- personal data involved
- affected persons
- decision type
- possible legal effect or significant impact
- required human oversight
- approval threshold
- logging requirement
- credential scope
- rollback or remediation path
- regulatory triggers
This is where legal classification becomes operational.
An agent that "helps with recruiting" is too vague to govern. An agent that "ranks candidates for interview selection using CV data and hiring-manager feedback" is governable. It has a domain, a decision type, affected persons, personal data, potential bias concerns, and a clear connection to high-risk employment use under the EU AI Act.
An agent that "helps DevOps" is too vague. An agent that "opens pull requests only" is different from one that "can merge to main" or "can deploy to production" or "can rotate credentials."
The action is the unit of governance.
Why This Matters for Enterprise Agent Platforms
General-purpose agent platforms create a specific problem. Their value proposition is flexibility: connect tools, define tasks, automate workflows, let teams build.
Their compliance problem is also flexibility.
If the platform provider does not technically and contractually restrict intended purpose, then the foreseeable use space becomes broad. A generic agent platform can drift from harmless productivity assistant to regulated decision support very quickly, sometimes because a business team connected one more API on a Friday afternoon and called it "just a pilot."
That is how enterprise AI programs get into trouble. Not through one dramatic rogue model moment, but through small capability accretions that nobody maps as a new compliance boundary.
The paper's implication is uncomfortable but useful:
If you cannot restrict the intended purpose of a general-purpose agent platform, you may need to design for the most demanding foreseeable use tier.
That is not how most internal platforms are budgeted today.
But it is how regulators, auditors, CISOs, and risk teams will increasingly look at them.
Drift Is the Hard Stop
The strongest line in the paper is the conclusion that high-risk agentic systems with untraceable behavioral drift cannot currently satisfy the essential requirements of the AI Act.
This is not an abstract legal point.
Agent systems are runtime systems. Their behavior may change because of:
- model version updates
- tool catalogue changes
- memory updates
- policy changes
- prompt changes
- retrieved context changes
- permission changes
- workflow changes
- external API changes
If those changes are not versioned, monitored, and compared against a baseline, then the organization may not be able to explain what the system was at the time of a harmful action.
For low-stakes use cases, that may be messy.
For high-risk use cases, it can be disqualifying.
The practical requirement is not just logging outputs. It is versioning the runtime configuration of the agent: model, tools, permissions, memory state, policies, prompts, routing, and oversight rules. Without that, "post-market monitoring" becomes theatre.
And regulators tend not to enjoy theatre unless they are sitting in the audience.
What Serious Builders Should Do Now
The paper is a legal analysis, but its message is mostly architectural. If you are building or buying agent systems in Europe, I would translate it into five immediate moves.
1. Build the action inventory before scaling pilots
Every agent should have a structured inventory of actions, systems, data, permissions, decision types, affected persons, and oversight requirements.
If this sounds bureaucratic, compare it to discovering after deployment that your "assistant" is effectively making employment, financial, or health-related decisions. Paperwork is cheaper.
2. Move controls out of prompts
Use prompts to guide behavior. Do not rely on prompts to enforce boundaries.
Access control, least privilege, tool separation, approval thresholds, and write restrictions need to exist in the system architecture.
3. Treat tool permissions as production infrastructure
An agent credential is not a casual API key. It is a non-human identity with operational power.
It needs scope, ownership, rotation, logging, revocation, and review.
4. Version the runtime
If the agent changes because tools, memory, prompts, policies, or models changed, that should be visible. For regulated workflows, it should be auditable.
The baseline is not only "which model did we use?" It is "which complete runtime state produced this action?"
5. Classify by action, not by ambition
Do not classify an agent as low-risk because the project sponsor says it is "just productivity."
Classify the actions. A productivity wrapper can become a high-risk system the moment it influences hiring, credit, education, health, access to essential services, critical infrastructure, or legally significant decisions.
The Bigger Pattern
The agent era is pushing AI governance from policy documents into system design.
That is a good thing.
Policy-only governance was always brittle. It depended on humans remembering boundaries, teams interpreting guidelines consistently, and auditors reconstructing intent after the fact. Agents force the issue because their risk is concrete. They touch systems. They move data. They trigger effects.
So the control plane has to become concrete too.
The most important enterprise artifact for agent governance may not be the model card, the prompt library, or the AI ethics principle deck.
It may be the action inventory.
Because once you know what the agent can actually do, you can start making serious decisions about permissions, oversight, logging, classification, accountability, and deployment.
Without that, you are governing a shadow.
And shadows make terrible production systems.
Kommentar abschicken