Agentic AI Security
How I think about securing AI agents in production: the attack surface once a model can act, the architectural controls that actually hold up, and where authorization and delegation security specifically fit.
Agentic AI security is the set of architectural controls that keep an AI agent's actions bounded and auditable once it can call tools, touch operational systems, and take actions with real consequences. It covers goal hijacking and prompt injection, tool misuse, identity and privilege abuse, memory integrity, supply-chain risk from connectors, and observability of what an agent actually did. The moment a model gains the ability to act, the central question shifts from what did it say to what is it authorized to do — which is why authorization and delegation are a distinct, load-bearing layer within it, not an implementation detail.
The Agentic Attack Surface
Agent Goal Hijack
Content the agent processes — a document, a page, a tool result — redirects it away from the user's actual task, typically via indirect prompt injection.
Tool Misuse & Exploitation
An agent invokes a tool outside its intended purpose, with unsafe parameters, or in a sequence its individual permissions never anticipated.
Identity & Privilege Abuse
Agent, sub-agent, and service-account credentials are over-scoped, under-audited, or don't narrow correctly as authority is delegated down a chain.
Memory & Context Integrity
Uncontrolled memory growth, stale context, or poisoned retrieved content lets attacker-influenced state persist and resurface across sessions.
Supply Chain & Connector Risk
Third-party tools, MCP servers, and plugins extend the agent's reach — and its blast radius — often with weaker vetting than the core model.
Insufficient Observability
Without a decision record for why an action was selected, what checks ran, and what evidence backs it, incidents are unreconstructable and audits fail.
Controls That Hold Up in Production
Guardrails that live only in a prompt don't survive contact with a determined attacker or a genuinely ambiguous task. What holds up is enforced by infrastructure the agent cannot talk its way around.
- 01Deny-by-default tool contracts: explicit schema, scope, idempotency, and side-effect classification per tool, not implicit trust in whatever the model decides to call.
- 02Scoped identity and least privilege: session-scoped credentials for agents and sub-agents, not standing, broadly-scoped service accounts reused across every workflow.
- 03Policy enforcement outside the model: authorization decisions made by infrastructure the agent cannot influence, not by an instruction the model is asked to follow.
- 04Runtime budgets and reversibility: timeouts, step ceilings, and a hard split between reversible and irreversible actions, with irreversible ones routed through approval.
- 05Evidence and audit trails: every privileged action tied to a recorded decision — identity, permissions checked, and outcome — reconstructable after the fact.
Where Authorization Fits
Content-level defenses against prompt injection are best-effort: frontier models still fall to adaptive, human-crafted attacks. Authorization enforced outside the model is not — even a fully manipulated agent can only execute what its current scope, budget, and composition restrictions allow.
That's straightforward for a single agent acting on a single request. It gets harder once authority has to flow correctly through a delegation chain — human to orchestrator to sub-agents to tools — narrowing at every hop, and once individually permitted actions can combine into an outcome none of them were permitted for on their own. That's the specific problem I work on: a formal treatment is in Bounded Agents, and a general explainer of why per-request models break down for agents is in AI Agent Authorization.
Standards & Frameworks
None of these mandate a single implementation. They describe the risk categories and control objectives an architecture like the one above is built to satisfy: the OWASP GenAI Security Project'sagentic AI guidance, the Cloud Security Alliance's MAESTRO threat-modeling framework for multi-agent systems, MITRE ATLAS for AI-specific adversarial tactics, and NIST's AI Risk Management Framework.
Frequently Asked Questions
What is agentic AI security?
Agentic AI security is the set of architectural controls that keep an AI agent's actions bounded and auditable once it can call tools, touch operational systems, and take actions with real consequences — as opposed to LLM security, which is mostly about the safety of generated content. It spans goal hijacking and prompt injection, tool misuse, identity and privilege abuse, memory integrity, supply-chain risk from connectors and tools, and observability of what the agent actually did and why.
How is agentic AI security different from LLM security?
LLM security is largely about what a model says: jailbreaks, harmful content, hallucination. Agentic AI security is about what a model does: which tools it calls, what data it touches, what it changes. The moment an LLM gets the ability to execute actions, the primary risk shifts from bad content to bad actions, and the controls have to move with it — from content filtering toward authorization, identity, and runtime enforcement.
What are the biggest security risks of AI agents?
In production, the risks that actually cause incidents are: ungoverned tool access (agents invoking tools outside their intended scope), unclear identity and permission models (no scoping of agent credentials or session identity), and authorization that only checks one action at a time — which lets individually permitted actions combine into an unintended outcome, such as reading confidential data and then sending it externally. Prompt injection is the delivery mechanism for many of these; authorization and tool governance determine how much damage it can actually cause.
How do you secure AI agents in production?
By treating security as an architectural decision, not a layer added after deployment: tool registries with strict contracts and side-effect classification, scoped identity and least-privilege authorization per agent and session, explicit approval paths for irreversible actions, runtime budgets and deterministic failure handling, and decision records that tie every privileged action to the checks that ran. This is the same approach applied in my Agentic AI Architecture and Controls and AI Security Architecture case studies.
What is the role of authorization in agentic AI security?
Authorization is the control that decides what an agent is actually allowed to do, and it's where a lot of agentic AI security either holds or fails. Content-level defenses against prompt injection are best-effort — frontier models still fall to adaptive attacks. Authorization enforced outside the model is not: even if an agent is fully manipulated, it can only execute what its current scope, budget, and composition restrictions allow. This is the specific problem my research addresses — see AI Agent Authorization and Bounded Agents below.
How does prompt injection relate to agentic AI security?
Prompt injection is dangerous specifically because the model has been given authority to act on what it processes. An agent that reads untrusted content and can also call tools is exposed to indirect prompt injection turning into real-world action. Content-level detection helps but doesn't reach zero against adaptive attackers, so agentic AI security treats injection as a standing possibility and focuses on bounding what a manipulated agent can execute — the approach taken in Bounded Agents' adversary model.
What standards apply to agentic AI security?
The most relevant current frameworks are the OWASP GenAI Security Project's agentic AI guidance (including the Top 10 for Agentic Applications), the Cloud Security Alliance's MAESTRO threat-modeling framework for multi-agent systems, MITRE ATLAS for AI-specific adversarial tactics, and NIST's AI Risk Management Framework. None of them mandate a single implementation — they describe the risk categories and control objectives that an architecture like the one described here is built to satisfy.