SE Radio 733: Max Corbridge on Securing AI Agents

AI agent security presents a novel and evolving challenge due to the autonomy and non-deterministic nature of these systems, making them inherently more vulnerable than traditional software. A core issue is prompt injection, a systemic vulnerability where attackers manipulate AI models through carefully crafted inputs, either directly or indirectly via poisoned data sources like web pages or documents. The flexibility of natural language allows for infinite variations of attacks, rendering traditional input filtering ineffective - similar to how SQL injection plagued early web applications, but far more complex due to the lack of rigid syntax in language models.

The risks are amplified by AI agents' increasing integration with real-world tools and systems via protocols like MCP, which expand their attack surface. Threats include malicious MCP servers, indirect prompt injection through third-party data, and data poisoning during training, where minimal contamination can lead to widespread model compromise. Current defenses such as guardrails, sandboxing, and human-in-the-loop approvals offer partial protection but struggle to scale. Effective security requires a shift from preventing all attacks to assuming compromise and focusing on runtime monitoring, behavioral analysis, and limiting agent permissions through least privilege and identity management - approaches that must evolve to handle the massive scale and dynamic behavior of AI agent deployments.

• What if you treated your AI agent like a zero-trust employee with full system access?

  • Move: Implement a runtime monitoring layer (e.g., Adrian or custom telemetry) that logs both the agent's actions (API calls, file access) and its reasoning chain (intermediate thoughts) for every execution. Enforce a strict output contract: only pre-approved action types (e.g., "send email via Postmark") are allowed to execute.
  • Why Now?: With AI agents increasingly interacting with critical systems (banking, HR, code repos), and prompt injection being declared "unsolvable" by frontier labs, assuming compromise is now a baseline necessity - especially as regulations like the EU AI Act loom.
  • Expected Upside: Detect malicious or rogue behavior (e.g., data exfiltration, unauthorized API use) even if prompt injection bypasses input filters. Gain 35% higher detection accuracy by combining reasoning + activity monitoring, reducing post-exploit damage.

• What if you sandboxed your AI agent from day one - before granting any real-world tool access?

  • Move: Deploy your agent in an isolated cloud environment (e.g., ephemeral container on AWS Fargate) with no direct access to production APIs, databases, or file systems. Use a mock MCP (Model Context Protocol) server to simulate tool interactions and validate behavior before promoting to production.
  • Why Now?: AI agents introduce a 10x larger attack surface than traditional software, and indirect prompt injection (e.g., via poisoned READMEs or LinkedIn bios) is already feasible. Most frameworks (LangChain, Llama Index) ship without secure defaults.
  • Expected Upside: Prevent accidental or malicious execution of destructive commands (e.g., delete all files, send .env to attacker). Reduce risk of data poisoning and lateral movement, while maintaining agility for solo development.

• What if you built your agent with least-privilege identity and dynamic permissions?

  • Move: Create a dedicated non-human identity (service account) with minimal, role-specific permissions (e.g., read-only access to HR docs, no email send rights). Rotate credentials automatically per session using short-lived tokens, and log all identity-bound actions to a separate audit trail.
  • Why Now?: Identity providers aren't built for millions of ephemeral agent identities, and granting broad access (e.g., Co-Pilot-style full permissions) creates systemic risk - especially with 350M+ agent executions observed in enterprise settings.
  • Expected Upside: Limit blast radius if compromised (e.g., attacker can't reset admin passwords or send phishing emails). Achieve clear attribution and compliance readiness ahead of EU AI Act enforcement, while scaling securely as a solo operator.

• Implement strict least privilege access for AI agents by granting only the minimum permissions necessary for their specific role, such as limiting a customer service agent to ticket-closing actions and blocking access to financial systems.
• Deploy AI agents in isolated sandbox environments - especially for coding or data-processing tasks - with restricted internet access and monitored outbound connections to limit exposure from prompt injection or data exfiltration.
• Integrate runtime monitoring tools like Adrian (AI Detection and Response) to analyze both agent actions and reasoning traces (chain-of-thought) in real time, enabling detection of subtle attacks such as indirect prompt injection or anomalous behavior.
• Adopt a "assume compromise" security model by focusing on behavioral monitoring and output validation rather than relying solely on input filtering; allow only pre-approved, contract-defined actions to execute in production.
• Establish agent identity management using dedicated non-human service accounts with audit trails, and implement dynamic UID rotation or data spotlighting to distinguish trusted instructions from untrusted inputs in agent workflows.