Garth Mollett on AI Supply Chain Security

The podcast explores the unique security challenges posed by AI supply chains, emphasizing how they differ from traditional software supply chains. Unlike deterministic software systems, AI models produce probabilistic outputs, and their control and data planes are intertwined - meaning inputs can influence both behavior and execution. This convergence complicates security, as external data sources, user prompts, and training inputs all shape model behavior, blurring the boundaries between instruction and data.

Key vulnerabilities are discussed across the AI lifecycle, from pre-training to deployment. Risks include data poisoning, malicious datasets, and undetectable backdoors introduced during training or fine-tuning. The use of third-party data, model weights, and frameworks like Hugging Face introduces supply chain risks, while quantization and compression techniques can unexpectedly expose models to attacks. Security measures such as model signing, attestation, and AI-specific bills of materials (BOMs) are explored, alongside transparency initiatives like data cards, model cards, and system cards that document training data, intended use, and system architecture. However, challenges remain in verifying training data provenance, ensuring model reproducibility, and detecting malicious intent in AI behavior, especially as models gain capabilities in code generation, tool use, and autonomous reasoning.

• What if you treated your AI-generated code as a compromised third-party dependency?
  • Move: Implement automated static analysis and behavioral testing on all AI-generated code before integration; require human-reviewed guardrails for any system with write or network access.
  • Why Now?: AI models like DeepSeek have demonstrated politically biased or malicious code injection under specific prompts, and the probabilistic nature of outputs makes traditional QA insufficient.
  • Expected Upside: Reduces risk of undetected backdoors or vulnerabilities entering your software, especially critical as solo developers increasingly rely on LLMs for rapid coding without dedicated security review.
• What if you applied supply chain provenance checks to every AI model or dataset you use?
  • Move: Adopt OpenSSF's OMS (Open Model Specification) to sign and verify model metadata (training data source, hardware, fine-tuning steps); reject unsigned or unverifiable models from Hugging Face or other registries.
  • Why Now?: Malicious datasets (e.g., Python pickle exploits) and stealthy model poisoning (e.g., post-quantization attacks) are already occurring and commoditized in exploit kits.
  • Expected Upside: Prevents silent compromise during model deployment - especially vital for solo operators who lack teams to audit models, giving confidence that the AI behavior aligns with expected inputs and training provenance.
• What if you restricted AI model permissions based on real-time user intent like an enterprise system?
  • Move: Integrate short-lived, scope-limited tokens for AI tools (e.g., retrieval-augmented generation from internal docs) and enforce access policies via SPIFFE/SPIRE-style identity workflows - even in lightweight cloud environments.
  • Why Now?: The merging of control and data planes means prompts can hijack models into accessing or leaking data, and indirect prompt injection via poisoned RAG sources is already feasible.
  • Expected Upside: Mitigates data leakage and unauthorized actions by ensuring AI agents operate only within verified, narrow bounds - protecting your business data and reputation even if a model is subtly influenced.

• Audit and verify the provenance of all AI training datasets by requiring data cards or equivalent documentation that detail origin, collection methods, and potential biases before integrating models into production.
• Implement cryptographic signing of AI models and their metadata (e.g., weights, training frameworks) using standards like OpenSSF OMS to ensure integrity and traceability within internal model registries.
• Enforce fine-grained, short-lived workload identities for AI systems using tools like SPIFFE/SPIRE, ensuring models only access resources explicitly permitted by user intent and minimizing overprivileged access.
• Deploy pre- and post-quantization security checks to detect behavioral deviations, especially ensuring safety mitigations are not bypassed after model compression.
• Maintain an AI Bill of Materials (BOM) that logs all components including model versions, dependencies, and alignment methods, enabling accurate vulnerability tracking and supply chain transparency.