9 Nov 2026 Distributed LLM Inference on Kubernetes with llm-d, Wasm, & TornadoVM
WebAssembly and Tornado VM enable efficient local inference, while LLMD optimizes distributed GPU usage and semantic routing for on-premises AI deployment.
More Podcasts by InfoQ episodes

Published 25 Mar 2026
Duration: 00:54:31
Challenges in operationalizing agentic AI systemsblending microservices, ML, and autonomyinclude security risks, architectural gaps, observability issues, and balancing innovation with ethical, scalable, and sustainable implementation.
Are you ready for your new non-deterministic co-workers? Autonomous agents promise to help build, operate, and run software systems, but they can also...
The podcast explores the challenges of deploying AI in production environments, emphasizing the gap between experimental AI models and their operationalization. It introduces agentic systemsa novel domain blending microservices, traditional machine learning, and autonomous decision-makingwhich present unique complexities such as non-deterministic logic and inter-agent coordination. Examples highlight agentic use cases like autonomous incident response systems and non-agentic ones like deterministic chatbots, underscoring the need for distinct approaches to design and safety. Security risks, including prompt injection, API key leaks, and supply chain vulnerabilities, are prioritized, alongside the necessity for boundary definitions and risk management frameworks to prevent unintended actions in autonomous systems.
Operational and architectural challenges include ensuring observability in dynamic agent workflows, redefining software development lifecycles to accommodate autonomous systems, and balancing innovation with standardization. The discussion also addresses the shift in human roles from passive users to stewards of AI outputs, emphasizing accountability, ethical considerations, and the importance of explainability in complex agent behaviors. Open-source innovation and platform evolution are highlighted as critical for scalability, with a focus on cost efficiency, sustainability, and early experimentation to avoid overreliance on unproven technologies. Future trends point toward tighter AI-hardware integration, cross-organizational agent communication, and a growing need for standardized governance to manage the rapid expansion of agentic systems.
9 Nov 2026 Distributed LLM Inference on Kubernetes with llm-d, Wasm, & TornadoVM
WebAssembly and Tornado VM enable efficient local inference, while LLMD optimizes distributed GPU usage and semantic routing for on-premises AI deployment.
2 Nov 2026 Educating an Architect: A Conversation with Luca Mezzalira
Architects must evolve from technical designers to social-technical enablers, using domain-driven design to balance AI automation with human oversight in core systems.
12 Oct 2026 What Does It Really Mean To Program With AI: A Conversation with Joy Ebertz
"Explores software architects' evolving role in balancing technical and business challenges, AI's impact on modular architectures, risks of automation, and the need for human oversight in AI-assisted development."
5 Oct 2026 Future Cybersecurity: Hardware Memory Safety, Automated Governance and Post-Quantum Cryptography
"Challenges and advancements in AI, security, and software development are discussed, emphasizing governance, open-source maintenance, hardware security, automation, and regulatory compliance like the EU's Cyber Resilience Act, alongside post-quantum cryptography and AI's dual role in security."
28 Sept 2026 The AI Revolution Fails Without Psychological Safety For Developers: A Conversation with Erin Doyle
"Software architects must adapt to AI-driven shifts in system design, balancing automation with human oversight, team alignment, and ethical considerations in modern development."