2 Nov 2026 Educating an Architect: A Conversation with Luca Mezzalira
Architects must evolve from technical designers to social-technical enablers, using domain-driven design to balance AI automation with human oversight in core systems.
More Podcasts by InfoQ episodes

Published 9 Nov 2026
Duration: 00:43:37
WebAssembly and Tornado VM enable efficient local inference, while LLMD optimizes distributed GPU usage and semantic routing for on-premises AI deployment.
In this episode, Edoardo Vacchi discusses the evolving landscape of WebAssembly on the backend, running Java workloads on GPUs using TornadoVM, and op...
WebAssembly has evolved from a browser technology into a backend sandboxing platform, with the component model driving much of current development. Its relevance has grown alongside the agentic workload trend, and it enables cross-language distribution: a library written in Rust can be compiled to WebAssembly and consumed from multiple languages. Running small language models in a browser via Wasm is feasible, though larger models generally require GPU acceleration; experiments exist to translate Wasm into GPU kernels.
Tornado VM is a JVM plugin that compiles Java code into GPU-specific kernels such as PTX for NVIDIA or SPIRV. It allows writing GPU kernels in Java, with an example of running Llama 3 entirely in Java, though it cannot run arbitrary Java applications like Spring on the GPU.
LLMD, a CNCF project that recently reached 1.0, orchestrates distributed inference on Kubernetes by solving routing challenges unique to LLM workloads. Because LLM conversations are stateful through KV and prefix caches, traditional round-robin load balancing degrades performance; LLMD balances sticky sessions against cluster saturation. It also provides batch inference for idle hardware, semantic classification to route prompts to right-sized models, and observability metrics. Vache notes that even single-instance or bare-metal deployments benefit from LLMD's flow control and quality-of-service features, and that smaller open models - such as 27-billion-parameter models - can run on consumer hardware, making on-premises inference increasingly practical.
What if you ran small language model inference directly in browsers using WebAssembly for niche applications?
What if you deployed a Java-based LLM directly on GPUs using Tornado VM to reduce dependency on Python tooling?
What if you used LLMD to optimize inference efficiency on affordable, on-premise hardware instead of relying on cloud APIs?
2 Nov 2026 Educating an Architect: A Conversation with Luca Mezzalira
Architects must evolve from technical designers to social-technical enablers, using domain-driven design to balance AI automation with human oversight in core systems.
12 Oct 2026 What Does It Really Mean To Program With AI: A Conversation with Joy Ebertz
"Explores software architects' evolving role in balancing technical and business challenges, AI's impact on modular architectures, risks of automation, and the need for human oversight in AI-assisted development."
5 Oct 2026 Future Cybersecurity: Hardware Memory Safety, Automated Governance and Post-Quantum Cryptography
"Challenges and advancements in AI, security, and software development are discussed, emphasizing governance, open-source maintenance, hardware security, automation, and regulatory compliance like the EU's Cyber Resilience Act, alongside post-quantum cryptography and AI's dual role in security."
28 Sept 2026 The AI Revolution Fails Without Psychological Safety For Developers: A Conversation with Erin Doyle
"Software architects must adapt to AI-driven shifts in system design, balancing automation with human oversight, team alignment, and ethical considerations in modern development."
25 Sept 2026 The Future of AI: From Enterprise Adoption to Open Source Sovereignty
"Businesses rapidly adopt AI for efficiency but face challenges like ethical concerns, scalability, and unclear applications, while debates on open-source vs. proprietary models and AI's societal impact persist."