9 Nov 2026 Distributed LLM Inference on Kubernetes with llm-d, Wasm, & TornadoVM
WebAssembly and Tornado VM enable efficient local inference, while LLMD optimizes distributed GPU usage and semantic routing for on-premises AI deployment.
More Podcasts by InfoQ episodes

Published 25 May 2026
Duration: 00:41:23
AI and architecture challenges, Java's evolving ecosystem with performance optimizations and legacy practices, columnar data formats like Parquet, dependency management, and balancing AI adoption with developer skill retention.
Gunnar Morling, technologist at Confluent and Java Champion, shares his experiences with building high-performance applications in Java, especially in...
The discussion explores challenges in AI and architecture, emphasizing the long-term impacts of decisions on system design and the difficulty of predicting future technological landscapes. In Java development, the ecosystem has evolved with new features like the vector API and foreign memory API, alongside ongoing use of legacy practices. Java 17 is highlighted as a modern baseline, offering performance and maintenance benefits over older versions like Java 8, with a focus on upgrading for smoother transitions and management-driven cost savings through observability tools like flight recorder.
A durable execution engine is detailed, leveraging SQLite for state storage to manage long-running workflows via resumable execution and idempotent operations. This contrasts with traditional workflow engines by prioritizing code-centric simplicity and minimizing infrastructure complexity. Columnar data formats like Apache Parquet are discussed for their efficiency in aggregation queries, but challenges include dependency bloat and performance limitations in multi-threaded parsing, addressed by projects like Hardwood, which aims to create a lightweight, high-performance Parquet parser with minimal dependencies.
AI's role in development is examined, including its use in building tools like the Hardwood project and its potential to aid coding while raising concerns about skill erosion and developer roles. Themes of mechanical sympathy, performance optimization, and environmental costs of infrastructure are also addressed, alongside reflections on evolving programming languages and the balance between automation and human expertise. The text underscores the importance of design documentation, open-source collaboration, and adapting to challenges in talent acquisition and industry shifts.
What if you prototype a durable execution engine using SQLite for state tracking in your current Java project?
Action: Implement a proof-of-concept for resumable workflows using SQLite as the state store (e.g., a simple process that checkpoints progress manually).
Why now: Javas recent improvements (e.g., ZGC, compact object headers) reduce overhead for long-running processes, and SQLites lightweight, embedded nature avoids infrastructure complexity.
Expected upside: Youll gain hands-on experience with durable execution concepts, reduce recovery time from failures, and validate the feasibility of stateful workflows without external dependency chains.
What if you build a minimal Parquet parser with no mandatory dependencies using Java 9+ features?
Action: Create a lightweight Parquet reader that uses Javas logging abstraction and optional dependencies (e.g., S3 integration) to avoid Hadoop stack bloat.
Why now: The industrys shift toward columnar storage (e.g., Apache Iceberg) demands efficient, dependency-free tools, and Javas modular features (e.g., module system) simplify isolation.
Expected upside: Youll deliver a high-performance, self-contained tool that aligns with AI-native development principles, reducing maintenance costs and supply chain risks in your projects.
What if you integrate AI-assisted coding into your workflow with explicit design documents (e.g., ADRs)?
Action: Draft a high-level architecture decision record (ADR) for a new feature, then use AI to generate boilerplate code while maintaining strict API boundaries.
Why now: The text highlights AIs role in accelerating development (e.g., Hardwood project) and emphasizes the need to avoid over-reliance on automation. Structured design documents ensure clarity.
Expected upside: Youll streamline development speed while maintaining code quality, reducing redundant refactoring, and preparing your codebase for future scalability and team collaboration.
9 Nov 2026 Distributed LLM Inference on Kubernetes with llm-d, Wasm, & TornadoVM
WebAssembly and Tornado VM enable efficient local inference, while LLMD optimizes distributed GPU usage and semantic routing for on-premises AI deployment.
2 Nov 2026 Educating an Architect: A Conversation with Luca Mezzalira
Architects must evolve from technical designers to social-technical enablers, using domain-driven design to balance AI automation with human oversight in core systems.
12 Oct 2026 What Does It Really Mean To Program With AI: A Conversation with Joy Ebertz
"Explores software architects' evolving role in balancing technical and business challenges, AI's impact on modular architectures, risks of automation, and the need for human oversight in AI-assisted development."
5 Oct 2026 Future Cybersecurity: Hardware Memory Safety, Automated Governance and Post-Quantum Cryptography
"Challenges and advancements in AI, security, and software development are discussed, emphasizing governance, open-source maintenance, hardware security, automation, and regulatory compliance like the EU's Cyber Resilience Act, alongside post-quantum cryptography and AI's dual role in security."
28 Sept 2026 The AI Revolution Fails Without Psychological Safety For Developers: A Conversation with Erin Doyle
"Software architects must adapt to AI-driven shifts in system design, balancing automation with human oversight, team alignment, and ethical considerations in modern development."