More Software Engineering Radio episodes
Published 14 Jan 2026
Duration: 54:47
A key factor in AI applications is reducing latency, which requires a balance between model size, accuracy, and performance, along with ongoing optimization and monitoring.
In this episode, Sahaj Garg, CTO of wispr.ai, joins SE Radio host Robert Blumen to talk about the challenges of building low-latency AI applications....
The podcast emphasizes the critical role of latency in applications, especially within AI systems, and how it affects user perception and overall experience. It explains that users are sensitive to both average and tail latency, and that acceptable thresholds vary depending on the specific application. The discussion explores various techniques aimed at reducing latency while preserving accuracy, such as speculative decoding and model distillation. It also outlines the challenges involved in managing AI workloads, including network delays, deployment configurations, and the necessity for ongoing monitoring and optimization.
The conversation further examines the trade-offs between model size, accuracy, and latency, highlighting the use of quantization and efficient compute architectures as ways to enhance performance. It underscores the importance of aligning AI outputs with user expectations, considering factors like the balance between response size and usability. The podcast also notes the value of tailoring AI systems to specific application contexts, such as voice dictation, recommendation systems, and interactive interfaces, to ensure optimal performance and user satisfaction.
16 Sept 2026 Milan Milanovic on the Laws of Software Engineering
"Explores key software engineering principles (like Conway's Law, Brooks' Law) and their impact on systems, teams, and decision-making, emphasizing context-dependent trade-offs, AI's role, and practical applications like measuring technical debt."
9 Sept 2026 Owen McGirr on Software Accessibility
"Accessibility in software development must be prioritized from the start, integrating inclusive design practices like multiple input methods, proper labeling, and user testing to benefit all users, not just those with disabilities."
3 Sept 2026 Sahil Walia on Apache Iceberg
"Apache Iceberg is a scalable, interoperable data framework that unifies OLTP and OLAP workloads, separates storage and compute, and enables efficient metadata-driven operations, governance, and cost savings across industries."
26 Aug 2026 Vivek Yadav on Regression Testing Microservices
"Explores microservices testing strategies, behavioral consistency in migrations, payment system challenges, regression testing, historical data validation, testable architecture, Spark's role, and AI-driven code changes, emphasizing data privacy and business insights."
13 Aug 2026 SE Radio 733: Max Corbridge on Securing AI Agents
"Explores AI agent security risks, focusing on prompt injection vulnerabilities, non-deterministic threats, and the need for dynamic monitoring and proactive defenses against evolving AI-specific attacks."