Published 20 Jan 2026
Duration: 1822
The podcast discusses the evolution of data systems, shifting from building from scratch to reusing established technologies like Apache Arrow and Parquet for efficient data storage and analysis.
This interview was recorded for GOTO Unscripted. https://gotopia.tech Check out more here: https://gotopia.tech/articles/412 Andrew Lamb - Staff Engin...
The podcast examines the evolution of data systems, focusing on the increasing preference for reusing existing technologies to construct modern data solutions. It emphasizes a shift away from building systems from scratch, instead utilizing established tools such as Apache Arrow, Parquet, and Iceberg. These technologies facilitate efficient columnar storage, standardized data formats, and interoperability between different systems. The discussion highlights the advantages of columnar storage for analytical workloads, the role of Apache Arrow in unifying in-memory data formats, and the use of Parquet for efficient, persistent storage.
Additionally, the conversation covers the integration of query engines like Apache Data Fusion and the significance of open source collaboration in advancing database technologies. The podcast also touches on the rising adoption of time series databases, the incorporation of distributed systems, and how modern hardware is influencing the efficiency of data processing. These developments are shaping the future of data infrastructure by enabling more scalable, interoperable, and high-performance solutions.
28 Aug 2026 The Forest & The Desert Are Parallel Universes Kent Beck
"Contrasts abundance-based ('forest') and scarcity-driven ('desert') mindsets in software development, advocating for collaboration, adaptability, and trust over rigid control, with real-world examples and calls for alignment to bridge cultural gaps."
14 Aug 2026 The Platform Engineers Handbook Ajay Chankramath & Kaspar von Grunberg
"Explores platform engineering's evolution, practical applications, and challenges, emphasizing DevEx, AI integration, and long-term infrastructure investment to enhance software development efficiency and scalability."
7 Aug 2026 State of the Art of Platform Engineering Abby Bangser & Charles Humble
"Explores generative AI's impact on software engineering, platform evolution, and best practices for scalable, maintainable platforms while addressing AI's role in automation, reliability, and trust."
31 Jul 2026 The C4 Model: Visualizing Software Architecture Simon Brown & Susanne Kaiser
The C4 model is a lightweight, structured approach to visualizing software architecture with four levels (System Context, Container, Component, and Code) that prioritizes simplicity, traceability, and effective communication among teams.
17 Jul 2026 Signals & Levers Elisabeth Hendrickson, Joel Tosi & Charles Humble
"Modern software development faces persistent challenges, requiring systems thinking to address dynamic problems, human factors, and cultural shifts beyond Agile and DevOps limitations."