Want to create an interactive transcript for this episode?
Podcast: Software Engineering Daily
Episode: The Data Exchange with Ben Lorica
Description: Data infrastructure has been transformed over the last fifteen years. The open source Hadoop project led to the creation of multiple companies based around commercializing the MapReduce algorithm and Hadoop distributed file system. Cheap cloud storage popularized the usage of data lakes. Cheap cloud servers led to wide experimentation for data tools. Apache Spark emerged from academia, and Apache Kafka came out of the corporate challenges faced by LinkedIn.Over these 15 years, Ben Lorica has been following the world of data engineering as an engineer, a conference organizer, and a podcaster. When he was host o...