Want to create an interactive transcript for this episode?
Podcast: DataTalks.Club
Episode: Data Observability - Barr Moses
Description: We covered:
Barr’s background
Market gaps in data reliability
Observability in engineering
Data downtime
Data quality problems and the five pillars of data observability
Example: job failing because of a schema change
Three pillars of observability (good pipelines and bad data)
Observability vs monitoring
Finding the root cause
Who is accountable for data quality? (the RACI framework)
Service level agreements
Inferring the SLAs from the historical data
Implementing data observability
Data downtime maturity curve
Monte carlo: data observability solution
Open source tools
Test-driven development for data
Is data observability cloud agnostic?
Centralizing data observability
Detecting downstream and up...