The Data Engineering Landscape in 2021

Episode #302, published Thu, Feb 4, 2021, recorded Fri, Jan 29, 2021

Transcript Embed

I'm sure you're familiar with data science. But what about data engineering? Are these the same or how are they related?

Data engineering is dedicated to overcoming data-processing bottlenecks, data cleanup, data flow and data-handling problems for applications that utilize lots of data.

On this episode, we welcome back Tobias Macey to give us the 30,000 ft view of the data engineering landscape in 2021.

Play on YouTube

Watch the live stream version

Links from the show

Live Stream Recordings:
YouTube: youtube.com

Tobias Macey: boundlessnotions.com

Podcast.__init__: pythonpodcast.com
Data Engineering podcast: dataengineeringpodcast.com

Designing Data-Intensive Applications Book: amazon.com
wally: github.com
lakeFS: lakefs.io
A Beginner’s Guide to Data Engineering: medium.com
Apache Airflow: airflow.apache.org
Dagster: dagster.io
Prefect: prefect.io
#68 Crossing the streams with Podcast.__init__: talkpython.fm/68
dbt: getdbt.com
Great Expectations: github.com
Dask: dask.org
Meltano: meltano.com
Languages trends on StackOverflow: insights.stackoverflow.com
DVC: dvc.org
Pandas: pandas.pydata.org
Episode transcripts: talkpython.fm

--- Stay in touch with us ---
Subscribe to us on YouTube: youtube.com
Follow Talk Python on Mastodon: talkpython
Follow Michael on Mastodon: mkennedy

Want to go deeper? Check out our courses

Course: Python for the Absolute Beginner course

Course: HTMX + Flask: Modern Python Web Apps, Hold the JavaScript

Course: Modern APIs with FastAPI and Python

Tobias Macey

Tobias Macey is a dedicated engineer with experience spanning many years and even more domains. He currently manages and leads the Technical Operations team at MIT Open Learning where he designs and builds cloud infrastructure to power online ...

Episode sponsored by

Ads served ethically