Learn Python with Talk Python's Python courses

The Data Engineering Landscape in 2021

Episode #302, published Thu, Feb 4, 2021, recorded Fri, Jan 29, 2021.

This episode is carbon neutral.
I'm sure you're familiar with data science. But what about data engineering? Are these the same or how are they related?

Data engineering is dedicated to overcoming data-processing bottlenecks, data cleanup, data flow and data-handling problems for applications that utilize lots of data.

On this episode, we welcome back Tobias Macey to give us the 30,000 ft view of the data engineering landscape in 2021.



Links from the show

Live Stream Recordings:
YouTube: youtube.com

Tobias Macey: boundlessnotions.com

Podcast.__init__: pythonpodcast.com
Data Engineering podcast: dataengineeringpodcast.com

Designing Data-Intensive Applications Book: amazon.com
wally: github.com
lakeFS: lakefs.io
A Beginner’s Guide to Data Engineering: medium.com
Apache Airflow: airflow.apache.org
Dagster: dagster.io
Prefect: prefect.io
#68 Crossing the streams with Podcast.__init__: talkpython.fm/68
dbt: getdbt.com
Great Expectations: github.com
Dask: dask.org
Meltano: meltano.com
Languages trends on StackOverflow: insights.stackoverflow.com
DVC: dvc.org
Pandas: pandas.pydata.org
Episode transcripts: talkpython.fm

Want to go deeper? Check out our courses

Tobias Macey
Tobias Macey
Tobias Macey is a dedicated engineer with experience spanning many years and even more domains. He currently manages and leads the Technical Operations team at MIT Open Learning where he designs and builds cloud infrastructure to power online access to education for the global MIT community.
Episode sponsored by
Ads served ethically
X
Become a friend of the show
Stay in the know and get a chance to win our contests.
See our privacy statement about email communications.