Monitor performance issues & errors in your code

Solving Negative Engineering Problems with Prefect

Episode #365, published Thu, May 12, 2022, recorded Mon, May 9, 2022

How much time do you spend solving negative engineering problems? And can a framework solve them for you? Think of negative engineering as things you do to avoid bad outcomes in software. At the lowest level, this can be writing good error handling with try / except. But it's broader than that: logging, observability (like Sentry tools), retries, failover (as in what you might get from Kubernetes), and so on. We have a great chat with Chris White about Prefect, a tool for data engineers and data scientists meaning to solve many of these problems automatically. But it's a conversation applicable to a broader software development community as well.

Watch this episode on YouTube
Play on YouTube
Watch the live stream version


Links from the show

Chris White: @markov_gainz
Prefect: prefect.io
Fermat's Enigma Book (mentioned by Michael): amazon.com
Prefect Docs (2.0): orion-docs.prefect.io
Prefect source code: github.com
A Brief History of Dataflow Automation: prefect.io/blog
Watch this episode on YouTube: youtube.com
Episode transcripts: talkpython.fm

--- Stay in touch with us ---
Subscribe to us on YouTube: youtube.com
Follow Talk Python on Mastodon: talkpython
Follow Michael on Mastodon: mkennedy

Want to go deeper? Check out our courses

Chris White
Chris White
Chris White is a mathematician turned machine learning engineer, passionate about building data-intensive tools which expose the right abstractions. He is the CEO at Prefect.

He received my Ph.D. in Mathematics from The University of Texas at Austin under the direction of Prof. Rachel Ward in August 2015. Broadly speaking, his research consisted of developing and analyzing new data-driven optimization problems. Outside of my research I also enjoyed working on various consulting and personal projects - anything involving analyzing data and/or modeling.
Episode sponsored by
Ads served ethically
Talk Python's Mastodon Michael Kennedy's Mastodon