Ask us about modern Python projects and tools

00:00 Here's an episode that I did not see coming.

00:02 Sebastian Wibowski and I put together a live stream, Ask Me Anything, as a follow-up to

00:06 some of his ideas around his recent course, Modern Python Projects.

00:09 We dove deep into comparisons of poetry versus pip versus pyenv and answered questions like,

00:15 do I need to use Docker?

00:17 And when should I?

00:17 And so on.

00:18 After the AMA was over, I realized it would make a great podcast too.

00:22 So here you go.

00:23 This is our AMA with Sebastian, all around the ideas of modern Python workflows.

00:28 I hope you enjoy it.

00:29 This is Talk Python to Me, episode 314, recorded April 19th, 2021.

00:35 Welcome to Talk Python to Me, a weekly podcast on Python, the language, the libraries, the

00:53 ecosystem, and the personalities.

00:54 This is your host, Michael Kennedy.

00:56 Follow me on Twitter where I'm @mkennedy.

00:58 And keep up with the show and listen to past episodes at talkpython.fm.

01:02 And follow the show on Twitter via at talkpython.

01:05 This episode is brought to you by 45 Drives and us over at Talk Python Training.

01:10 Please check out what we're offering during those segments.

01:12 It really helps support the show.

01:13 Everyone out there, thank you so much for being here today.

01:18 It's exciting to have you here in the live stream, or if you're watching later, watching

01:22 the record.

01:23 So Sebastian, welcome.

01:25 It's great to have you part of this Ask Me Anything.

01:28 In fact, it's your Ask Me Anything, really.

01:30 Thank you.

01:30 Yeah, I'm excited to be here.

01:31 Yeah, of course, I'll give my thoughts as well.

01:34 But you've done a lot of thinking about tooling and putting the right tools together and what

01:39 maybe constitutes what you might call a modern Python project or the tool chain of a modern

01:45 Python developer.

01:46 And there's a couple of things that we've already done together that are maybe worth

01:50 calling out that we'll build on here today.

01:53 So a while ago, I guess, what was this?

01:55 Oh my gosh, this was back last year, August 29th, 2020.

01:59 You were on Talk Python.

02:01 You came on and talked about the modern Python developers toolkit.

02:04 And then we talked a little bit after that and said, you know, it would be really cool actually

02:07 to put a course like this together for people.

02:09 So over at Talk Python training, you created the Modern Python Projects course.

02:13 This is almost a nine hour course that takes a lot of the ideas you mentioned there previously

02:18 on the podcast and makes them concrete, makes it something that actually people can employ

02:24 and use.

02:25 And so, yeah, those are some of the things that we've done together before.

02:29 But we're just going to take a broader view and talk about what this whole idea of modern

02:35 Python projects might be.

02:36 Yeah.

02:37 So let's just kick it off with a quick high level overview.

02:41 Like, what are your thoughts?

02:42 What constitutes modern Python development?

02:45 How has it changed over time?

02:47 Things like that.

02:47 Then we'll get to the people's individual questions.

02:49 Yeah.

02:50 So as you mentioned, this whole idea started as PyCon workshop where I wanted to share with

02:55 people some ideas for tools that I've been using.

02:59 And I know that a lot of other people have been using because I sometimes see people like

03:04 start programming in Python and they still stick to using the default Python shell.

03:09 And they don't know about many great tools that are in the ecosystem.

03:13 You'll hear like, oh, I used idle to try to run this.

03:16 And you're like, whoa, whoa, whoa, please don't use idle.

03:18 I mean, it's built in, but there's really more helpful options these days, right?

03:21 Something like this.

03:22 Yeah.

03:23 I mean, Python is really cool with the batteries included because you have a lot of things

03:27 there.

03:27 But at the same time, there are a lot of other different tools and projects that can make

03:32 your life much, much easier.

03:33 And that's what I wanted to share with people.

03:36 Nice.

03:36 And I would like to point out for people who maybe don't know, there's a lot of amazing

03:41 tools that are not part of Python on purpose.

03:44 And the reason they're not part of Python is not that they're amazing.

03:47 It's that Python releases, it used to be every 18 months and now every 12 months.

03:52 And once something goes into Python, it can't be taken out almost ever.

03:55 So it's very hard to move fast and break things type of mentality with Python itself and the

04:02 standard library and so on.

04:04 But stuff outside of it can be much more rapid.

04:07 For example, there was a conversation with the core devs around making requests, the package

04:12 part of Python as a better way to do it because it's vastly a better way to do it than the built

04:17 in HTTP libraries.

04:18 They decided not to put requests into Python itself because they said it would actually

04:24 slow down and hamper requests and make it less valuable.

04:27 And it's better to leave it as its own standalone thing.

04:30 And I think that touches on a lot of what you're talking about, Sebastian, is that there's what's

04:36 built into Python and a lot of it is really good.

04:38 But oftentimes there's better things outside and they're not likely to be moved into Python

04:43 for their own good.

04:44 Yeah, exactly.

04:45 And since installing new packages or tools in Python is just one peep command away, it's

04:50 very easy to add new tools, experiment with them.

04:53 But at the same time, it's worth knowing which tools are kind of like the backed up by the

05:00 Python community, like which one are the most popular.

05:02 Right.

05:03 What are the de facto primary ways?

05:05 Like pytest, for example, would be in a case there.

05:07 Yeah.

05:07 Exactly.

05:08 Okay.

05:09 Yeah.

05:09 For example, I wanted to mention Cookie Cutter because if you're looking for a way to start

05:14 a new project, then you're wondering, no matter if it's like a Django website or if it's like

05:19 a Python module, a lot of people know about Cookie Cutter templates.

05:22 And they are a great way to start because you use a template that many people use before you.

05:27 It has a lot of same defaults already set up for you.

05:30 But if you never heard about Cookie Cutter templates, there is like no way you're gonna

05:35 randomly find it because like Cookie Cutter doesn't even have Python in its name.

05:40 So unless you know there is a tool for that, there is like no way for you to find it.

05:43 Right.

05:43 Exactly.

05:44 But it's so super helpful.

05:45 Okay.

05:46 Fantastic.

05:46 Well, we have a bunch of questions from the Ask Us Anything or Ask You Anything that we've

05:52 already gotten because we sent out an email and said, hey, send us your questions.

05:55 We're also, Norbert has a question here in the live chat.

05:59 Which would you like to go with first?

06:01 You want to do the live chat question?

06:02 You want to pick some out of our previous ones?

06:05 So we make sure we get to them since they asked first.

06:07 What are your thoughts?

06:07 So maybe go with the one that we got beforehand because there are some interesting questions

06:12 that we picked up and I think they will be useful to more people.

06:15 And then we will do the live chat.

06:16 Yeah.

06:17 All right.

06:17 That sounds great.

06:18 So folks out there, please keep putting your questions and follow up questions to what we're

06:22 about to talk about in the live chat and we'll get to them.

06:24 All right.

06:25 So the first one is what's the point of setting up something like pyenv using VENV and so on

06:31 when you can just use Docker?

06:33 Yeah.

06:34 So maybe a little bit of background just for people who are like not really sure what these,

06:38 all these things are and then the trade-offs.

06:40 Exactly.

06:41 So pyenv is a tool that you can use to manage different Python versions on your computer.

06:46 So for example, if you want to have Python 3.6, 3.7, 3.8, 3.9 install at the same time

06:52 and easily switch between them, you can use that.

06:55 And then VENV is a tool used to create virtual environments.

06:58 So it's a way to isolate the dependencies of your project.

07:03 So basically you would use pyenv and VENV to isolate Python and Python packages.

07:08 Right.

07:09 So pyenv is getting the version you want and then VENV is isolating that for a particular project, right?

07:15 Exactly.

07:15 Okay.

07:16 They work together.

07:17 And then the same thing could be solved with Docker where you have like Docker image that you use to spin Docker containers

07:24 and everything is isolated inside of it.

07:27 So you have a specific Python version that you choose based on which Docker image you use.

07:33 And then inside this Docker container, you install pip packages and they are isolated from both your computer

07:39 and other Docker images, Docker containers.

07:42 And so coming back to the question, I actually use both.

07:45 I use Docker in a lot of my projects and it's very convenient.

07:51 And especially if you want to later deploy your project or share it with your colleagues.

07:57 But at the same time, I don't want to spin up a Docker image if I just want to run a Python script.

08:03 So I still use pyenv to manage Python versions on my computer.

08:07 So I can use pyenv to install, to change the global version of Python on my computer.

08:14 And I saw that it's already useful for me because sometimes when I'm installing packages with Homebrew,

08:20 by the way, I'm using Mac.

08:22 So Homebrew is the package manager for Mac.

08:24 Yeah, I use Homebrew as well.

08:25 And I think that's also part of the modern pool chain is that you have something that updates your tools.

08:30 Yeah, exactly.

08:31 Automatically.

08:32 And then each time it's installing something, I see that it's trying to install some Python version in the background.

08:39 And at the beginning, I was worried because it was messing up the global Python versions.

08:44 But with pyenv, I have Python versions separated from my system.

08:49 So whatever Homebrew is messing up there, I don't really care.

08:52 I have pyenv that lives in a separate folder and it modifies the system path variables.

08:58 So these separate folders are being read before the system Python.

09:02 And that works in front of the Homebrew.

09:07 So I don't really care what the system Python on my computer.

09:10 By the way, Mac is still using 2.7.

09:14 I don't care what Homebrew is installing.

09:16 I have pyenv and that works really flawless.

09:19 But Docker is also really great because if you want to start a project using a different Python version, you just change one variable in the Docker file and that's it.

09:29 Yeah.

09:29 So there are many benefits to using that.

09:31 I think Docker is really useful and I do use it some of the time.

09:35 But it's not the first tool that I reach for because a lot of times I don't need that complexity.

09:42 You know, if I'm going to say fire up something like, oh, I need Redis just like this and I need Celery like that and I need MongoDB like set up like that.

09:50 And then I want to just make that whole package work together.

09:53 And I really would like it to be exactly on Ubuntu, which is the way I run things in production.

09:57 Then I would definitely reach for Docker.

09:59 But if I just want to just develop one of my web apps or some API I'm working on, chances are running it locally is totally good.

10:07 And it's just fewer things in the mix to worry about.

10:10 So to me, a virtual environment seems totally good for that.

10:13 To me, it's easy to see what everyone is super excited about and then go, oh, I have to do that as well.

10:18 And like that makes sense when there's enough complexity or you're in the same sort of situation that they're also excited about.

10:24 But it doesn't mean it applies everywhere.

10:26 Like if I just wanted to do basic Jupyter work on something that I could run on my machine, I wouldn't necessarily fire up Docker to make that happen.

10:32 Yeah, exactly.

10:33 I want to also point out that this question is specific about using Docker to replace Pi and virtual environments.

10:40 And it's not about using Docker in general.

10:42 I think we have another question coming up later about using Docker in general.

10:46 Yeah, yeah, exactly.

10:47 All right.

10:49 Another thing worth pointing out while we have this on the screen is it doesn't work exactly on Windows, right?

10:55 But there is ienv-win, W-I-N.

10:58 Yeah.

10:59 So the question is, what's the benefit of using that on Windows over, say, Conda?

11:04 And I feel like almost it's like a general question.

11:07 What's the advantages of something like Pi, ENV plus virtual environments and pip over Conda almost?

11:13 I would say if you're using Conda on Windows or whatever operating system and you are happy with it, I don't really see any reason to switch to something else.

11:23 I mean, Conda is like this one huge tool that you can use to manage both Python versions, dependencies, installing packages and stuff like that.

11:31 So I would say if you're happy, just stick with it.

11:34 But if you're not using Conda and for some reason you don't want to use it, but you're on Windows and you want to install different versions of Python, then Py and Win is a great tool to do that.

11:45 Because unlike Conda, it's very specific.

11:48 It's just for managing Python versions.

11:51 It doesn't do a lot of other things that Conda does.

11:54 But if you're looking for this specific type of tool, then I would stick with Py and Win.

11:59 This portion of Talk Python is brought to you by 45 Drives.

12:04 45 Drives offers the only enterprise data storage servers powered by open source.

12:10 They build their solutions with off-the-shelf hardware and use software-defined open source designs that are unmatched in price and flexibility.

12:19 The open source solutions 45 Drives uses are powerful, robust and completely supported from end to end.

12:25 And best of all, they come with zero software licensing fees and no vendor lock-in.

12:30 45 Drives offers servers ranging from 4 to 60 bays and can guide your organization through any sized data storage challenge.

12:37 Check out what they have to offer over at talkpython.fm/45drives.

12:42 If you get in touch with them and say you heard about their offer from us, you'll get a chance to win a custom front plate.

12:47 So visit talkpython.fm/45drives or just click the link in your podcast player.

12:53 I think one of the things that I would point out here is that the conda packages to some degree and certainly the official anaconda version, not initially the Honda Forge version.

13:07 So you'll see if you're going to get a little bit more than the other thing.

13:10 So you'll see if you're going to get a little bit more.

13:11 You'll see if you're going to get a little bit more.

13:12 You'll see if you're going to get a little bit more.

13:13 You'll see if you're going to get a little bit more.

13:14 You'll see if you're going to get a little bit more.

13:15 You'll see if you're going to get a little bit more.

13:17 And then eventually, you know, once it's been verified to be working well and all that, it gets brought into anaconda.

13:24 And generally, that's not a big deal.

13:26 But sometimes if you want to be on the absolute latest, there's a little bit of a delay.

13:30 And the place where absolute latest matters the most is if there's a security vulnerability in an API framework or a web framework.

13:38 You want to update if something like that were to come out, say, with Django, which has happened recently.

13:44 You would like to update to the new version immediately before anybody starts poking around on those things.

13:49 And things like Honda and these other more verified places sometimes lag behind.

13:55 And if it takes a little bit longer to get the latest notebook, UI or whatever, who cares?

14:00 But if there's an active exploit on the Internet against my thing, then you definitely are going to want the latest, latest, latest as soon as possible.

14:09 So I think that's one thing to consider if that's your world.

14:13 That's something I run into.

14:14 Yeah.

14:15 All right.

14:15 Let's take a couple of things out of the live stream comments just because they're follow-ups, right?

14:21 So one question is, what's the best homebrew equivalent for Windows?

14:25 I've got some thoughts, but go ahead, Sebastian.

14:27 Okay.

14:28 I'm not a Windows user.

14:30 I have no idea.

14:31 I've heard about Chocolaty, but I only heard about it.

14:35 Sorry.

14:36 Yeah.

14:36 So.

14:37 Yeah.

14:38 And Crested says Chocolaty and then Scoop and Winget.

14:42 I don't know about Scoop or Winget, but Chocolaty is very much like Homebrew.

14:47 And I'm familiar with that one.

14:48 And that one works pretty well.

14:49 So that's pretty cool.

14:51 But I definitely recommend if you're on Windows, check out Chocolaty.

14:54 If you're on Mac, check out Homebrew.

14:56 And then a lot of the Linux distributions already actually have this, right?

15:00 Like apt on Ubuntu and so on.

15:02 You don't need to go searching for those because that's the primary way.

15:05 Yeah.

15:05 And then finally, Mike Levin says, the best Homebrew equivalent for Windows is an Ubuntu

15:12 repo under WSL.

15:13 Okay.

15:13 That's an interesting one as well.

15:15 But yeah.

15:15 So you also have WSL, but then you have apt as well.

15:17 All right.

15:19 Cool.

15:19 Thank you everyone for that.

15:20 And we'll get back to some of the main questions as well.

15:23 So next one, Sebastian.

15:25 is about Poetry.

15:28 I've been using setup.py in my Python projects, but I see a lot of tutorials recommending other

15:33 tools like pipenv or Poetry.

15:35 Should you switch?

15:36 So basically, Poetry and pipenv can be used to manage projects.

15:42 So they are much more than just like a setup.py replacement.

15:46 Actually, pipenv is not a replacement for setup.py, but I will get to that in a moment.

15:51 So basically, those are like bigger tools that you can use to manage your

15:55 whole project.

15:56 You can use it to like add dependencies, to update dependencies.

15:59 They will manage the virtual environments for you.

16:02 So you don't have to activate anything and things like that.

16:05 And with Poetry, you can also publish packages to pip, which is basically a replacement for

16:11 setup.py.

16:12 You don't need to have setup.py with Poetry, which is kind of convenient because I always

16:17 have to Google out what I should put in the setup.py.

16:20 There is also a package called fleet if you want a lightweight replacement for setup.py,

16:25 but that's an off topic.

16:26 And if you like using setup.py, and again, you don't have a real reason to switch to something

16:34 else, I would say stick with setup.py because that's like the de facto way of doing things

16:41 in Python.

16:42 Setup.py is supported by Python itself.

16:46 There is also this project Toml that is now supported with like recent pip.

16:51 But for many years, setup.py was the way of building packages.

16:55 So I wouldn't just switch for the sake of switching.

16:59 But if you're looking for a tool that can manage your whole project, then I would use either

17:05 pip.env or Poetry.

17:06 And if you want to publish, then I would go with Poetry.

17:09 Yeah.

17:09 And I think you also want to consider, are you building a package or are you building an

17:14 application?

17:14 Because some of the things that you do differently, particularly around like pip.env, is a little

17:20 bit more targeted at building applications than it is packages, from my understanding.

17:24 Yeah, exactly.

17:26 And one thing to consider is that both Poetry and pip.env will be an external dependency for

17:32 your project.

17:32 So if for some reason they break or stop being maintained, then you're gonna have to migrate

17:39 to something else.

17:40 While if you still use setup.py and requirements files, then all the tools are there when you

17:47 install Python.

17:47 Yeah, that's a good point.

17:49 But also, yeah, it's like a bootstrapping step to get started.

17:52 So if you have setup.py, you don't need to install anything else to kind of like check

17:57 it out and try it.

17:58 Whereas, you know, if you have Poetry, the first thing you have to do with a new environment

18:04 is set up Poetry.

18:04 So then you can then initialize all the stuff, right?

18:07 Yeah, exactly.

18:08 I think I was recording an episode for the courses with Poetry.

18:12 And the first package that I wanted to add was the latest pytest.

18:16 And the first command I got was an error because Poetry already had an older version of

18:21 pytest pin down.

18:23 So I had to like open the config file and remove it.

18:26 But the first impression was kind of weird.

18:29 You try to add a package and it breaks.

18:31 But otherwise, I would say it's a good tool.

18:34 If you like it, use it.

18:35 But keep in mind that it's yet another dependency for you.

18:38 Okay.

18:39 Let me ask a follow-up from Incrusted on the live stream.

18:45 What is the best way, if any, for distributing Python applications to non-developers?

18:49 So this conversation of Poetry and PIP-EMV and setup, all that is for developers, right?

18:54 Yeah.

18:55 So what's the story on where's the icon thing I can put my doc or pin to my taskbar and let

19:01 people click?

19:02 What do I do there?

19:03 Yeah, that's a good question.

19:04 I also covered this in the course because I was also looking for a way that you can distribute

19:09 your Python application with people who don't know what Python is.

19:12 And there is this PyInstaller package that you can use to create an executable files that contains

19:19 all the dependencies.

19:21 And it works really great.

19:22 I was able to spin up a Windows machine, create an executable, then spin up another Windows machine

19:29 without even Python installed, move it there, and it was working fine.

19:32 It was using like a graphical user interface.

19:34 The only downside I saw is that they are not cross-compilers.

19:39 So if you want to build an application for Windows, you have to compile it on Windows.

19:44 If you want to build a Mac application, you have to do the same thing on Mac.

19:47 Otherwise, I would say PyInstaller is a really great way to build executable Python binaries.

19:53 Yeah, I agree.

19:54 And so does Patrick Lobiel.

19:55 Hey, Patrick.

19:57 Thanks for that.

19:57 Also add one thing really quick to this.

20:00 I really, really quickly, partly to show it as an example for people, and partly because

20:04 this is the thing that I do all the time.

20:06 I built the little thing, a little Mac menu bar app, 100% in Python, and distributed it as a

20:13 .app file, just like you all are talking about here.

20:17 So I created this thing.

20:18 It looks like this.

20:19 It does a few more things now.

20:20 But if you're copying some title and you want to turn it into a file name, or you want to

20:25 make sure you strip off stuff or clean out the text before you paste it somewhere else,

20:29 I built this little thing that goes in the menu bar, filled it with Python, and it runs

20:34 with PyInstaller, I believe.

20:36 No, that uses PyToApp, actually.

20:38 It's the one that it uses, I think.

20:42 So anyway, there's a couple of options.

20:44 But yeah, it's usually some variation of what you need to do on Windows and on Mac, and you

20:49 actually have to have a Windows machine and you have to have a macOS machine in order to

20:54 create those distributables.

20:55 So kind of a pain there.

20:57 By the way, if someone is wondering what's this PyToApp, because I was checking this as

21:01 well.

21:01 So there is PyInstaller that can be used for Windows, Mac, and Linux distributions.

21:06 And there is this PyToApp that is like a PyInstaller, but for creating Mac application.

21:12 And there's also PyToWin, which is also, again, like a subset of...

21:16 PyToExe, right?

21:17 Yeah, PyToExe.

21:18 Yeah.

21:18 That is like a subset of PyInstaller for Windows.

21:21 I feel like they might be more specific, like covering, you know, adding things like icons and

21:26 other stuff, maybe a little bit better.

21:27 I'm not sure.

21:28 For example, like this one, it has like an icon set that you create for it.

21:33 And the reason I chose PyToApp is this is based on Rumps, which is a ridiculously uncomplicated

21:40 like Mac notifications or something.

21:42 I don't remember what the URL, the acronym stands for, but it's 100% a Mac app.

21:47 So I figured I'd just use the Mac app builder thing for it because there's no chance it's

21:52 running on any other thing because the framework itself is only for Macs.

21:56 So yeah, anyway, let's go to the next question.

21:58 Here we go.

21:59 Which one should I choose?

22:01 pip, ENV, or Poetry, or something else?

22:04 I would say they are both very similar.

22:06 It's just that Poetry allows you to easily publish packages on PyPI.

22:11 So if you're building a Python package, I would go with Poetry.

22:15 If you're not building a Python package, then I think both should be fine.

22:19 They offer a similar set of functionality.

22:21 Okay.

22:22 Yeah, very cool.

22:23 Very cool.

22:24 Kondo versus pip.

22:25 We sort of touched on this a little bit, but maybe a little more.

22:28 Yeah.

22:29 So that's an interesting question because with Conda, you install binaries from the Conda

22:35 repository.

22:35 While with pip, you install packages from PyPI and they are not always binaries.

22:41 So the main difference is when there is no binary.

22:46 So if there is no binary, pip will try to build this package from sources.

22:50 And if you're missing some dependencies on your computer, then it's going to crash.

22:54 Right.

22:54 If you're on Windows and you see cannot find vcvars.bat, this is the problem.

22:59 Yeah.

23:00 If you're somewhere else and it just fails to install with something that looks like

23:03 a C++ compiler, that's the problem.

23:06 And that's part of the problem what Conda is trying to solve, right?

23:08 Exactly.

23:08 So with Conda, there is no such problem because every package that is published in the Conda

23:14 repository is a binary.

23:16 So it contains all the dependencies, which can be super helpful if you're not like, if you're

23:22 going to have a computer science background or if you're not very experienced with debugging,

23:26 there's missing whatever you said, Mike.

23:30 So let's say you, I don't know, if you're a data scientist and you just want to have your

23:36 code running, you don't care about setting up stuff, you just want to install some packages

23:41 and have it running.

23:42 Conda is an excellent choice because it has like a very good support for all those data

23:47 science packages.

23:48 You just install them, you have all the dependencies installed.

23:51 So it makes things much, much easier in that sense.

23:54 Yeah.

23:54 Especially in the data science world where there's really weird dependencies, like you need a Fortran

23:59 compiler, like I definitely don't have that on my machine.

24:02 Yeah.

24:02 On the other hand, as you already mentioned, Michael, those binaries don't always have the

24:06 latest version.

24:07 So someone first have to create this binary.

24:09 And if there is like a vulnerability, then you have a problem.

24:13 And there might be some like less popular packages that are available on PyPI, but no one created

24:19 a binary for Conda.

24:20 So either you have to wait until someone does that or you have to create it yourself.

24:25 Yeah.

24:25 Because with Conda, you can also install packages from pip, from PyPI, but you basically

24:31 have to build a binary yourself.

24:33 So again, if you're not like very experienced developer, you probably don't know how to do

24:38 this.

24:39 Yeah.

24:39 So I would say like the benefit of using pip is that you can install any package from PyPI

24:44 while with Conda you can't, but with Conda installing things is much easier.

24:49 Yeah.

24:49 I think that sounds like a pretty good summary.

24:51 It's worth pointing out that the workflow is quite different from a virtual environment

24:56 from Conda, right?

24:57 Like Conda, you create the environment off somewhere magical with the name and then you activate

25:00 it by saying the name.

25:01 Whereas pip, it's more, you explicitly say this one here in this location.

25:06 Yeah.

25:06 So Mike Levin has a chat stream.

25:08 I'll go through here.

25:10 Thanks, Mike.

25:11 So Conda was, he says only necessary, I'll say was more necessary before pip included

25:15 binaries, specifically with wheels.

25:17 And now that wheels, it doesn't happen as much pip can replace Anaconda now.

25:22 That's true.

25:23 But Frankie one says, yeah, but not everything has a cross platform wheel as well.

25:28 So I do agree with that.

25:29 For example, MicroWSGI, when I install that, it seems to like compile away for good 30 seconds

25:35 instead of coming with some binary version that can just drop in or so on.

25:40 So yeah, I agree, Mike, that that's generally true, that things are much better now that

25:46 we have wheels, but it's not also that it's not a hundred percent.

25:49 What do you think?

25:49 Yeah, I agree with that.

25:50 And one thing that I want to also add is that while we are talking about binaries and dependencies,

25:56 if you are, let's say, a web developer and you're installing Django or FastAPI, there

26:02 is like very little chance that you're going to be missing some dependencies.

26:06 So you probably will never have problems with pip.

26:10 So for some cases using pip, it's going to be perfectly fine for all your work life.

26:15 Yeah.

26:16 So it's the weirder, farther out it gets.

26:18 Like, like I said, Fortran dependencies with some weird C library or something.

26:23 And then also Joan Pino says, I've used PyInstaller for some side projects and it works really well.

26:30 Yeah, quite cool.

26:31 That's definitely a neat option.

26:32 All right, Sebastian, onto the next.

26:35 How do you hide secrets for development for production?

26:39 We've talked about, I think we haven't talked about yet, but there's, as in secret,

26:44 which is quite unnerving to see, should get going along here and finding all the secrets that people have put into public repos

26:52 or what were private repos, but then got turned public and people are pulling these out.

26:57 That's very much not good.

27:00 This is not what you want to do.

27:01 Just put the secrets right in your source code.

27:04 So what do you do?

27:05 What are your recommendations here?

27:06 So my recommendation is pretty standard to use the environment to hide the, well, the secrets and start them as environment variables.

27:15 And for example, you can have like a file with the YAML extension.

27:21 You can put it in the Git repository.

27:23 It's just, you don't put the secrets there.

27:26 You just put the structure and let's say dummy values or whatever.

27:29 So you remember that you have to set up all those environment variables.

27:33 Right.

27:33 That's one of the challenges because if you put it in environment variables or you put it into the, some like secrets template.yaml versus secrets.yaml,

27:41 where the secrets.yaml is not in the repository.

27:44 In a real project, there's going to be a bunch of stuff.

27:46 You got to have the names just right.

27:47 And you got to like knowing what you're even supposed to fill out for the app to work is a challenge.

27:51 So having this template around is super helpful, but not with the real secrets, right?

27:56 Yeah.

27:56 And there are some libraries that you can use that will make managing those templates much easier for you.

28:02 And why am I saying about those templates is that quite often when there is no template,

28:08 people start putting variables into the environment and they spin up a new server.

28:13 And they realize that they forgot about some environment variables and then things stop working.

28:19 So good practice is to have this like example environment variables file where you list all the variables that have to be set.

28:27 And then you have like a proper environment file that you never put to the GitHub repository where you actually specify the real secrets and then you load it to the environment.

28:37 Yeah.

28:38 And Crested has an interesting location.

28:40 Under a rock.

28:43 Under a rock.

28:44 Yeah.

28:44 Yeah.

28:44 No, but then more seriously points out that there are tools for keeping secrets in version control too, like certain vaults, right?

28:51 Like HashiCorp has something and we've had some sponsors with Talk Python and things where basically set up like the equivalent of one password or LastPass, but for your server, right?

29:02 Yeah, definitely.

29:02 Yeah.

29:02 Definitely.

29:02 If you have to be more careful about your secret, it makes sense to use an external tool.

29:08 But if a lot of those things can be mitigated by setting up the API keys properly.

29:13 For example, if you have like an API key for AWS, you just limit what this account can do.

29:21 And that way you don't spin up.

29:23 If it leaks, then you don't spin up 1000 servers mining Bitcoins for you.

29:27 And then you wake up with like $1 million debt or something like that.

29:30 So there are different ways to mitigate depends on what's the level of risk.

29:34 But the easiest thing is to use the environment.

29:36 Yeah.

29:36 And you can combine them as well, right?

29:38 Like you could have an encryption key in the environment variable and then actually encrypt the other elements in ways and then use instead of, because that way, if it's just in the environment, you could log in.

29:49 If you hack into the process, you could just go, well, OS.env, like what's in here?

29:53 Let's just look around.

29:54 And if it's all full of encrypted stuff, you're like, I don't have no idea how to unencrypt this.

29:58 There's layers, right?

29:59 And obviously running your app is lower privileges is really important.

30:03 Firewalls and isolate.

30:05 There's like layers.

30:06 This is not the only thing, but one of the things is not to put raw passwords and API keys in your source code.

30:12 Cool.

30:13 All right.

30:13 Let's talk Docker again.

30:14 Yeah.

30:15 So.

30:15 Should people use Docker?

30:17 What do you think?

30:17 More broadly.

30:18 Yeah, exactly.

30:19 This is a more broad question than the first one we had.

30:21 It depends who you ask.

30:22 Because there are some people who will tell you that you should always use Docker.

30:26 And when you start using Docker, you should spin up a Kubernetes cluster and then you finally will be able to deploy your almost static block there.

30:33 Yeah.

30:34 But don't listen to those people.

30:36 I mean, it really depends on what you're trying to do.

30:38 Because Docker has those amazing benefits that it makes deploying your application much, much easier.

30:45 So especially when you want to scale something up, Docker really pays off.

30:49 Because for example, let's say you're deploying your website for the first time and probably renting a virtual private server and then installing a Django website there.

30:58 It's easier than writing a Docker file.

31:07 It makes it much easier because you always use the same Docker file.

31:10 And spinning up 100 virtual private servers by hand is probably not something that you want to do.

31:16 There is a higher cost up front.

31:24 And another huge advantage of Docker is that it makes collaborating with people much easier.

31:33 So when I was joining a new team, there were like some setup scripts and you had to install some dependencies and you had to set up some environment variables.

31:40 And then after two days, you were finally able to start coding.

31:43 And now when I joined new projects that are using Docker, it's just two commands.

31:48 Docker build and Docker run.

31:50 And you have this whole work, all development environment running.

31:53 You have the website there.

31:54 You have database.

31:55 You have the ready server, whatever.

31:57 So it makes joining a new team much, much easier.

32:01 On the other hand, let's say if you have like a Python script or like one off script that you want to, I don't know, scrape some data from the internet, there is like absolutely no point in adding yourself more work to write a Docker file if you know that you're going to be throwing away this code like tomorrow.

32:19 So it depends on what's your use case.

32:22 If it's for like simple things, I wouldn't bother unless you really like using Docker.

32:26 But for like more advanced projects, it will probably pay off.

32:30 Yeah, I was speaking with one of the Python listeners having a, what do you think about this and how do we solve this problem type of conversation?

32:38 And it was basically, how do I make sure that everyone on my team has exactly the same version of Python, right?

32:47 I want to make sure that they're using 3.9.2, not 3.9.3, not 3.9.1.

32:54 You definitely don't want one because that has that floating point vulnerability.

32:57 But you know what I mean?

32:58 Like you don't want variation.

33:00 You want exactly that thing.

33:01 And if you really care down to that degree, you know, Docker is great because then you just make everyone run the same Docker image.

33:08 You provide it in some, I don't know, local Docker hub type thing, or just put the Docker file into version control.

33:16 And if it changes, then it just rebuilds and it runs and off you go, right?

33:19 Yeah.

33:20 Yeah.

33:20 So it helps a lot for that.

33:21 But I totally agree.

33:22 I wouldn't use it for like super simple things.

33:25 You're just causing yourself more challenges when often it's not really that useful or that needed, I guess is the way to put it.

33:33 So there's some interesting follow-ups here in the live stream.

33:36 In addition to under a rock, Frankie one says, I thought GitHub now automatically detects when you commit secrets accidentally.

33:43 I think I did hear that it takes an attempt.

33:46 I wouldn't count on GitHub catching all of my secrets.

33:49 I would be more deliberate about it.

33:51 Do you know anything about this?

33:52 Not really.

33:53 I know that there are those bots that check like for the vulnerabilities and stuff like that.

33:57 So I would imagine that it also has a bot to detect secrets in the commits.

34:02 But I'm just wondering how it works.

34:03 Because when you push something to GitHub, it's automatically available there.

34:07 So does GitHub put some blocker on your repo to not show the latest commit?

34:14 Because usually what I see people doing is that they commit, they push a commit with a secret and they realize they push that.

34:21 So they push another commit removing the secret.

34:23 But there are like bots running through GitHub harvesting those.

34:27 So the moment you push it, it's already distributed to some people or to shigit.

34:32 It's too late.

34:33 It's too late.

34:35 The stream has already been observed.

34:37 It's really not good.

34:39 Related to this sort of follow on, I think related to the Docker a little bit, maybe in some ways.

34:45 So THP says, if you're not an expert, Pythonista, how do you manage supply chain, full chain attacks appropriately?

34:53 You want to maybe give people an idea what supply chain vulnerabilities are and then we could talk about this?

34:59 I don't know what our supply chain vulnerabilities.

35:01 I'll give you the run.

35:03 So for example, recently there were about 4,000 malicious packages that were put into PyPI.

35:08 Then there was another vulnerability where if you're running a private PyPI server so that you can have like your team can publish and share packages across applications.

35:17 And then often those will like fall through to the real PyPI.

35:21 So I could just say pip install requests and it'll get the one from PyPI.

35:25 If I say pip install data layer, it'll get our local data layer.

35:28 Right.

35:29 One of the vulnerabilities was if somebody finds a way to publish your private name thing, but to PyPI with a higher version number, pip would prefer that one.

35:39 And go give that.

35:41 But that might be one that is some sort of vulnerability or something along those lines.

35:45 Right.

35:45 So this may be more like more concerning is in PHP.

35:50 Somebody put in some code to run that if you put Xerodium, I believe, as a header.

35:57 I think it was in the headers or no, it was in the user agent.

36:00 If you put the word Xerodium and then some PHP code in the user agent and request it against a PHP site, which 79% of the Internet runs on, I think a lot.

36:11 It would actually just run that arbitrary code.

36:14 So those are the types of things I think that we're talking about here.

36:16 That's a very good question.

36:17 And to be honest, I don't really know how you can mitigate those kind of attacks because like, yeah, the package name typos can happen.

36:28 And there is probably not much you can do.

36:31 The only thing that comes to my mind is to use like a, if you don't know how to solve those problems, it's probably worth paying someone to take care of that.

36:40 So I don't know, maybe using Heroku would be a bit more helpful.

36:45 I don't know how they can mitigate this stuff.

36:48 I don't know.

36:49 What about you, Michael?

36:50 Yeah, I got a couple of ideas.

36:51 None of them are perfect.

36:53 You can pin your versions.

36:54 So you can go and look at all the libraries you're using and say, okay, I trust them as they are now.

37:00 And if you pin the version and your requirements.txt or your pyproject.tml or whatever, you're not going to automatically get the latest version, which may have some kind of vulnerability snuck into it.

37:11 So you can consciously decide to move to the next version.

37:15 Hopefully, you know, if somebody were to hack into Django, maybe that gets out, but it wouldn't last very long.

37:21 Right.

37:22 And so if you don't automatically upgrade on just the next deploy, then you're going to at least be a little bit better off.

37:28 So pin your versions, probably one.

37:30 You could whitelist.

37:31 You can do things like set up a private PyPI server and just whitelist packages so people can't arbitrarily install various things, but they only install ones you approve.

37:42 You can use this thing.

37:43 You can use this as the sneak package advisor.

37:47 So if I want to learn about FastAPI, like what's the story of FastAPI?

37:51 This is cool because it's not just security, but it's more than just security.

37:54 Gives you like the health, the maintainability level or how much it's maintained, whether it has any known security issues, whether community is active, whether it's influential, all sorts of cool graphs about who's working on it, when it's been worked on, amount of funding, security analysis by sneak.

38:13 And so on.

38:14 So this is a pretty good thing.

38:15 So together, right?

38:17 So if I were to say I'm going to pin my versions and then those versions are going to be verified or with or maybe I'm going to create a whitelist and only allow you install from the whitelist.

38:26 And I'll make sure that everything that gets through the whitelist at least has been checked out by sneak.

38:30 Not perfect, but it's better than just pip install random thing and hope for the best.

38:35 So yeah, anyway, let's see.

38:37 And Crested also follows on with is vendoring dependencies an appropriate mitigation?

38:43 Like instead of pip installing requests, like finding the key bits of requests and just copying that code into your application.

38:50 It certainly would help.

38:52 On the downside, if a non-intended vulnerability were to get into requests, not a hack, but just something that was a ultimately became a security hole.

39:01 You've now vendored in that instead of automatically getting the upgrade, right?

39:04 I don't know.

39:05 What do you think?

39:06 Yeah, I think it's easy to do this because you can basically put the virtual environment folder with your project and that's it.

39:13 In the long run, it's probably harder to maintain.

39:15 And I think, as you said, like hitting the dependency versions, making sure you don't have typos probably can save you from a lot of those problems.

39:25 Yeah, be very careful when you pip install something that you get the spelling right.

39:28 Because there was one of those vulnerable ones.

39:31 It was something like asteroids versus asteroid.

39:34 I mean, it was like plural versus singular and that was it.

39:37 Right?

39:38 It's very subtle.

39:39 Yeah.

39:39 And I guess if you're using like more popular packages, then probably should be safe.

39:44 I would say like with some more obscure packages, it can maybe happen that they will get abandoned and someone will either hack the Git repository or like squat it.

39:54 I don't know if it's actually possible with PyPI, but with bigger packages, it's probably not an issue.

39:59 Yeah, for sure.

40:00 I think it's a relative thing.

40:02 So let's see.

40:04 PyLang, hey PyLang, also says you can develop in a VM or Docker.

40:08 And related to that, Doug Farrell.

40:11 Hey, Doug.

40:11 He says, really interested in how to set up a Docker-based development environment.

40:15 So one of the things I think you want to distinguish here in this supply chain thing is I'm trying something out.

40:23 If it's going to run that setup high when I pip install it, it could pip install virus onto my machine.

40:29 If that's your concern, you can develop in a virtual machine.

40:32 You can develop in Docker.

40:33 Because then all it sees is what's inside the Docker machine, which is nothing, basically.

40:38 So that's really cool.

40:39 So that's really cool.

40:40 But also, as Encrusted points out, you know, that if this is developing your application and pushing it to production, that vulnerability comes along onside the web server.

40:51 So that's a challenge, right?

40:53 So in this exploratory world, I think things like Docker actually help a lot solve this problem.

40:58 But in a production world, it does nothing.

41:01 Nothing to help.

41:02 But maybe let's follow on real quick with that.

41:05 I think this development environment based on Docker is pretty interesting.

41:09 For example, PyCharm now lets you set up a Docker image as the run target, which you can even debug into.

41:16 And so you just press run, and it starts the Docker thing.

41:19 It attaches to it.

41:20 You just run it there.

41:22 I think VS Code also has something along these lines.

41:25 What do you think about this?

41:26 Yeah, because I'm using VS Code.

41:28 I was trying to set it up with Docker some time ago, and it was possible to also put the breakpoints there and maybe run, let's say, a Django application in Docker and have breakpoints work there.

41:38 It just requires a bit of setup.

41:41 But I would guess that either by now or soon in the future, the guys from the VS Code will further simplify it.

41:50 Because they just keep adding things to make stuff easier.

41:53 Yeah.

41:53 So it's definitely a viable option.

41:55 Yeah, that sounds good.

41:57 Here is the thing for PyCharm, right?

42:01 I basically configure an interpreter using Docker.

42:04 And just press go, and then boom, off it runs in Docker, which is, I think, pretty awesome.

42:11 Talk Python To Me is partially supported by our training courses.

42:14 Do you want to learn Python, but you can't bear to subscribe to yet another service?

42:19 At Talk Python Training, we hate subscriptions too.

42:22 That's why our course bundle gives you full access to the entire library of courses for one fair price.

42:28 That's right.

42:29 With the course bundle, you save 70% off the full price of our courses, and you own them all forever.

42:35 That includes courses published at the time of the purchase, as well as courses released within about a year of the bundle.

42:41 So stop subscribing and start learning at talkpython.fm/everything.

42:49 Yeah, some comments about this basically being so much about dependencies and stuff.

42:53 But I do think a lot of it really is, it's like, how do you get the right libraries?

42:57 How do you keep those libraries up to date?

42:59 How do you do that securely?

43:00 How do you deploy those things out to the world?

43:02 How do you share that with the user?

43:03 So much of this stuff is a pretty interesting thing to talk about.

43:06 One follow-up here from Doug is, are there resources or links that would help me and everyone understand how to build a Django debuggable Docker environment?

43:14 I think actually, the thing that I have on the screen here, if you're into PyCharm, is one I'll throw that into the live chat here.

43:21 I mean, that is, click these three buttons and then press the debug button and it runs in Docker.

43:27 Do you have something like this for VS Code?

43:29 So what I did, I checked the documentation.

43:31 I think VS Code was also using Django as an example in the documentation.

43:36 So you would have to dig in their docs, but they're pretty easy to follow.

43:40 From the top of my head, I don't know anything.

43:43 Yeah, okay.

43:44 So let's see.

43:45 I think Incrusted has a pretty interesting comment, question here.

43:49 What's the story around code formatters, linters, and type checkers?

43:54 Autocomplete got the better of them there, but no problem.

43:56 Could you be more specific?

43:58 Because I'm not sure.

43:59 Well, yeah, I'm guessing like, did you use black?

44:03 Did you use mypy?

44:05 Things like that, I would guess.

44:07 But yeah, Incrusted, give us a follow-up if you want.

44:10 So I will start and you can clarify the question.

44:12 So I would say definitely use some of those tools.

44:14 Definitely, I would recommend using black.

44:18 Because some people don't like it.

44:21 It's opinionated and you might not like how it formats some code, even though you can actually

44:26 put comments to disable formatting if you want to like preserve your beautifully form, a dictionary

44:32 of list of tuples of dictionaries and stuff like that.

44:35 But if everyone on your team is using code formatter, no matter if it's black or if it's something

44:40 else, then at least during the code reviews, you don't have some silly discussions about

44:45 code styles.

44:46 And I had those discussions in the past and they were terrible because code reviews should

44:51 focus on what's happening in the code, not how the code looks like.

44:55 So definitely some kind of formatter is very, very useful.

44:59 Black is the most popular one.

45:01 It works out of the box.

45:02 So I would suggest using that.

45:04 And speaking of linters, I really like Flake 8, but there's also PyLint.

45:11 And you can actually use both of them.

45:13 And even though I have like, I don't know, eight years, nine years of Python, Flake 8 still

45:20 finds some silly bugs that I'm doing in the code.

45:22 And because like sometimes you're tired, you may like assign to an undefined variables and

45:28 stuff like that.

45:28 So instead of like waiting for your code to actually run and give you this error because

45:33 there is like no compilation, so you can't get the errors beforehand, you immediately can

45:39 get those easy to spot errors for free.

45:42 So definitely spend some time adding them to your code editor.

45:46 And then you can keep going crazy because like Flake 8 has plenty of plugins that you can

45:51 further extend it.

45:52 You have a bunch of other static analysis tools.

45:55 I'm listing them in the course.

45:57 I think there is like a website with the resources for the course that you can find some cool plugins.

46:01 And there is stuff like, for example, Sorcery that I really like that gives you some

46:06 recommendations of how to reformat your code.

46:08 For example, when you're writing a for loop and it's basically building a list, you will

46:12 get a recommendation to use a list comprehension showing you how this list comprehension will

46:17 look like.

46:17 So that's yet another tool that...

46:19 Yeah.

46:20 One of the things about Sorcery that annoyed me quite a bit is in the early days, it would

46:25 recommend all these refactorings.

46:27 And like nine out of 10, I'm like, yeah, that's a great refactoring.

46:30 Other times like, no, this is a horrible refactoring.

46:32 Like, I don't want this change.

46:34 And I know the pattern I'm using is better than this, even though you are your program to think

46:39 this is it.

46:39 And you couldn't disable notification.

46:42 So I just have this permanent mark of a warning on my screen for certain bits of code.

46:48 They recently added a way to add for you to disable that on a per project basis.

46:52 So now I'm also all about Sorcery.

46:54 Now that I can turn off the one or two refactorings that I'm not a huge fan of, and then the rest

46:59 are really nice, like you say.

47:00 Yeah, I didn't have this experience with Sorcery.

47:03 I like it.

47:03 But as I said, there are like plenty of tools, so you can check them out.

47:07 Each of them works in a slightly different way.

47:09 You can combine them.

47:10 You can disable some stuff if you don't like.

47:12 For example, I didn't like piling because it was too strict.

47:15 Yeah.

47:15 Like when you had a class that had too little, too few methods, it was saying like, well,

47:20 this shouldn't be class and stuff like that.

47:22 Right, but maybe you're putting it there because it's going to get bigger in a month as the

47:26 thing grows, but you're like putting that flexibility in now, and it's going to tell you, no, you

47:32 shouldn't have it, right, which is not necessarily the case.

47:34 Exactly.

47:35 I mean, it's trying to make you write the most beautiful code.

47:38 It's just sometimes I prefer my ugly code.

47:41 Yeah.

47:42 Well, yeah, for sure.

47:43 And so a couple other follow-ups.

47:45 Patrick Leveal says, I love using black and flake 8 with pre-commit.

47:48 Very cool.

47:50 The thought, Doug had the same thought as I did, that what he really likes about IDs

47:53 like PyCharm and VS Code and the tools and plugins is that they basically do this while

47:58 you're writing code.

47:59 And to me, like I don't run flake 8 separately.

48:02 I just use PyCharm with all its settings turned on, and it's super obvious if something's wrong.

48:07 And actually what I did when I redid all my stuff with Sorcery, I went through like the

48:12 20,000 lines of Python code that are Talk Python training, and I went and accepted every refactoring

48:18 that Sorcery was giving me.

48:20 Except for that one that I don't like.

48:22 And then while I was going through, I found there's a few PyCharm warnings that I had been

48:27 ignoring.

48:27 They were fine, but they're like, you really should do this other thing.

48:29 And I just fixed everything.

48:31 Got it pristine and perfect.

48:33 And so now I really pay attention to those warnings because there's no like leftover junk warnings.

48:39 And I think that's actually a really big deal about a lot of this.

48:41 And who was it over here?

48:43 Someone, I don't see the comment, maybe I'll find it, but was asking basically, how hard

48:48 do you enforce this?

48:49 Yeah, here you go.

48:50 And Chris, it says, my Py and others should be hard enforced.

48:54 How strictly should you adhere to those?

48:55 Like, for example, do you break continuous integration if some linter fails or something

49:00 like that?

49:01 Yeah, that's a very good question.

49:02 I think it's a matter of how the rest of the teams like it.

49:06 I work with some teams that were like very reluctant to use any of those tools.

49:11 I work with some team leaders that said that CI has to work.

49:16 Your commit has to pass all the CI checks.

49:19 So it's a matter of preference.

49:22 I mean, tools are there to help you, not to hinder you.

49:25 So if some tool doesn't work well for you, you should kind of configure it or maybe replace

49:30 it.

49:30 Sure.

49:31 So I know one question that we had kind of queued up and some people maybe a little bit

49:37 touched on it, but we haven't really talked about the language hardly at all, have we?

49:41 Like using language features and stuff.

49:43 So what do you think about things like Hypen's, like this kind of stuff?

49:47 Yeah, that's another sensitive topic.

49:49 I would say they can be definitely useful.

49:52 I mean, if you, especially if you have like a lot of code, some legacy code, being able

49:58 to immediately see what are the argument types that the given function takes, what's the return

50:03 value, what's the type of the return value a function returns in your code editor, it's

50:09 super useful.

50:09 Yeah.

50:10 And even though with Python, we have duck typing, so we can get quite far with this.

50:16 There can be some subtle bugs that type hints can help you.

50:20 But the thing with type hints is, well, my problem with type hints is the same problem

50:27 as with test and documentation.

50:29 It's not strictly a problem, but it's a yet another thing that you have to take care of.

50:33 I'm not saying I don't like writing type hints, but for example, let's say you're writing code

50:37 and then you have to add tests.

50:39 You have to add documentation and you have to add type hints and then you code changes and

50:43 you have to update all three places.

50:45 Because it's not only about updating tests and documentation, you also have to make sure

50:49 that your type hints are updated because you have wrong type hints and they are useless.

50:54 Yeah.

50:54 And of course there are tools like mypy, but they are external tools.

50:58 So you have to configure them.

50:59 And at the beginning, you'll probably have a lot of warnings and stuff like that.

51:03 So I would say that type hints will give you back as much love as you give them.

51:08 So if you spend time making sure that the type hints are properly declared, that you're not

51:13 using any in all around your code, then they are going to be very, very useful.

51:19 Yeah.

51:19 I absolutely love the type hints, but I think you should remain Pythonic as you think about

51:26 using them.

51:26 So there's, I think there's two blends is one is do you want to go down the mypy, have the

51:31 checkers completely verify everything is absolutely hanging together like C# or C++

51:36 compilers do like it's a hundred percent.

51:39 This is an integer and everywhere it's an integer.

51:42 So you like lay all that out.

51:43 That's one way of using these.

51:45 The other way, which is the way that I like is put them on the boundaries, right?

51:49 If I'm creating a package and I want to put that package on PyPI, it would be awesome if

51:54 the public API of that package all had type hints or type annotations, because then when

52:00 I use it, I know automatically my editor knows automatically, am I doing the right thing or the

52:04 wrong thing without having to constantly dive into the documentation?

52:07 Is this a string or a class that contains the information that I'm supposed to pass?

52:12 for location, right?

52:14 Is that a quote Portland comma Oregon or is it a location object?

52:19 But those two things said or a dictionary or what the heck goes in here, right?

52:22 Like those kinds of answers just go away instantly.

52:24 And when you're using it wrong, your editors will like put some kind of squiggly or something

52:28 and say, no, no, you're using it wrong.

52:30 And I think that's the biggest value of type hints is like right on those boundaries where

52:35 you're maybe unfamiliar with it or something.

52:37 It'll really communicate that across.

52:39 Yeah.

52:39 I wouldn't use type hints on like throwaway code, but I usually try to start to add them

52:46 as my project grows bigger.

52:47 So for like small project, it's easy to kind of understand what's going on, but as it keeps

52:51 growing, it really pays off to start adding type hints.

52:55 Yeah, for sure.

52:56 All right.

52:58 Getting short on our time here, but I know there's a couple of people who got in right at the beginning

53:02 and we kind of skipped over it a little bit just because we were getting to the pre-questions

53:07 first.

53:07 But Patrick Lobiel has one that I think is worth touching about.

53:10 What do you think that good Python GUI development, why do you think they're rare, especially for

53:14 beginners, you know, real UI is way more motivating than terminal output?

53:18 I agree that it is way more motivating.

53:20 And I also agree that we're quite short on good UI development, GUI development options.

53:26 Sebastian?

53:27 Yeah.

53:27 Same opinion.

53:28 I was never using a lot of GUI, so I'm not very familiar how many tutorials are there, but

53:34 I was building one for the course and I decided to use TK-Inter because it comes with Python.

53:41 And there weren't that many tutorials, you're right.

53:44 I found one that was very useful because it was showing a lot of different components of

53:49 TK-Inter.

53:49 But apart from that, I would say there weren't many showing you how to build some specific

53:57 thing, like a specific project or something like that.

54:02 Yeah.

54:02 I just dream of the days when we got something like VB6 or Windows Forms was in .NET where

54:08 you just have a nice UI thing.

54:10 Oh, here's a button.

54:11 Here's a text box.

54:12 You put it over there.

54:12 You want code to run, you double click it.

54:14 You write the three lines of code.

54:15 You go back to working.

54:16 Like you push a button, you get a thing that you distribute to people's applications.

54:20 Like I don't really understand why we don't have something like that in Python where it's

54:25 sort of a visual layout, push a button.

54:28 It does something, PyInstaller magic, PyToApp magic.

54:31 Here's your app.

54:32 Give it to someone.

54:33 Like that would be transformative and we just, we don't have it.

54:36 I think it's a bit of a chicken and egg thing, but one can dream, one can dream.

54:40 All right.

54:40 Norbert Patrick also talks about some enterprise development stuff.

54:45 What's your take on things like Azure Active Directory and security topics like DB encryption,

54:50 major cloud platform deployments and so on.

54:53 Bit of an open-ended one, but what are your thoughts generally?

54:55 I have almost no experience with Azure.

55:00 And yeah, as you said, it's kind of a broad topic.

55:03 So don't really know what to say here without like a specific question.

55:08 I'll throw a few things out there.

55:09 One, Azure Active Directory seems pretty interesting to me.

55:13 One of the things that's an ultra pain is federated identity or trusting other people's identity.

55:18 Like if I want to let all of your users single sign on at your place and then single sign on

55:23 on my site, like that's not an easy thing to do right now.

55:25 And I think Azure Active Directory makes that quite easy.

55:28 I haven't done a lot with it, but in that use case, I think it's pretty useful.

55:32 I honestly don't do much with Active Directory at all, like corporate accounts and like building public web apps and things like that.

55:38 So that they don't make sense in that world.

55:40 Database encryption.

55:41 Very interesting.

55:43 There's a lot of levels.

55:44 Is this data encrypted at rest?

55:46 Is it encrypted in memory?

55:47 Is it encrypted in the response?

55:49 A lot of the modern databases have like encrypted columns.

55:52 For example, MongoDB has different encrypted columns you can have.

55:56 And then certain clients, as they talk to it, they either may or may not be permitted to decrypt that column.

56:02 So it could return all the columns or values without overexposing the data potentially.

56:07 I don't do anything like that, but also encrypting backups, right?

56:10 Like, what do you think about, I got to take a backup of the database.

56:14 Now what?

56:14 This is like the worst possible thing I could lose is the backup to the database, right?

56:20 It's the only thing worse is letting people get access to the live database.

56:23 I don't got thoughts on that.

56:25 Like, do you have any recommendations?

56:26 I can tell people what I've been doing lately, but it's not perfect.

56:29 No, I will leave this question to you.

56:31 So I've been using like an encrypted volumes.

56:35 Like in Mac, you can create virtual encrypted volumes that you can just open and close.

56:39 Like highly, highly encrypted.

56:41 And so, but it appears as a drive.

56:43 So I've been doing my backups lately straight to that thing and then closing the encrypted vault

56:49 when I'm not actively backing up a database.

56:51 So then I put that somewhere safe.

56:54 But if for some reason something were to happen to it, at least what they get hold of is an encrypted

57:00 vault with a huge password that they probably can't mess with.

57:03 So I don't know.

57:04 Lots to think about.

57:06 Lots to be up late at night worrying about.

57:08 Yeah.

57:09 And then Vincent throws out there, we have VB style and Anvil.

57:12 And yes, Anvil was certainly on my mind.

57:14 I didn't quite call it out, but.

57:15 I forgot about it, but then I remember them from some conferences and yeah.

57:19 Anvil is really interesting in that it has that.

57:21 It's the thing that that's different here is what you get is Anvil is super neat, but you

57:26 do get a specific type of web application.

57:29 But if that specific type of web application works for you, then they do have this VB style,

57:35 drag and drop, double click, run your code, which is absolutely beautiful.

57:39 But if I wanted to build Todoist or one password in Python, there's no real way to do that nicely.

57:48 I think one nice thing about Anvil is that it actually generates a nice code that is readable.

57:54 Because the one problem I have with this, what you see is what you get text editors is that I remember them from the time when you were building websites like that.

58:02 And the resulting HTML was an abomination.

58:06 You just couldn't modify it by hand anymore.

58:08 So I'm worried that if we get tools like that for Python, it's also going to result in like unreadable code.

58:15 But I know that Anvil actually generates a pretty nice code.

58:18 Yeah, I remember some of the tools you would look at the, here's the auto generated code.

58:24 And it was like, basically, there's giant comments.

58:26 Don't touch this.

58:27 Don't try to read this.

58:28 Just leave it alone.

58:29 Any attempt to mess with it will probably just break everything.

58:32 Don't touch it.

58:34 So Doug also thinks Anvil is awesome.

58:37 Yeah, very cool.

58:38 And it used to be it had to run on the Anvil cloud.

58:41 I believe they've open sourced their runtime so you can like self host it and stuff.

58:44 That's worth checking out.

58:46 And then, I don't know, maybe, maybe this is the last, last comment.

58:50 I'd say maybe one more out there.

58:52 We're throwing out something out there.

58:53 But Kristen says, are there any courses out there that teach you how to get started with a lot of what we're talking about without any background knowledge?

59:01 I certainly will give another shout out to your course, Sebastian, Modern Python Projects over at Talk Python Training.

59:07 That is definitely a great course that goes deep into all these things, like nine hours of conversation of like, here's my screen.

59:15 Let's go do these things.

59:16 Is it with no background knowledge?

59:18 I mean, you do have to know Python, right, to do this.

59:21 So basically, what are the assumptions for your course?

59:23 You have to know Python.

59:25 And that's it.

59:26 I show you a lot of those tools, but I always start from scratch.

59:30 Like, I even had like empty macOS accounts.

59:34 So I made sure that I don't miss some dependencies and stuff like that.

59:38 I also like spin up a Windows VM for some parts that require Windows.

59:42 So I think you should be good with just knowing Python.

59:46 Yeah.

59:46 Okay.

59:47 A couple of real quick ones here.

59:49 Doug says, oh, man, yes.

59:51 Visual Python would be amazing.

59:52 I like Visual Basic.

59:53 I agree.

59:54 And then THP says, Fman build system is a pretty cool.

59:58 Fman is a pretty cool UI in the Fman build system for building those things up and distributed.

01:00:04 And that's pretty interesting.

01:00:05 I haven't done anything with Fman.

01:00:06 Have you?

01:00:06 No.

01:00:07 First time I heard the name.

01:00:08 Yeah.

01:00:08 It's like a file management tool, I believe, in Python.

01:00:11 The person who created it then also built this build system and this UI on top of it.

01:00:16 So pretty interesting.

01:00:16 Looks cool.

01:00:17 Let's see.

01:00:18 There's another one from Joan.

01:00:20 Is Quart and its async capabilities ready to level up Flask webmaps for medium-sized enterprises

01:00:25 or currently better to stick with Flask?

01:00:27 I have no experience with Quart.

01:00:29 So I don't know.

01:00:30 What about you, Michael?

01:00:30 None.

01:00:31 I mean, I interviewed the guy who did it and I've played around with it and it seems okay.

01:00:36 But at enterprise, like it's got to be five nines level of up.

01:00:40 I've never run anything like that.

01:00:41 Honestly, if I was really, really looking to build a web app with async capabilities,

01:00:46 I would either look at FastAPI or Starlet.

01:00:49 Like I know people might think, oh, with FastAPI, that's an API thing, not a web thing.

01:00:54 But to be honest, you know, it can be, definitely can be done.

01:00:58 And I actually built a whole class on how to take FastAPI and sort of replace Flask with it.

01:01:04 So it's not that hard.

01:01:05 It supports templates, supports static files, all those kinds of things.

01:01:08 It is super cool.

01:01:09 And yeah, thanks Norbert for the shout out on the 10 apps course.

01:01:13 That's really a fun one.

01:01:14 But yeah, so I think actually if I was trying to go all in on async, I feel like FastAPI is one of the best async frameworks out there.

01:01:23 It's just, it all holds together really, really well.

01:01:26 So, all right.

01:01:27 Yeah, I think that might be it.

01:01:29 Sebastian, you want to just maybe give us some wrap up thoughts here on this whole idea?

01:01:34 Well, thank you for coming.

01:01:35 And thank you for those questions.

01:01:36 They were really great.

01:01:37 And I hope we answered all of them.

01:01:41 Yeah, I think, you know, there's so much variety.

01:01:44 I've worked in different technologies over the years.

01:01:47 I've been doing development like 25 years or so.

01:01:49 And some of the time you'll find some language or some ecosystem where there's sort of a central,

01:01:56 like this is how you do things, like specifically in the Microsoft space with .NET.

01:02:00 Here's the one web framework you use.

01:02:03 Here's the one database ORM you use to talk to the one database you use.

01:02:07 And while that's really helpful, now I know what to do.

01:02:11 I do these four things and I'm good.

01:02:12 This is the recommended way.

01:02:13 In Python, we don't have that.

01:02:15 And that's absolutely a blessing, but it's something of a curse as well in that there's

01:02:21 this paradox of choice, right?

01:02:22 Oh, it's amazing.

01:02:24 We have this cool ORM library.

01:02:25 Oh, wait, there's 10.

01:02:26 Well, if there's 10 ORM libraries, which one should I choose?

01:02:29 And we were having this conversation.

01:02:30 Should we use Flask?

01:02:31 Should we use FastAPI?

01:02:32 Should we use Court on top of Flask?

01:02:34 Should we use Django?

01:02:35 Like you can just go on and on and on about all the trade-offs that you have to make.

01:02:40 So I think one of the big challenges is choosing a path and then going down it.

01:02:46 Choosing, say, I'm going to use Poetry, Flask, Microwisgee, let's go, or something like that.

01:02:52 Whatever your branch that you happen to follow.

01:02:54 I think that's a big challenge.

01:02:56 And maybe I'll throw out one more thing for people out there on the live stream, if I can

01:03:01 pull it up quick enough, is awesome-python.com.

01:03:04 Awesome-python.com.

01:03:07 This kind of helps at least narrow that list, right?

01:03:10 So for example, if I'm interested in caching, like here are probably the eight most popular

01:03:17 caching libraries that you might use or something like that.

01:03:20 It's not perfect, but at least it gives you some way to explore when you're totally new.

01:03:24 Because I think that that's one of the big challenges here, Sebastian, is not that there

01:03:28 are not enough choices, but there are so many choices at each step of the way that that's

01:03:33 a challenge, right?

01:03:34 There's the t-shirt, I learned Python in a weekend.

01:03:37 It was, I learned Python.

01:03:38 It was a great weekend, right?

01:03:40 Like that's funny and also kind of true at the language level.

01:03:42 But I don't know about how you feel, Sebastian.

01:03:44 I'm still learning Python after so many years and spending all day in it.

01:03:48 Yeah, same here and like tools come and go.

01:03:50 I bet in like five years, this list will be completely different than it is now.

01:03:55 So it's definitely important to spend some time trying to figure out which tool you should

01:04:01 use.

01:04:01 Because as you say, with Python, you can mix and match different tools.

01:04:05 And sometimes there are like no tutorials explaining how to do things.

01:04:09 Sometimes the tool might get popular, but then it's abandoned and it's no longer maintained.

01:04:15 So yeah, I mean, a lot of tools to choose from.

01:04:18 Yeah, I guess we'll leave with that.

01:04:20 There's a lot of tools to choose from and we've covered a bunch of options and a lot of trade

01:04:24 ops here.

01:04:24 So Sebastian, it's been great to chat with you about that.

01:04:27 Thanks for coming on the live stream.

01:04:28 Thank you.

01:04:28 Likewise.

01:04:29 It was great to talk with you.

01:04:30 Yeah, you as well.

01:04:31 And thank you everyone for all the questions, those who emailed them in previously and the

01:04:35 live ones.

01:04:36 It was a great conversation.

01:04:37 See you around next time.

01:04:38 Bye guys.

01:04:39 This has been another episode of Talk Python to Me.

01:04:42 Our guest in this episode was Sebastian Witwowski.

01:04:45 And it's been brought to you by 45 drives and us over at Talk Python training.

01:04:49 Solve your storage challenges with hardware powered by open source.

01:04:54 Check out 45 drives storage servers at talkpython.fm/45 drives and skip the vendor lock-in

01:05:01 and software licensing fees.

01:05:02 Want to level up your Python?

01:05:04 We have one of the largest catalogs of Python video courses over at Talk Python.

01:05:08 Our content ranges from true beginners to deeply advanced topics like memory and async.

01:05:13 And best of all, there's not a subscription in sight.

01:05:16 Check it out for yourself at training.talkpython.fm.

01:05:19 Be sure to subscribe to the show.

01:05:21 Open your favorite podcast app and search for Python.

01:05:23 We should be right at the top.

01:05:25 You can also find the iTunes feed at /itunes, the Google Play feed at /play,

01:05:30 and the direct RSS feed at /rss on talkpython.fm.

01:05:34 We're live streaming most of our recordings these days.

01:05:37 If you want to be part of the show and have your comments featured on the air,

01:05:41 be sure to subscribe to our YouTube channel at talkpython.fm/youtube.

01:05:46 This is your host, Michael Kennedy.

01:05:47 Thanks so much for listening.

01:05:48 I really appreciate it.

01:05:50 Now get out there and write some Python code.

01:05:52 I'll see you next time.

01:06:12 Thank you.