#431: Visualizing CPython Release Process Transcript
00:00 Every year Python has a new major release. This year it's Python 3.12 and it'll come out on October 2nd, 2023. That's four days from when this episode was published. There is quite a process involved to test build and ship Python across the many platforms and channels. We have Seth Michael Larson here to give us a detailed rundown on exactly what is involved in releasing CPython.
00:22 This is Talk Python to Me, episode 431, recorded September 28th, 2023.
00:27 Welcome to Talk Python to Me, a weekly podcast on Python. This is your host, Michael Kennedy.
00:44 Follow me on Mastodon, where I'm @mkennedy, and follow the podcast using @talkpython, both on fosstodon.org. Be careful with impersonating accounts on other instances, there are many.
00:55 Keep up with the show and listen to over seven years of past episodes at talkpython.fm.
01:00 We've started streaming most of our episodes live on YouTube. Subscribe to our YouTube channel over at talkpython.fm/youtube to get notified about upcoming shows and be part of that episode.
01:11 This episode is brought to you by JetBrains, who encourage you to get work done with PyCharm.
01:19 Download your free trial of PyCharm professional at talkbython.fm/done-with-pycharm.
01:25 Hey there, before we jump into talking about building CPython, I have a new course to tell you about HTML plus Django full web apps hold the JavaScript. This is an excellent new Django course written by Christopher Trudeau. In just a few hours, you'll learn how to make your app much more interactive in the code even a little cleaner than pure Django.
01:47 You may be familiar with our HTMX plus Flask course I released a while ago, and this one is its Django sister course. And it is out now at talkpython.fm/htmx-django. The link is in your show notes. If you Django and have been wanting to adopt HTMX, you should definitely give this course a look. And a heartfelt thank you to everyone who has purchased one of our courses or library bundles. It truly makes this show possible. Now on to that interview.
02:16 Hey all.
02:16 Hey Seth.
02:17 Hello.
02:18 Awesome to have you here on the show.
02:20 Yeah.
02:21 Thanks for having me.
02:21 Absolutely.
02:22 I'm looking forward to talking about the details, the steps of releasing CPython.
02:28 And for those of you who have not been watching and paying attention by the time this comes out, it'll probably have already happened.
02:35 I'm sure it's already happened by the time this video is up, though, it's not quite happening.
02:40 So as of the recording next Monday, this whole process that we're going to going to talk about goes into action, right?
02:46 Releasing Python 3.11.
02:48 Yeah.
02:48 This, or not 3.12.
02:50 This will have happened.
02:51 3.12.
02:52 I got to get used to it.
02:53 It's like the, you know, in January, all you always write the, the, the year wrong out, so 3.12, the new version, the still faster, faster CPython 3.12 with lots of cool features is going to be out and how does that happen?
03:07 Right?
03:08 There's, there's a lot going on behind the scenes and you wrote a really cool blog post called visualizing the CPython release process or something like that.
03:16 I'm getting off memory and has a great diagram with a whole bunch of boxes.
03:20 And so we're going to go through and kind of just talk about how that happens and so on, but before we get into it, maybe a quick intro for folks who don't know you yet.
03:27 Yeah.
03:28 So I'm, I'm Seth Larson, online everywhere.
03:32 I am Seth M Larson.
03:34 That's my handle.
03:34 And I am the security developer in residence at the Python software foundation, the inaugural security developer in residence.
03:41 but hopeful for more, obviously.
03:43 But yeah, this whole role is funded through the OpenSSF Alpha Omega project specifically, and it's kind of a project that's focusing on Alpha being really, really important projects that everyone agrees, yep, that's a really important project to secure.
03:59 So among them is Python and PyPI.
04:01 And then Omega is kind of like the long tail, right?
04:04 Like all the other projects, 'cause there's open source is huge, there's millions of different projects.
04:09 How can we secure those?
04:11 And so, yeah, this role exists because of them.
04:13 So I'm very thankful.
04:14 And it's mostly-- - As are we all, right?
04:16 Speaking for the community.
04:17 This has been something-- - Yeah.
04:19 - There's been a lot of places where it's like, well, why doesn't someone take care of this?
04:22 Like, well, there's nobody's whose sole job it is.
04:25 But like Python has been slowly rolling in these new roles, which is fantastic.
04:29 Like Lukas Lange at the developer in residence, you're the security developer in residence.
04:33 We have Mike Fiedler, who I spoke to about on PyPI security.
04:37 Like there's people, there's developers working on stuff.
04:40 awesome.
04:40 There's something to be said, like having having someone working on stuff full time, like you can, you can get a lot done.
04:46 It's, it's, it's quite nice.
04:48 Like, cause we're so used to, you know, how, how the speed of open source, right.
04:52 Which is not, I guess when someone says the speed of open source, they don't immediately think, oh yeah, that's going to be, that's going to be fast and snappy because people have lives.
04:59 Yeah.
05:00 Well, sometimes it is when, when things are brand new, they, they, they can have a lot of momentum.
05:05 But I know my personal view is like on all these software projects, there's these little gnarly edges that is just as fun for no one.
05:12 Right.
05:13 And like, I want to add this new feature or I want to add async to this thing so people can do this and like, you can get so inspired, even if it's not your job and it's just your project and really go on it, but like, you know, really polishing out that little bit of documentation or, or updating that detour, like it's, it's all the, the fine polish, I guess, that I think you all are bringing to Python, which is awesome.
05:34 Yeah.
05:35 there's tons of work that's like, it really does need that amount of time investment and like a sustained time investment.
05:42 So like when you're working with external groups or like, you know, developing processes and stuff like that, like you just need, you just need to be there a lot more than, you know, weekends.
05:52 So it's good to have that full time presence.
05:55 Yeah.
05:56 Yeah.
05:56 I think it is too, although it's absolutely mind blowing how much stuff gets done in traditional pure open source.
06:02 It really is.
06:03 Honestly, it gives a whole new perspective to it, right?
06:06 -It's like, okay, you know, all of this is happening, -Yeah.
06:09 but, you know, almost all of the people that I interact with on a day-to-day basis are volunteers.
06:15 And so, like, I kind of wrote about this a little bit in my opening when I announced myself as a security developer in Resonance, but it's like, you need to, you can't have expectations and, like, assign people work, you know what I mean?
06:28 Like, this is not a traditional work environment where you can just tell people what to do And, you know, it's a certain way of working.
06:36 And I'm used to it because I am an open source maintainer myself.
06:40 So it makes a lot of sense to me and I feel very comfortable with it.
06:43 But it's very different than a regular security developer job would be.
06:46 Would you say fewer TPS reports?
06:48 Yeah, a few fewer. Yeah, yeah.
06:51 Just a couple fewer.
06:53 Hey, since you brought it up, maybe we'll come back to this if we have time at the end.
06:57 But, you know, you've got some pretty notable projects that you're working on here.
07:01 Like give a quick shout out to some of your open source work that predates.
07:03 Yeah.
07:04 So I am the lead maintainer of your lib three, which is one of the most downloaded Python packages on PyPI.
07:11 I also help maintain requests.
07:13 It's a user of your lib three.
07:15 And then in addition to that, I have a library, which I quite love, which is called truststore, which is being adopted by a lot of package managers right now, like pip and conda and and PDM, which allows those packages to use system certificates as opposed to something like certify, so that you can take advantage of all of the benefits that you get for using a system trust store for HTTPS, as opposed to like a static bundle of certificates, because operating systems are actually constantly checking, updating all of these things.
07:47 It's just a little bit better.
07:48 >> Some certificate store or some certificate authority turns out to be shady and they get revoked and...
07:53 Yeah, or just not have the best practices.
07:58 And so, yeah, just like, okay, we're going to pull that one.
08:01 But pulling a certain...
08:03 Yeah, they even get stolen, right?
08:05 Sometimes these certificates get stolen.
08:07 They get stolen, you know, there's all sorts of things that can go wrong.
08:10 Thankfully, it doesn't go wrong very often.
08:12 Otherwise, it would be a lot bigger of a problem.
08:14 But yeah, it does happen every once in a while.
08:17 Yeah, that's cool.
08:17 Yeah, and you were on Python Bytes, my other podcast I do with Brian, to talk about Trust Store, which is cool.
08:23 All right, but back to this.
08:24 So tell us a bit about this developer, security developer in residence role.
08:29 Like people were like, Oh, this maybe they've heard of it or is maybe even new to them.
08:33 But like, what is your, what is your job here in the PSF space?
08:37 Yeah, it's, it's, it's kind of funny talking about my role too, because people see security through so many different lenses.
08:44 And this is even like outside of talking to people online that are in the open source space.
08:49 So whenever I talk about this job, people will say, Oh, so you do it security for the PSF? I'm like, Well, no, I don't do that. And then they'll say, Oh, so you secure? Yeah, right? I'm in.
09:00 Right? No, no, no. So I don't do that. I don't do it security for the PSF. And then they the next question they usually ask is like, Oh, so you make it so that Python is more secure, like the code of Python is more secure. And I'm like, not really, honestly, I don't do that either.
09:17 And what I really do is try my best to make it so that bad stuff doesn't happen to Python users.
09:25 And so, you know, obviously those things are important, like securing Python, the actual code itself. But on, like, the scale factor, maybe those are a little bit less important than some things like making sure that when you download something from Python.org, it is the correct thing. Making sure that when a release is happening for Python, nothing can subvert that release and get injected in and then distributed to everyone. And then it goes beyond Python when you start thinking about the packaging space. So making sure that pip release process is good. Making sure that all of these tools and stuff that are using all of these binary libraries, how can you be sure that those libraries that are bundled along with them don't have vulnerabilities. And then a lot of things around process. So vulnerability management and making sure that vulnerabilities that are discovered in Python and reported to us, they actually get to the end of the process where they're fixed and released.
10:24 There's just a whole bunch of things like that where it's more of a scale and safety and making sure that my effort goes towards things that are going to keep on churning as opposed to things that are like spot fixes. Because spot fixes, they're important, but they don't have the scale and they don't have that, like, keep on giving you the value aspect of them.
10:46 Yeah.
10:47 And the consequence of a lot of what you described sounds like supply chain type issues.
10:51 The consequence of that is so bad.
10:54 It doesn't matter how patched your computer is or how good your firewall is.
10:59 If CPython itself ships with a virus, right?
11:02 Right.
11:03 Yeah.
11:04 Or somebody snuck it in right at the builds.
11:05 Yeah.
11:06 Yeah.
11:07 Right.
11:08 Right.
11:09 Nobody wants that.
11:10 Well, almost nobody wants that.
11:11 - Very few people want this, so.
11:13 - Those are not good people.
11:14 We don't want them to want this.
11:15 - Yeah, we don't take them into consideration for their use cases.
11:20 - We sure don't.
11:21 Awesome.
11:22 And let's see, I believe you're on, well, when I read the article, you're on week nine.
11:26 So you're on week 10.
11:27 Is that where we are?
11:28 - I think I'm on week 12 at this point.
11:30 - Oh man, time flies.
11:31 - Yeah, I know.
11:32 I'm actually drafting up like a quarterly review and it's a big thing, so yeah.
11:37 - Okay.
11:38 A quick question from the audience here on the live stream, you know, Lewis asks, is there a visibility or will there be a public visibility for CVEs in packages?
11:47 That is an excellent question.
11:49 It is a good question.
11:50 And I bring this up because didn't the PSF just get certified as like a CVE authority or whatever that is like ability to create CVEs.
11:57 I don't know.
11:58 Yeah, I can, I can talk so much about this, right?
12:00 So there, there's a whole bunch of stuff happening in that space.
12:03 So the, the, I'll start off with stuff that I did not personally do, but I'm helping maintain now.
12:08 which is there is an advisory database called, if you go on GitHub, it's github.com/pypa/advisory-database with a dash in between advisory and database.
12:22 And that database is essentially trying to categorize all of the CVEs and what versions they affect for every single package on the Python package index, which is an impossible task, but so we're trying our best there.
12:38 But yeah, so one of the packages, no problem.
12:40 No problem, right? Like that's a manual task right there.
12:43 Yeah, no. So what this actually does is for the bulk of it, what it will do is it'll go out and it'll pull the CVE feed from NBD and Miter, and it will parse it and try to find references to Python packages and versions inside of CVEs.
12:59 So obviously that's sometimes there's false positives, but a lot of the times there's just, it just works out okay.
13:05 And there's a little bit of triage involved.
13:07 There's a little bit of manual submission of this too.
13:09 But yeah, so tools like pip audit, which is a tool that I love so much, uses this database in addition to other scanning tools.
13:19 Like any scanning tool can use this database.
13:22 But yeah, this is the canonical location for vulnerabilities affecting Python packages.
13:27 So for Python itself, the Python Software Foundation just recently, like a few weeks ago announced that we are now a CVE numbering authority, which means that we are, we kind of have like a scope of just Python and pip right now. So those are the only two projects that we're emitting CVEs for. And what that means we can do is it means that people have to submit reports to us, which is good, because then they can't submit them to other places and get CVS that way can be kind of frustrating, because in the past, sometimes that would happen. And then And we wouldn't know about the vulnerability until it got published as a CVE.
14:02 And we'd be like, what just happened?
14:03 >> We have a CVE?
14:04 Like, what?
14:05 >> Yeah, right.
14:06 >> Like, one of the big deals is you're supposed to be told, in an ideal world, you're supposed to be given like a, come knock on the back door, like, that WebP code.
14:13 We need to talk about the WebP code before.
14:16 >> Yeah.
14:17 Oh, gosh.
14:18 Don't get me started on WebP.
14:19 Don't get me started on WebP.
14:22 >> This portion of Talk Python to Me is brought to you by JetBrains and PyCharm.
14:26 Are you a data scientist or a web developer looking to take your projects to the next level?
14:31 Well, I have the perfect tool for you.
14:33 PyCharm.
14:34 PyCharm is a powerful integrated development environment that empowers developers and data scientists like us to write clean and efficient code with ease.
14:42 Whether you're analyzing complex data sets or building dynamic web applications, PyCharm has got you covered.
14:49 With its intuitive interface and robust features, you can boost your productivity and bring your ideas to life faster than ever before.
14:56 For data scientists, PyCharm offers seamless integration with popular libraries like NumPy, Pandas, and Matplotlib.
15:02 You can explore, visualize, and manipulate data effortlessly, unlocking valuable insights with just a few lines of code.
15:10 And for us web developers, PyCharm provides a rich set of tools to streamline your workflow.
15:14 From intelligent code completion to advanced debugging capabilities, PyCharm helps you you write clean, scalable code that powers stunning web applications.
15:23 Plus, PyCharm's support for popular frameworks like Django, FastAPI, and React make it a breeze to build and deploy your web projects.
15:32 It's time to say goodbye to tedious configuration and hello to rapid development.
15:36 But wait, there's more.
15:38 With PyCharm, you get even more advanced features like remote development, database integration, and version control, ensuring your projects stay organized and secure.
15:47 So whether you're diving into data science or shaping the future of the web, PyCharm is your go-to tool.
15:52 Join me and try PyCharm today.
15:54 Just visit talkpython.fm/done-with-pycharm, links in your show notes, and experience the power of PyCharm firsthand for three months free.
16:05 PyCharm, it's how I get work done.
16:07 - The real, like the ideal process is someone would Google like Python and security or like Python and report a vulnerability.
16:18 And then they would find these like really lovely instructions that someone else wrote up to say, just send an email to security@python.org with whatever information you have.
16:27 And then we triage that.
16:28 And if we decided it's a vulnerability, we would ask for a CVE on your behalf and like credit you and all this stuff and then fix it.
16:36 And then the release and publication of the CVE would happen at the same time.
16:40 So that's what we call coordinated vulnerability disclosure.
16:43 And so instead of like, oh, there's a vulnerability, but there's no fix available for that vulnerability.
16:47 Like that's what you want to avoid because it causes panic.
16:52 And in theory, it could cause exposure, right?
16:55 Where there's some dwell time in between when people, or it doesn't minimize the dwell time between when people are able to patch and when the vulnerability is known to potential attackers.
17:05 - Right, there's already a latency from when it's announced to somebody actually fixing it.
17:10 But if when you hear it, you can't actually fix it for two more weeks 'cause it's not released.
17:14 That just gives people two weeks to try to dissect like what exactly do they mean by there's this image problem?
17:20 Exactly. Yeah. Oh, yeah. Yeah We're gonna start talking about libwebp actually. Yeah, the no, i'm just using it like it's the most recent example because last night there was a And really relevant to cvs because there was some drama that apple patched Some of their code when google announced it It looked like it was only applied to chrome but it actually applied to anything that used WebP, which looking at the updates is like everything on my computer needs to be updated because of it. You know, like that, that mismatch was like caused a lot of drama. I definitely agree. Yeah. There was like a week of time in between, when that CVE just mentioned Chrome and Apple. And then when it actually ended up mentioning LibWebP and then right when it mentioned LibWebP, I'm like, Oh, I know a lot of Python packages use And so I then went out and go and talk to those people.
18:08 Okay.
18:09 And they fixed those things now.
18:10 So, yeah, that's great.
18:11 Another good reason to stay on top of dependency management and stuff, right?
18:16 Like tools.
18:17 One of the things I think we're coming to sort of understand and manage better in the Python space is not your dependencies, but the transitive closure of your dependencies.
18:27 Right.
18:28 So I might use, let me just say, I guess, what if I use pillow to work with images, images, but Pillow itself uses some other library that it pip installs that itself might have some kind of image code.
18:39 If I say I use Pillow, oh, I need to update it.
18:41 pip install --upgrade Pillow doesn't upgrade its, its dependencies and those dependencies of the right.
18:49 So like, but that could be where a lot of silence issues live.
18:53 And so things like Poetry, Flit, pip-tools, there's a lot of tools coming on that I think are gaining popularity that treat not just what you directly wanted a pip install, but it's friends and it's dependencies.
19:04 - Yeah, definitely.
19:05 - What's your thought about that from a security perspective?
19:07 - Yes, so having the most important thing from like a kind of dependency management side of things is having those lock files and having them have hashes.
19:18 Like it's less of a problem now.
19:21 Like it likely won't come to fruition that those hashes end up being important, but luckily lots of lock file formats do generate hashes anyways, which is great.
19:29 So that subverts attacks against PyPI partially.
19:32 - Right in the middle, DNS madness.
19:36 - Right, yeah.
19:36 Like if somehow you've been tricked into using a bad certificate or something, right?
19:42 Like it saves you from a lot of stuff like that.
19:44 So having hashes, they're basically free.
19:47 They're great, just use those.
19:49 The other side of it is running a vulnerability checker against those dependencies.
19:54 'Cause then you'll, or either that or having a tool that's doing that for you.
19:57 So like Dependabot is a tool that does that for you.
20:00 It will check vulnerability databases and then give you like a security notification or a PR to say, hey, this is affected by a medium severity vulnerability or you can run them your own via pip audit with it can either check an already installed environment or it can check like a requirements file.
20:18 I don't know the full extent of all the different file types that it can check.
20:21 I know that it works on requirements.txt, but yeah, it'll check those files for known vulnerabilities all of the transitive dependencies, just everything that's available. And then from there, it'll tell you, hey, these are the things that you need to fix here, the fixed versions, like try upgrading to at least this doesn't mean that it's easy. It does go back to that advisory database you talked about? It does go back to that advisory database. Yeah. And it also can hook into osv.dev, which is kind of like this global aggregation of tons of different vulnerability databases.
20:54 So for example, we have one for Python that I pointed out, but then like Rust has one, Debian has one, what is it?
21:01 Rocky Linux, I think has one, like there's just a whole bunch of them.
21:05 And you can kind of interface with this database using the same API.
21:08 And it's kind of nice.
21:09 - Yeah.
21:10 So making this the security show, we just talked to Mike Fiedler a little while ago about that as well, diving into it.
21:17 I mean, we'll come back to the release process, but I do want to ask you about this is One of the challenges I see is like, I got to depend about a warning.
21:24 Luckily, it was for basically the requirements-dev side, not the tool requirement.
21:31 You know, like I'm going to use a bunch more tools to do like Jupyter stuff on my data, but I'm really just running a website and there's like a much smaller, well, a smaller set of things on there.
21:40 But there was a vulnerability report for something, some package that was in my requirements for the dev verge, the dev setup.
21:49 However, when I tried to go update it, one other package that is the reason that was there said, no, it has to be less than something.
21:57 And that something was less than the fix.
22:00 And I'm like, how do I square these two things?
22:02 Like I can pip install it, but then it keeps complaining that you're, you're running the wrong version.
22:07 Like, no, I'm not.
22:08 The other one has our remote CVE in it, you know, a remote code execution in it.
22:12 And I'm like, I really don't want that.
22:14 I mean, not a server.
22:15 So it's probably fine, but I really don't want that.
22:17 No, I'm I don't care if it's slightly more unstable. I want the new one. Give it to me.
22:22 Right. And that's a bit of a challenge. I think when you say like, I've got 50 dependencies and I've got to somehow square, I need to get a brand new one right away. What are your thoughts on that? Yeah, this this is a super tough problem. And I think it's one that open source will be coming to reckon with even more now. Right. Because we're kind of in a place now where there's a few really important libraries which are kind of moving something that is very extensively and widely used to be not maintained anymore. For example, OpenSSL. OpenSSL 1, or 1.1.1, which is extremely widely used, is no longer maintained. There's going to be no more security patches for that. And it's going to be tough, because my guess is that there's also just a lot of software that's not built for OpenSSL 3 being the OpenSSL of choice, right? Luckily, Python is not in that situation, but it's certainly a thing that other pieces of software might have to deal with. In terms of Python packaging, this is kind of why the guidance on don't specify an upper bound unless you know that that upper bound makes sense. And so, backwards incompatibility, Yeah, maybe you want to say, okay, if this is version 2.1.whatever, and it's semper, you can say, okay, anything less than three is probably fine.
23:44 But even that, right, if that package goes unmaintained.
23:47 >> Yeah, this went from two to three in its main version, right?
23:50 It's major version, so.
23:51 >> Right.
23:52 Yeah, so this is a difficulty.
23:53 >> There's a lot of packages.
23:54 Yeah, there's a lot of packages that set hard upper bounds.
23:57 And I just, I don't know why, because maybe it's going to break it, but I feel like it's speculative.
24:05 we should probably not let a major version run. But you know that that leads into trouble, right?
24:09 Like, even if you just want to use some other library that says greater than or equal to three, and this one says less than three, like, great. Now, what do I do?
24:15 Yeah, I think that the toughest thing is that you can't go back and then like amend it.
24:19 Yeah, that's the hardest part is because so I will speak very, very highly of all of these, anything that can make pip automatically get the right thing is just magical is so good. And so People like that and I totally get why people are doing these upper bound sorts of things but yeah it does have the issues that you that you talked about but yeah there's something to say about how great the those bound checks are I wish there was a way that you could add something like some sort of guidance to pip to say after the fact like okay I said this was open but now I'm gonna go ahead and tell you that like this definitely doesn't work with this version maybe that's something that can get recommended but yeah it's just so nice when when it does work out.
25:04 Yeah.
25:05 I, I can relate to Frank here who writes with 1.3 thousand or 1,300 dependencies in our project because of this dependencies are hell at all times.
25:15 Yeah.
25:16 I have on Talk Python Training, the courses website, I think there's 248 packages if you look at what's installed and then the dependencies.
25:24 And then you try to throw in the data science-y stuff.
25:26 And it's just like, there's a 50, 50 chance that I can actually install everything because like half the time, something has got a less than something's got a greater than, and I'm like, Oh my God.
25:34 You add, as you upgrade, right?
25:36 Like at any point in time, it's deterministic, but you know, over at, over the evolution of it.
25:41 Right.
25:41 Yeah, no, it's, it's tough.
25:42 It's tough when you start getting to that number, like there's just so many intertangled things.
25:47 There are.
25:47 And like, the reason I bring this up now is it's, it's a hassle when like, I want to use this library and that library and somehow I've got to like juggle things.
25:54 So they work, but it's concerning when you get a dependable alert that says there's a remote code execution, you're going to need to deal with it and your dependencies won't let you upgrade.
26:04 You know what I mean?
26:05 Like that's, that's why I bring this up.
26:06 It's definitely concerning.
26:08 Yeah.
26:08 Another thing that you can do, and this is, I feel like they don't talk about this as much with Dependabot is that there are ways to mitigate vulnerabilities.
26:16 That doesn't necessarily have to come from upgrading.
26:19 Like I'm always going to say upgrading your dependency is the best way to go forward because then you know that there's no chance of at least that vulnerability in particular being exploited.
26:29 But if it's something like remote code execution, but it's this one component.
26:32 Right.
26:33 As long as you're not using that component, then you're fine.
26:35 Or if it's remote code execution, when you get some sort of input, whatever, right, that's, that's malformed or whatever.
26:42 And you're like, okay, well that component's not exposed to the network.
26:46 It's I only use it internally.
26:48 So there is something to say.
26:50 Not even a server.
26:51 It doesn't open a port.
26:52 Right.
26:52 It's just a thing I run on my computer.
26:54 It's like, it probably doesn't matter.
26:56 Exactly.
26:56 Yeah.
26:57 And that's, that's kind of the one typing into it.
26:59 And it's like, but if it's in your website, then all of a sudden it gets a little trickier.
27:03 Right.
27:03 Then it gets dicier.
27:04 Yeah, definitely.
27:05 Yeah.
27:05 The stakes are higher when you attach the internet to anything, but when it's like a local script, it's just cail your log file and his request, request, hack, hack, request, attempted hack requests.
27:16 It's just WP log in.
27:17 Yes.
27:17 Yeah.
27:18 WP admin, all them things.
27:20 Yeah.
27:20 It's usually, luckily it's WP something or something most of the time.
27:25 If you're safe against 90%, we're all good.
27:27 As long as you don't return a 200 to any of those, like your traffic will remain low.
27:31 - Yeah, absolutely.
27:32 (laughing)
27:34 All right, so that's what you're doing, focusing on these kinds of things.
27:37 That's really, really awesome.
27:38 And thanks for giving us an even more detailed look than I expected into it.
27:43 (laughing)
27:44 Yeah, let's talk about your, let's talk about, you know, the timely thing.
27:49 I mean, security's always timely, but the CPython, let me see if I get this right, 3.12 release coming up here on Monday.
27:55 - Dang, he's got it.
27:56 - Amazing.
27:57 I got it, man.
27:58 I'm ready.
27:58 I'm ready for this new world.
28:00 So there is a PEP and let's see who put this together.
28:03 This is Barry Warsaw and Guido and it is PEP 101.
28:07 I love it.
28:08 (laughs)
28:09 Pep 101.
28:10 - The one you need, right?
28:11 - Yeah, what a perfect number.
28:12 But it's the title is doing Python releases 101.
28:17 That's kind of a circular, I love it.
28:19 And it just talks about all the steps and it's kind of a manual of now you do this, Now you do that, but it's just a really long doc.
28:28 I love that.
28:29 Stop, stop, stop, stop.
28:31 There's like, there's like a all caps all across.
28:35 Stop, stop, stop, stop, stop.
28:35 Did you do this right?
28:37 Right.
28:37 Yep.
28:38 That you can tell, you can always tell when there were problems in the past in a checklist, right?
28:43 Like if there ever was something that happened incorrectly, it's, it's very obvious.
28:47 And so yeah, the stop, stop, stop.
28:49 I feel that that I, I don't personally know why that's there, but I do laugh every time I see it.
28:55 So, yeah.
28:56 let's see.
28:58 Please note how PEP 101 replaces PEP 102 time traveling was invented there.
29:03 What is it?
29:03 Yeah.
29:04 At the very top.
29:04 It's I wanted to comment on that too.
29:06 It said replaces PEP 102.
29:08 And I wonder if it was more like a, Oh, we should just use 101 sort of situation.
29:14 Well, I think this is doing micro releases, right?
29:17 So this one, maybe more like full, like the full deal.
29:20 I don't know.
29:21 Yeah.
29:21 I think security releases are, they're slightly different.
29:24 We don't actually build binary installers for security releases.
29:28 We only build source builds, like source tarballs for micro releases.
29:33 And so maybe that was the distinction in 102.
29:36 - Sure.
29:37 Okay, so this has existed since 2001, but you got to really dig into it.
29:43 And you know, there's some interesting stuff there, but picture a thousand words, all these things, you put together a really nice graphic here.
29:50 And somehow you got this in like HTML, I'm impressed.
29:54 Yeah, that's embedded SVG.
29:55 I, I always use draw.io for my, my diagrams.
30:00 I take pride in my diagrams.
30:02 So yeah, it looks good.
30:03 All right.
30:04 So there are 10 steps.
30:06 Some of the steps have ABCs in them.
30:08 So, and you know, just for people who pull up this picture and obviously being a podcast, it's hard to see the picture.
30:16 If you're not watching the live stream, I will put a link to the article, which has the picture in there and by remember, right.
30:24 I'll even put it as the chapter art.
30:25 We'll see about that.
30:26 but there's a legend in here.
30:28 So you've got like different types of things.
30:30 You've got human actors, which are golden release artifact, which is blue, a source of risk, which is where you're paying extra attention and the start and end state.
30:40 And so, the colors mean stuff here and the legends at the bottom, but yeah, let's, you know, start with number one.
30:46 Now what happens here?
30:47 Yeah.
30:47 So starting with number one, and I'm actually going to increase the size of my screen as well, because I'm going to pull it up separately myself.
30:54 Going on here.
30:57 Yeah.
30:58 A lot going on here.
30:59 You need some, you need some commandor control plus on it.
31:01 Yep.
31:02 Okay.
31:03 All right.
31:03 So in the very beginning, what happens is essentially the release manager decides it's time to make a release.
31:10 It'll be live right now.
31:12 So we're, we're deciding, okay, we're going to make a release.
31:15 And so something that happens even before this is we kind of talk amongst all of the other.
31:21 It says release managers in there, like windows release manager, macOS release manager.
31:24 They kind of flip between that language and experts.
31:27 So like windows expert, macOS expert.
31:30 Some people understand this weird windows things that can make a MSI installer.
31:34 The people, what do you do on Mac?
31:36 I don't even know how to build that right.
31:37 That person.
31:38 Yeah.
31:38 Yeah.
31:39 Talk to, talk to them, get, get their, get their buy-in that indeed they're going to be around when you decide to do this because this is kind of like a big coordinated thing. Everyone has to be around to do their part. And then if that is the case, then the release branch gets frozen. So in this case, that release branch would be 312. And so that's done by essentially putting a block on all future like pushes to the branch like in GitHub, use like branch protection. And that's basically to make it so that no one can merge pull requests anymore so that you have something that's stable that you can kind of work off of.
32:14 And then after that, you'd move on to step two, which is the actual release manager will update their fork of the repo with whatever is on that branch.
32:23 So like the three 12 branch will get updated and then you'll pull that branch locally now on step three.
32:30 So now you're on the actual release managers machine.
32:33 And so like, I kind of have like gray boxes drawn kind of throughout.
32:37 So those are kind of like security boundaries a little bit.
32:40 The top most one is the GitHub organization of Python.
32:45 And then the middle, the tiny one that has the fork repo, that's the GitHub user for the release manager in particular.
32:52 And then the furthest left, the gigantic one, is the release manager's actual computer.
32:58 It's not documented anywhere.
32:59 It's just how I was thinking about it.
33:01 But yeah, so then there's this thing called release tool.
33:04 And so release tool is basically just a Python script.
33:06 on GitHub, if you go to github.com/python/release-tool, it's basically just a script that runs and then does all of the stuff that's needed to build the source distribution and documentation of a new Python release. And so you run that, and as a result, you kind of get like a huge amount of code that needs to be committed. And so there's a whole bunch of different little tools that kind of get run in addition to that. So things like Blurb, Sphinx gets as a part of that.
33:37 Yeah, you need a whole lot of dependencies installed.
33:41 So you like you need like late tech, you need a whole bunch of stuff installed for this to actually work.
33:45 Do you make sure you've got like the right right compiler?
33:50 Is there a lot of concern about that?
33:52 Like is it clang or LLVM or GCC or, you know, I believe it's GCC.
33:58 I believe it's GCC.
33:59 You know, I didn't dig into that exact point.
34:01 I'm pretty sure it is.
34:02 I feel like that's another part that you like that choice matters, right?
34:06 Like you might get different outputs, you might discover bugs that weren't there.
34:09 If you can.
34:09 So you gotta kind of get that gray box all set up as well.
34:12 Right?
34:13 Yeah, definitely.
34:14 And so the fact that it's on someone's machine, I would say reproducibility is, is quite tough in those sorts of situations, right?
34:22 Like if, if, if someone were to build a release of Python and then a day goes by, like I guarantee you that the, if you tried to do it again with the exact same starting input, it would be potentially different.
34:34 Right.
34:35 And so that that's actually a problem from a security perspective for reproducibility, because it means that someone else can't verify that what you did is correct.
34:45 Right.
34:46 Like the hash might change or something like that.
34:48 Right.
34:49 Hashes might change or if there's like somehow the time is getting embedded into a binary somewhere.
34:55 Like there's just a whole bunch of different ways that reproducibility could be different between two different runs.
35:01 Yeah, and so that after the release tool happens, you basically get handed a couple of source tarballs that just have everything in them.
35:09 And then you also get a bunch of Git changes to the repository.
35:13 So these are things like the change log has been generated from all of the blurb notes.
35:19 And so what you do is then you take all of those changes, and you create a big commit that says this is the release of CPython 3.12.0.
35:28 And then you tag it and you push that to the fork.
35:32 You don't push that to the main repository yet.
35:34 And so the reason you don't push that to the main repository, and this is actually something that this phenomena is an interesting one, especially because a lot of security build tooling doesn't take this work workflow into account, which is you don't push the tag to the origin until the very end.
35:55 A lot of security tooling like Salsa and all of these things will kind of, like, they won't assume because you can do it without tags.
36:03 But, like, they'll have features that are, oh, we'll capture the git tag in the output of, like, this, like, document that says that your build is, came from this repository, this setup tooling, whatever, right?
36:16 But a lot of projects actually wait on the tag until the very end because they want things to be repeatable without causing confusion.
36:25 So if something went wrong throughout this whole release process, you wouldn't want there to be a tag on the main repo that is now confusing people, right?
36:33 Because those tags cause notifications.
36:34 >> I'm going to get the code.
36:35 You're like, actually, no, the Mac build wouldn't build.
36:38 We're going to fix that or whatever, right?
36:40 >> Right, exactly.
36:41 And so this whole thing is like very, it's a common thing.
36:45 So pip actually has this exact same workflow where they do everything in the release until the very end, and then they push the tag to the repo because they know that if they push that tag, and it needs to change, it's going to cause confusion. So, yeah, that kind of is like step five in there. It pushes that tag. So that tag is highlighted red as a source of risk because that is a fork of CPython. And tags, tags are not in themselves verifiable.
37:14 Because anyone can push a tag. If you're just looking at just the name of the tag, anyone can write a tag if you have write access to a Git repository or a GitHub repository.
37:26 And so if you push a tag and say someone has access to your account, someone could move that tag to a different commit, completely different commit.
37:33 And if it were pulled in that time, and no one decided to like check something else, so like the commit hash, for example, you would be circumvented, like it would be able to inject code into the process, right?
37:44 That's as far as we know, that has not happened.
37:46 Yeah, that is bad.
37:47 bad to inject someone else's arbitrary code into Python directly.
37:53 It's actually doubly bad because there's different degrees of bad in supply chain security.
37:59 One degree of bad is if you're able to do bad stuff, but then people notice.
38:05 Or people can see what happened.
38:07 And so this is kind of like degrees of bad of you can inject code and then you can clean up after yourself afterwards.
38:13 Put the tag back.
38:15 Put the tag back.
38:16 you have write access, right? And so that that sort of like put the tag back or being able to circumvent it in that way is not good. Yeah, don't give people. These are not new ideas. So I feel less bad talking about these. Yeah, another thing in this article is actually like tying every step of the release process back to a known attack that has succeeded against another project. So like SolarWinds was about build, like release artifact poisoning, essentially, right? You're like taking advantage of the fact that these release artifacts are being distributed. And the you know, they're signed, everything looks good, but you've gotten code injected into the actual artifact itself, and it didn't get noticed. Right. And so like that sort of attack could happen against Python if if there's not mitigations against it, right. And so And so yeah, then the next step is six.
39:11 And so at this point, the release manager has source tarballs from the result of just running the release tool.
39:17 And so they're kind of just waiting.
39:18 They're sitting around waiting for the other experts to do their thing now.
39:22 And this is where Windows and macOS installers are actually built.
39:28 And so these aren't built for security releases, but for 3.12.0, these will be built.
39:33 They're two completely different sets of build processes.
39:38 One of them happens in Azure Pipelines.
39:40 So if you go to that release tool, there's actually a Windows folder that just has 100 YAML files in it for all of the different Azure Pipeline configurations.
39:50 - Yeah, Azure Pipelines, a little bit like GitHub Actions.
39:54 People who are familiar with that, but not Azure, right?
39:57 Something roughly?
39:58 - Yeah, yeah.
39:58 And I think there was a time where Azure Pipelines and GitHub Actions were basically the same thing.
40:03 Yeah, like in the very beginning.
40:05 yeah, so Azure pipelines, there's a whole bunch of stuff that happens there.
40:10 Because there's Windows embeddable packages is something that gets uploaded to python.org.
40:15 Windows installers also get uploaded to python.org.
40:18 But that also there's like NuGet gets updated, the Windows Store gets updated there.
40:23 So like, there's a lot happening there.
40:25 And there's actually a lot of things that happen that require the actual release manager, or the Windows release manager to approve them.
40:33 So they're gated.
40:34 These are things like signing keys that are as a part of the process, you have to sign off to say, "Yep, we want to give this job access to the signing key." So that's great.
40:44 - Do you know if WinGet is kind of like the newer package manager for Windows?
40:49 Do you know if Python goes there as well these days?
40:52 Let's see.
40:53 - I haven't used Windows in a really long time, so I don't know.
40:56 - I do, but I only use it for playing games and simple Windows tests, so I don't know it.
41:01 All right, but yeah, so NuGet is like one of these kind of package managers for Windows, maybe Winget.
41:06 People in the audience, if anyone knows, they can let us know.
41:08 - Yeah, definitely.
41:10 I've not even heard of Winget, so I feel really out of the loop right now.
41:14 - It's only a couple of years old, I believe.
41:16 - Gotcha, gotcha.
41:17 - And following on with that, this probably won't help you if you don't do a lot of Windows.
41:21 Louis asks, "Does Python maintain Chocolaty?" Which is similar to Winget, but more independent than Winget.
41:30 They tried Winget and it was buggy and wasn't great.
41:32 Chocolatey is maybe a little older.
41:33 I don't know.
41:34 Same story probably, right?
41:35 - Yeah, I also don't know, unfortunately.
41:37 Windows is not my strong suit.
41:39 (laughing)
41:40 - Chocolatey.
41:41 - Yeah.
41:42 - Yeah, it's all good.
41:43 - Definitely a tastier name.
41:44 That's how I choose all my tooling, is how tasty is the name.
41:47 Yeah, and so then the, like the counterpart to that is the macOS binary installers.
41:54 And so those are, There's basically just another set of scripts that get run on the macOS release manager, like their machine, and it builds just everything having to do with macOS.
42:06 It does notarization.
42:07 It does, you know, making I know not as much about macOS either, but I know that notarization happens and actually I've talked to a few folks about this one too, because this one, if you notice is just one square and it's just red and it's like, yeah, there's there's a lot more to dig into there.
42:25 yeah.
42:26 But yeah, the MacOS notarization stuff is a serious pain as well.
42:29 Like it's a little bit like submitting something to the iPhone app store where it, you know, it gets reviewed and yeah.
42:36 Yeah.
42:36 No, I think once you have it already notified, you can publish updates.
42:39 Cause I was thinking as you were talking about that, like how do you design or deal with the latency, you know, of like I've submitted it and how long does someone picks it up and reviews it?
42:49 I don't know.
42:49 But I guess once it's kind of approved for notarization, then it, it just It goes through pretty quickly.
42:54 You just set a really long time out on your CI job.
42:58 Like exactly time out seven weeks.
43:00 Yeah.
43:01 Give it a week at least.
43:02 Yeah, no.
43:04 And so then after all of those things kind of happen, there's this stage in, you know, there's a stop, stop, stop line in the, PEP 101 that just basically says for the release manager to wait for everyone to be done with their thing.
43:17 And then once everyone's done with their thing and has uploaded everything to to python.org, there's basically this phase of testing everything.
43:24 So you download everything, you make sure everything works on all the operating systems the way that's expected to.
43:29 You don't wanna blow the horn and then realize the very last mile that there's something wrong.
43:35 And so lots of testing happens, and then after the testing happens, at that point, then everything that will get uploaded to python.org, or that has been uploaded to python.org, gets signed by the release manager in particular with SIGSTORE and GPG.
43:49 And then from there, all of those signatures get published.
43:52 And that is the end of python.org.
43:55 All of the artifacts are published.
43:56 And only then does the release manager, this is step 10, do a git push to the actual upstream CPython repo.
44:04 And at that point, then the branch can be unblocked, all of these things, the release is done, everyone's happy, we can celebrate.
44:11 Yeah, that's kind of the very high level view of what happens.
44:16 - Amazing.
44:17 A couple of thoughts here.
44:18 First of all, the Windows Square is like tangibly different than the macOS Square.
44:25 The Windows Square says, send off some source code to Azure pipelines, a external repeatable CI build system.
44:33 Whereas the Mac line is like, build it.
44:35 You know what I mean?
44:36 That's actually pretty interesting.
44:38 And I don't know that there's an Azure pipeline for macOS, right?
44:42 Like that's just the sort of the story of Mac and servers and.
44:45 But just highlighting those differences, right?
44:48 Like it's, it's a different thing to say I build on my machine with whatever I got versus I pushed to something like Azure pipeline CI CD and get some results.
44:56 Yeah.
44:56 Both the source tarball builds and the macOS builds are all local.
45:02 Right.
45:02 And they're not done with a repeatable like CI provider.
45:06 I was actually able to get like the actual source tarballs to work on GitHub actions.
45:12 And, and so that, that's something that I've worked on and have provided that to release managers and we're talking about it right now.
45:19 And macOS is actually, to my knowledge, is actually going to be revamped a little bit because that tooling has been around for so long.
45:28 I think that there's going to be some work done on it.
45:30 And so I haven't dug into it as deeply.
45:32 I've kind of more just provided a bunch of context to the release managers about things that I want to see.
45:37 Like if I were to do this myself, this is the other things I'm interested in.
45:41 Right.
45:41 Provided that you actually had access to a macOS system, I'm sure you could do it all from the CLI because for example, for the talk, Python and courses apps, we use Flutter and you just say, what a run.
45:53 And it compiles that, that iOS or macOS version somehow through the command line and then like outcomes a, a thing that is already deployed onto, you know, a simulator or something.
46:04 Right.
46:04 So it could happen.
46:06 I mean, there is, there is some hosting, but it's kind of like, here's your Mac mini in the cloud.
46:12 Go, go.
46:13 I mean, there is a, what is it?
46:15 GitHub action supports macOS.
46:16 So does it?
46:17 Okay.
46:17 So yeah, so possibly, possibly.
46:19 Yeah.
46:19 Maybe, maybe it could happen.
46:20 I'm not saying it has to, I'm just, it was a kind of a striking difference that one is like a CI/CD process and one is local and manual.
46:26 Yeah.
46:27 Actually windows is the most repeatable, most like actually isolated from, from any, from the, just the machine itself, right.
46:36 The release manager themselves.
46:37 So yeah.
46:38 Yeah.
46:38 That's pretty interesting.
46:39 And then we have binaries and they get published and everybody is happy.
46:43 We all go and install it and it's been in testing for quite a while, right?
46:47 Like we've had a bunch of betas.
46:48 We had alphas.
46:49 We have had three release candidates and on Monday we'll have a dot zero.
46:54 It's exciting.
46:55 It's really exciting.
46:56 It is exciting.
46:56 The other question I would ask you is like, what is the time from like step one to step 10?
47:02 How much clock time has passed wall time?
47:05 I would say that it is on, it's on the scale of hours.
47:10 It's not like a day.
47:12 which is kind of incredible, right?
47:15 The fact that you get three volunteers together to do a bunch, all of this stuff, and it only takes a couple hours.
47:22 It obviously depends on how many problems you run into on the way, because if you get to the very end and then there's a problem that you find in testing, you have to kind of start over.
47:31 But yeah, if everything goes to plan, which has happened, I've coordinated security, like there was a couple of security fixes that were in 3.11.5 that I coordinated.
47:43 And yeah, I got to kind of witness the, okay, we've decided we're making a security release to the actual, it's out there, we can now talk about the vulnerabilities.
47:52 Yeah, it's on the scale of ours.
47:53 Is that the int parsing thing?
47:54 It was the, what was it?
47:56 There was an int part or was it int parsing?
48:00 No, it wasn't.
48:01 Really long ints, I don't know.
48:02 No, that's a different one.
48:03 No, it was, there was a TLS and this one sounds really scary.
48:09 It's not as scary. It's a TLS bypass on the server side. So like if the client is supposed to be authenticated, for example, if you're using MTLS, you can if you like send a handshake and then immediately close your socket, you can get lucky during like a brief period of time where the socket on the other side, like the server socket will say like, oh, it's closed. So it's it's fine, right? And the data that whatever you sent over the wire won't will still be readable. But then the handshake hasn't actually like completed. And so that sounds like scary, right? But it's it's actually not as bad because so many protocols, well, for one, so many protocols don't even use client authentication at all. So that's one huge point. The other side of it is that all protocols that people do end up using mutual TLS with are protocols that you have to send something back to... So like HTTP, for example. You have to send something back, and at that point, the socket is closed. And so it can't send the data back.
49:21 So there's no data exfiltration. It's a pretty narrow scope of what's actually vulnerable.
49:26 Yes, the fix for that vulnerability is in 3.11.5.
49:31 And there's another one too, but it's a little bit more minor.
49:34 Well, we'll start fresh with 3.12.
49:36 And hopefully you will not have to witness one of these releases.
49:39 Honestly, given how large CPython is and how wide ranging its standard library attempts to be, it does not have many vulnerabilities.
49:48 I mean, you go look at like a web browser upgrade.
49:50 It's like, here's the 27 remote code execution vulnerability.
49:55 Maybe not quite that many.
49:56 But here's the 27 security fixes this month, you know?
49:59 - Yeah, we've maybe, maybe it's a little more front of mind right now, now that like Chrome is on their fifth zero day of the week.
50:05 But yeah, it's, it's pretty tough week for Chrome right now.
50:08 - Yeah.
50:09 - And Firefox and the rest of us.
50:10 - Yeah. Yeah, for sure.
50:12 Let's see another comment from Frank says, "SIGstore and GPG seem a bit at the end of the process, considering these are built locally in some cases.
50:23 How does one guarantee that the sign sources are still right?" Yeah, so this is kind of getting to the, like, I agree with you, how does one track that they're still right?
50:34 So they get uploaded to Python.org.
50:37 And at that point, they're kind of like in a holding pattern for testing.
50:41 And so those binaries are there, they're on Python.org, they've got, you know, the hashes have already been taken.
50:47 Then the testing happens.
50:49 So like, anything that happens between that and the Sixer styling, like, yeah, at that point, it's kind of protected.
50:54 But yeah, there is this really big dwell time where you just have artifacts sitting on someone's machine, and they're going to get signed and sent out the door. And that's kind of like the due diligence of release managers, they need to make sure that those are the right artifacts that they've been tested properly, that the ones that you tested locally are the ones that are going to get ended up being signed. Yeah. And so this is something this is like a, a piece of the whole puzzle that gets mitigated by having those builds happen on external services as opposed to on someone's local machine. Because at that point, you're just giving --
51:28 >> Yeah, you're just giving this, like, set of scripts, like, very, very narrow set of input. Like, I want this exact git commit and this version number to get built. And then out pops, like, a tarball that's already been signed and verified and everything. And then from there, you just put that on Python.org, right?
51:45 >> Awesome.
51:46 >> Like, that is a lot less -- there's a lot less of that risk where it's just on someone's machine and how, how do, how does one know that that is the exact thing that they built?
51:56 It's not provable.
51:57 Yeah.
51:58 What about virtual machines for these gray boxes?
51:59 I mean, obviously the Azure one, that's, that's its own thing, but have you considered like a, a Docker or an official just RLs VM go like here, take this, run that, you know what I mean?
52:12 yeah.
52:13 To, to make it a little less dependent on the person whose role it is that year.
52:17 It definitely would help with the reproducibility side of things, because as we know, Docker is just the whole machine shipped in a box, right?
52:25 So yeah, in theory, if you build the same thing again in that exact same image, you would end up getting a pretty similar result.
52:31 So it helps on that front, which is good.
52:34 I think the tough thing is that you still don't have control of the inputs, and the output you also still don't have.
52:42 It comes out of the Docker container, you get a tarball.
52:45 There's still that time in between when the tarball is out of the container and when it's actually on python.org, where it's on someone's machine that is not in a container.
52:55 And containers also aren't the best in terms of like, you did mention virtual machines as well, but even that, like on a machine that is a high value target, you know, maybe it's worth it.
53:09 Who knows?
53:10 Yeah.
53:11 Yeah.
53:12 Who knows?
53:13 All right.
53:14 It's happening Monday.
53:15 So very exciting.
53:17 And I'm looking forward to all the new work.
53:20 I mean, are there new features in 3.12 that are you particularly excited about?
53:25 You know, since being in this role, I actually haven't been paying attention to Python releases as much, Python features as much.
53:32 I'm mostly excited about it getting faster.
53:34 I mean, the generic, it's just straight up faster.
53:37 Like there's no one on the planet that's not happy about that.
53:41 What are the major features this release?
53:43 So there's like a some more broad F string thing.
53:46 So you should be able to have a subset of the language in the F string.
53:49 And now you can kind of like program in the F string, I believe.
53:52 type statement, better generic types.
53:56 Yeah.
53:56 There's like a, a simpler way to express generics in the type system.
54:00 And then I'm with you, you know, honestly, like f-strings being nicer.
54:05 That's awesome.
54:06 Thanks for that.
54:07 But faster, faster, faster is good.
54:09 this per interpreter, Gil's kind of part of that.
54:12 faster CPython thing, right?
54:14 Like it's hard to leverage, but if you could just say in your threads, new interpreter for this bit, right?
54:19 All of a sudden you escape the GIL for computational stuff.
54:23 That'd be pretty interesting.
54:24 but you know, also this, this is another interesting thing.
54:27 If you pull up this list, like the size of the scroll bar, I don't know.
54:31 We were talking about scary scroll bars earlier.
54:33 Like this is next level.
54:35 Let me see.
54:36 I'm going to, I'll throw this into Omnivore.
54:38 Are you a fan of Omnivore?
54:39 This app?
54:40 Omnivore.
54:40 Never heard of it.
54:42 What is it?
54:42 Omnivore.app.
54:44 Kind of a Instapaper pocket replacement.
54:46 Oh, I'm already logged into it.
54:47 Look at that.
54:47 Oh, super, super cool.
54:49 So I go here and you say, add a link.
54:52 Actually, it just, I think it just went in there.
54:54 Let's see.
54:54 40 minutes just to read that.
54:56 What's new.
54:56 That's what I was trying to pull.
54:58 It just, yeah.
55:00 It calculates that for you.
55:02 It's like, it's going to take a while.
55:03 So just the, a sense of like, what is new in Python 3.12.
55:07 That's a ton, right?
55:08 Yeah.
55:09 A ton of stuff, like tons of fixes.
55:11 Tons of fixes and improvements.
55:13 Mike Fiedler out in the audience says Python dash M, SQLite 3 is nifty.
55:17 Mike, you're going to have to tell me more about this.
55:19 Does that, does that open up a database now?
55:22 That would be lovely.
55:23 Yeah.
55:24 That would be cool.
55:25 Kind of like the HTTP server.
55:26 Yes.
55:27 Yeah.
55:28 Or the JSON tool.
55:29 The JSON tool is the one I use the most.
55:31 What's that one do?
55:32 so you can like pipe JSON into it and it'll just make it pretty.
55:35 It's pretty, it's questionable how useful it is, but it sure does make pretty JSON.
55:40 Yeah.
55:40 which I really appreciate.
55:42 Like two space indented JSON.
55:45 That's perfect.
55:46 - Indeed. All right.
55:47 One more comment from the audience and then we'll maybe wrap it up.
55:50 Karen, hey Karen, says, let me, she's clarifying above, like VM Docker for less changes and better reproducibility.
56:02 "VM Docker for less heterogeneity reproducibility than the current situation, which is affected by a given user's environment." Yeah.
56:10 Like what web browser did you install today or something completely unrelated or, you know, or like you upgrade, if it's your personal machine and you're using tools, like if you upgrade those tools, it could pull some other system dependency, right.
56:23 That ends up being relevant to the build of CPython.
56:26 And yeah, it's just, there's just so much that could be different over the course of even a few days.
56:31 Yeah.
56:32 Yeah.
56:32 Which is why it's interesting that the windows one goes to Azure pipelines.
56:37 Cause it kind of pulls, puts that to the side.
56:39 Right.
56:39 Yeah.
56:39 All right, Seth.
56:40 Well, this has been super, super interesting.
56:43 And thanks for giving us a look into this world.
56:46 So, yeah, very, very excellent.
56:48 And I guess, you know, final two questions.
56:50 Can you write some Python code?
56:52 What editor are you using these days?
56:54 Yeah, I'm using PyCharm.
56:56 I love PyCharm.
56:57 Specifically, you know, I have the--what is it?
57:01 I had the Professional Edition for a good long while, and then I let that subscription lapse, and I need to just renew it.
57:07 So I'm using the community edition right now, but yeah, I love PyCharm so much.
57:11 So really good editor.
57:13 Awesome.
57:14 I'm there with you.
57:15 And then notable PyPI package, something that is interesting to you.
57:18 I already talked about these.
57:20 So pip audit, just use pip audit.
57:22 There's so much work happening somewhere with pip audit that it just makes sense for you to use it because there's just a ton of work happening.
57:30 And if you just add it to your workflow, you'll just know that things are vulnerable or like what's vulnerable and like what versions you should update to.
57:36 >> Yeah, I already have some tooling that, like some aliases that do three or four steps to actually update my dependencies and install them.
57:45 Like I might as well just throw this as like another and and pip audit.
57:49 >> Yeah, and you can actually, so I think there's a way you can tell pip audit to install packages, but then it will say something if there is a vulnerability in whatever you're installing.
58:01 So you can even replace pip install with that, I believe.
58:04 >> Use pip audit.
58:05 I think that that is a feature.
58:06 All right.
58:07 I think so.
58:07 It has a features thing.
58:09 Let me look at it.
58:09 Oh, let's see it.
58:10 Oh, it's not going to work.
58:11 Cause it's not the read me.
58:12 It's true.
58:14 I got to go to the homepage, which will take me here.
58:17 So I gotta go to the source.
58:17 no, where's the GitHub.
58:21 I only go to the stars.
58:22 All right.
58:22 You know what?
58:23 Maybe I don't want to click this thing.
58:24 Features.
58:25 There we go.
58:25 Multiple editing.
58:27 We haven't talked about S bombs, but seamlessly reuse here.
58:32 listening. Yeah. If it reuses the pip caches, maybe, maybe, maybe. All right.
58:37 What is it?
58:37 I'll play with it. We'll see. People can check it out.
58:39 Citation needed. Citation needed.
58:41 Exactly. Citation needed. That's right. All right. Well, yeah. People check out pip audit.
58:46 That looks excellent. And yeah. Final call to action. People are interested in this. I mean, give a thought to it on Monday, theoretically, at least in the past, if you listen too far in the future, but you know, the next release.
58:59 Yeah, no, I think the biggest so in terms of like, what can you personally do? Because like, there's just I just talked about so much stuff that is just kind of happening in the background.
59:08 But there's also stuff that like individual people can do. And the most impactful URL that you can visit for security, if you're an open source consumer or like a maintainer, like it doesn't matter, this is just gonna be impactful if you work in software is best.open ssf.org. So So that is basically just like a web page and it just has a few URLs and you click into any of those URLs and it just gives you like a checklist of your things to think about here's and then if you click into those checklist items, it gives you it's kind of like this recursive nice, like guiding force of like, here's the things you could be doing, right?
59:46 And if you want to dig in more concise guide for developing more secure software, in other words, NPM best practices, and so on.
59:54 Yeah, excellent.
59:55 I'm hoping to put a Python best practices guide there by the end of the year that I'm here.
01:00:02 So don't you worry. Look forward to that.
01:00:03 Excellent. I'll be right there. We'll get it above on the list.
01:00:08 Yeah, it'll somehow not sort alphabetically and it'll be okay.
01:00:12 Tip the scales a little bit.
01:00:15 Exactly. Cool. Well, thank you for being here, Seth.
01:00:19 And thank you for all this hard work that you're doing. It's good to know that you're out there keeping an eye on all these things.
01:00:24 Thank you so much for having me on the show. This has been lovely.
01:00:27 Yeah, you bet. Catch you next time.
01:00:29 Bye.
01:00:29 >> This has been another episode of Talk Python to Me.
01:00:33 Thank you to our sponsors.
01:00:35 Be sure to check out what they're offering.
01:00:37 It really helps support the show.
01:00:38 The folks over at JetBrains encourage you to get work done with PyCharm.
01:00:43 PyCharm professional understands complex projects across multiple languages and technologies, so you can stay productive while you're writing Python code and other code like HTML or SQL.
01:00:55 Download your free trial at talkpython.fm/donewithpycharm.
01:00:59 Want to level up your Python? We have one of the largest catalogs of Python video courses over at Talk Python. Our content ranges from true beginners to deeply advanced topics like memory and async. And best of all, there's not a subscription in sight. Check it out for yourself at training.talkpython.fm. Be sure to subscribe to the show, open your favorite podcast app, and search for Python. We should be right at the top. You can also find the iTunes feed at /itunes, the Google Play feed at /play, and the Direct RSS feed at /rss on talkpython.fm.
01:01:33 We're live streaming most of our recordings these days.
01:01:35 If you want to be part of the show and have your comments featured on the air, be sure to subscribe to our YouTube channel at talkpython.fm/youtube.
01:01:44 This is your host, Michael Kennedy.
01:01:45 Thanks so much for listening.
01:01:46 I really appreciate it.
01:01:48 Now get out there and write some Python code.
01:01:49 [MUSIC]