#360: Removing Python's Dead Batteries (in just 5 years) Transcript

00:00 Python has come a long way since it was released in 1991. It originally released when the Standard library was primarily the totality of functionality you could leverage when building your applications. With the addition of pip and 3680 package on PyPI. It is indeed a different world where what we need and expect from the Standard library is not the same. Brick Cannon and Christian Heinz have introduced Pep 594, which is the first step in trimming outdated and unmaintained older modules from the Standard library. Join us to dive into the history and feature of Python's Standard Library. This is Talk Python to Me. Episode 360, recorded March 29, 2022.

00:56 Welcome to Talk Python to Me, a weekly podcast on Python. This is your host, Michael Kennedy. Follow me on Twitter where I'm @mkennedy and keep up with the show and listen to past episodes at 'talkpython.fm' and follow the show on Twitter via @talkpython. We've started streaming most of our episodes live on YouTube. Subscribe to our YouTube channel over at talkpython.fm /YouTube to get notified about upcoming shows and be part of that episode.

01:22 This episode is sponsored by Microsoft. For startups foundershub, check them out at talkpython. fm/foundershub to get early support for your startup, and it's brought to you by Fusion auth your authentication and authorization platform built for devs by devs. Check them out at talkython.fm/Fusionauth. Transcripts for this and all of our episodes are brought to you by AssemblyAI do you need a great automatic speech to text API? Get human level accuracy in just a few lines of code? Visit talkpython.fm/assemblyai.

01:56 Brett Christian, welcome to Talk Python to me.

01:59 Thanks for having us.

01:59 Yeah, thank you.

02:00 First time for me. Yeah.

02:02 Christian, it's great to have you here. That's super. And Brett, I think you may have been on before memory recollects.

02:08 Yes, I have been on it. I had been on previously.

02:10 I love having you on. It's always fun to talk about stuff, and it always seems like somehow we touch on web assembly. I think one of the very last conferences I went to was PyCon, and we recorded like on the Expo floor talking about web assembly. So maybe we'll find a way to get back to that.

02:24 Do you remember that conversation very distinctly?

02:26 Actually, yeah. That was fun. Even though you've been on a lot. Maybe just real quick, tell people about yourself. You've been Python core diving for a long time and something to do with VS Code and Python as well.

02:36 So tell people about that stuff.

02:38 Yeah, just keep it short and sweet. I am the Dev Manager for the Python experience in Vs code at Microsoft. I have been a Python core developer for 19 years, and I've been a member of the Python Steering Council since its inception back three and some odd months ago. I think I don't want to keep track of when that started.

02:58 Yes, that's me. Awesome. Christian, how about you tell people about yourself?

03:02 I work for Red Hat.

03:04 May have seen that you have a red Hat and you work for Red Hat. That's awesome.

03:08 Yeah.

03:08 Red Federal, by the way.

03:10 So work for Red Hat and security engineering Department work on something that could be easiest to describe as an open source implementation of active directory. I'm currently working on discontinuizing all the things and getting that up to the Internet, so that's a fun one. As part of my security work, I also do security for CPython, so I take care of some of the models and I'm part of the security response team. Been a quadrupler for 14 years, give or take. Probably long time eventually. So I started doing Python like 20 years ago. That was fun.

03:48 Awesome.

03:48 Yeah.

03:50 Anything else you want to know about me?

03:51 No, I would just like to say thank you on behalf of everyone for keeping our machines from getting rooted because we're running Python web apps. That's pretty awesome.

03:59 You're welcome.

04:01 Seriously though, how often are there meaningful security problems in CPython? I know that one of the big talking points about moving away from Python Two was like, well, if you don't come along for the ride, you're not going to get security updates. But how often is that a big problem? Really? How often are there problems where our CV is filed? Like, oh, we got to jump on this and quickly get it out before the last version of Python that came out before we had the emergency release for the bug fish.

04:27 So the 310 Two, I think had like eleven or 13 CVE, although only one that directly affected was in the Python code. So we also ship a bunch of extensions. Like we ship a library to pass XML, they had like six or eight DVEs. We had a ZLab fixed with something else. So it's not just Python core that's affected, but since we bound ship several libraries with Windows and the Mac in stores, you may be also affected. You have Linux Distros, they typically have their libraries debunked and use system libraries and they update them out of bounds.

05:09 Okay, well, it's actually more than I realized. To be honest, you don't hear too often about it being a big public problem like Log for J or something like that.

05:18 Yeah, I'd say typically the CDs we end up dealing with is the bundling that Christian mentioned. Christian does a lot of work to make sure we're constantly compatible with the newest versions of OpenSSL. So for instance, on 27, had you not moved over, you're going to have to do the work to be able to use OpenSSL. I think three is the newest one or the one coming up. So that's the real kind of concern. Unless of we screwed up as cordevs and have a nasty CV, it's usually something we depend on that typically triggers this kind of thing.

05:46 Thank goodness interesting.

05:48 Yeah.

05:49 Well, speaking of the standard library and stuff, let's start out by talking about Python just on GitHub, because I think maybe the time of year releases, it might be already a week old. But you were one of the big proponents of moving CPython and the Python organization over to GitHub, right?

06:09 Yeah, I drove that move.

06:11 Thank you.

06:11 That's awesome. It's so nice to be over there. But one of the challenges has been that bugs.python.org has kind of been its own island, especially since this move. Right?

06:21 Yeah. So when I helped move us off of material to get and let's move us to GitHub for pull requests and stuff, I basically had to choose my battles. And the battle I chose was changing our version control system and no longer self hosting our version control system. I was not up mentally to the challenge of also trying to move the development team over to a new issue tracker as well. So I purposely punted on that problem and tackled the one I thought I was capable of handling and dealing with. And that's why we end up with this somewhat split personality problem of Python being on GitHub for pull requests and for code hosting, but not for issue tracking.

07:04 Well, you just said yesterday, I believe was yesterday, maybe the day before yesterday on Twitter that, hey, it's happening, the bugs Python.org is migrating over, and then it got delayed until April 1. So hopefully that happens really soon. But what was really surprising to me was just how long that migration is. Right. It's not like copying the files over, get in, it get commit or the equivalent for the issues. It's days to migrate all the issues.

07:33 Yeah. So there's tricky parts to this one is we're getting internal help from GitHub, but obviously it's not common to move issue trackers into GitHub per se. Like a lot of projects, I think, just start on GitHub and have for like a decade. So the idea of moving over is kind of a new thing. And on top of it, not very many projects have the volume of issues historically to over that we have. Right. We have like 7k open issues, I think, alone. So the volume there also kind of causes an extra overload. And because GitHub, as I said, they have some internal tooling for this, but it's not like something that they work on month over month. It's there and we're using it, but it's not optimized because it's just not a typical business concern, which I mean, totally makes sense. Right. How often do you have a project of our size and age wanting to migrate over like this just for the numbers?

08:27 So we have almost 59,000 in total on the backtrack and almost 8000 open 7,700.

08:36 Yeah. And trying to translate over as much metadata as possible is computationally expensive. Right. Because what we have to do is have to generate a dump and then the dump has to be it's a different data model.

08:49 Right.

08:49 And it's a different day model on top of it. Correct. So for instance, one of the reasons this took so long was we had to first decide we wanted to do this, convince people that we should do it, find someone. It was at some malade initially to come in and kind of figure out how to map things and get things working. And then earlier this year we had Wookash step in and help out, as well as the developer and residents to help push this over the finish line. And on top of all that, having to process all that data, export it to the right format, and then have GitHub's tools pull it in and not cause a strain on their system because this is a lot of data to suddenly dump in and have to replicate across their entire cluster. Right. So there's a reason why it takes so long and we also don't want to have it happen during because we're probably going to be running at a low like process priority, which also means we want to do it like on a Friday in the afternoon, so that it's over the weekend and not when GitHub gets hit the hardest. So there's a lot of coordination going on. And as we all know, GitHub had some stability issues the other week and so they just said, can we just wait till a little later to make sure that Soul is good and solid and everything looks good? So the plan is April 1.

09:59 Excellent. Very exciting.

10:00 We also have a good track to actually break GitHub. I'm not sure if anybody has used the live stream. Pablo changed the master branch to main branch to rename the default branch, but yeah, we broke GitHub and it took a while to recover from that too. So since our repos are so gigantic. So we started with ZVS, then moved to the end of version, then went right out of all this stuff to mercurial and then to Git. And I'm not sure how many revisions we have in the tip, but it's gigantic.

10:37 Just to be ultra clear, thanks to GitHub for working with us directly. And actually they donated money to help us make this all happen. So they've been really great partners in all this. Thanks to ezio for getting this started, Marietta for the initial pet, by the way, and even starting this conversation and wukash to stepping in and helping get it finished.

10:56 Yeah, definitely a team effort.

10:57 A lot of people involved and thanks to all of them.

10:59 So Tuchar out in the audience is joking. Is this why GitHub has been crashing the whole week? But Christian, you were joking. Like maybe. Did you guys actually cause problems?

11:07 Not that I know of specifically.

11:08 Okay.

11:09 But not that I know of. Yeah, okay, I hope not. I apologize to the world if we did somehow well.

11:15 It wouldn't be the first time there's other outages and other crazy things, unfortunately.

11:22 Yeah, for sure.

11:24 I think there was a huge DDoS attack not too long ago anyway, also came out in the audience asked, assuming their usernames and email addresses on the original issue. So like, how much did you all care about having Fidelity across those? And you know, this could have been an opportunity to just say, you know what's awesome? Command a archive in your inbox to just kind of catch up and be okay. You could have just dropped it. So we're just going to start over and if it's important, it will find its way here. And if it's not, then it wasn't.

11:49 Yes. I mean, there has been talk about this. There's varying opinions on how important the digital archives are. Like, do we need to move all the closed issues over? Should we only move open? Like, we actually discussed this when we were looking at potentially up to a week to do the migration versus the two or three days it's actually going to take now. But everyone has different opinions of how important the history is. Specifically answered the question, though, from the audience about usernames. As long as you have your GitHub username attached to your bugs Python.org account.

12:20 That should map over and you've got until April 1 to get it in there.

12:24 Yeah, I don't remember if we map email the email. There might be some Privacy issues. I can't remember where that all landed, like lawyers were consulted and I was out of the loop on those. But I believe if you at least have specified your GitHub username on bugs.python.org, we will be able to do the mail.

12:40 Yeah, fantastic. All right, well, thank you both for the update on that. It's not exactly why we're here, but it's so timely and I think you're both involved in it and stuff like that, so quite cool to see it coming along. I personally really think that it's fantastic that CPython is on GitHub. I know it's not that different if it was a self hosted git or GitLab or even Mercurial or SPN, but there's just something about it seems more open to contributors given just sort of the status of GitHub where a lot of people seem to hang out and the whole PR flow and those kinds of things.

13:12 That was part of the motivation. Somewhat Ironically, we actually had to teach a bunch of core devs how to use GitHub as part of the migration, because to be fair, some decent chunks of the Core devs don't contribute to other open source. I mean, Python alone is a big enough of a project that it absorbs a large chunk of your time. So they just didn't have to know how to do any other open source development for a different project other than, however Python did it. And we weren't on GitHub, they just didn't have a need to.

13:37 Plus, we also didn't move a while ago, so I think it's coming close to a decade at this point, so it was also a different time.

13:46 This portion of Talk Python to me is brought to you by Microsoft for Startups Founders Hub Starting a business is hard. By some estimates, over 90% of startups will go out of business in just their first year. With that in mind, Microsoft for Startups set out to understand what startups need to be successful and to create a digital platform to help them overcome those challenges. Microsoft for Startups Founders Hub was born. Founders Hub provides all founders at any stage with free resources to solve their startup challenges. The platform provides technology benefits, access to expert guidance and skilled resources, mentorship and networking connections, and much more. Unlike others in the industry, Microsoft for Startups Founders Hub doesn't require startups to be investor backed or third party validated to participate. Founders Hub is truly open to all, so what do you get if you join them? You speed up your development with free access to GitHub and Microsoft cloud computing resources and the ability to unlock more credits over time. To help your startup innovate, Founders Hub is partnering with innovative companies like OpenAI, a global leader in AI research and development, to provide exclusive benefits and discounts through Microsoft for Startups Founders Hub. Becoming a founder is no longer about who you know. You'll have access to their mentorship network, giving you a pool of hundreds of mentors across a range of disciplines and areas like idea validation, fundraising, management and coaching, sales and marketing, as well as specific technical stress points. You'll be able to book a one on one meeting with the mentors, many of whom are former founders themselves. Make your idea a reality today with a critical support you'll get from founderhub. To join the program, just visit Talkpython. Fm/founderhub all one word the links in your show Notes thank you to Microsoft for supporting the show.

15:36 So let's start by maybe talking about speaking at different times. Let's talk about the standard library, if you guys could approach it from a historical perspective to start, because we're going to focus on stuff that's in there that maybe should be removed in the benefits of taking it out and stuff. But when it got put in, it seemed like a good idea. So when I think about Python, one of the key phrases I hear, I don't know it's origin, but the batteries included story, right. Python comes with batteries included, and then it's frameworks did like Django and so on. And it's a huge selling point, right. This is a language that has a bunch of support built in, right?

16:15 Yeah. So the rough history for those who don't know Python went public February of 1991, right. So 32 years ago, it predates Linux.

16:26 It's really crazy when you look at the history of when projects came out, how long Python has been around but the other thing to think about is who is on the Internet or the World Wide Web even in 1991 who even had the Internet or even a modem back then. And so what that led to was people contributing things to Python that Guido would look at and go, oh, yeah, that's useful. And then just pull it in and put in the standard library. And it grew and grew. And at some point in the 90s, there was a website called the Vaults of Pernassus, which had animated gifs of wall sconces with little candles with the flickering whatever, like Polly bit like you can imagine what this is like back in GeoCities days, right. And that's where you went to get your code. And all it was was zip files of Python code. There's no concept of wheels or conda packages or anything like this. There are literally just zip files of code that you just unpacked and just copied over the directory that contains the other Python code. And you just vendored everything.

17:25 There was nothing else and stuff.

17:26 Moving stuff around the Internet was slow as well back then. I mean, a lot of people were on dial up.

17:31 Internet made noise back then.

17:32 You were blazing fast with your 56K modem, if you were lucky enough to have that. Right. So that meant that it was really hard to find high quality code out there. Right. And getting done and put up and all that. So things just kept getting added out of the standard library and it kept getting bigger and bigger and continued to be useful. And it was just a different time. Right. Like hell, not even everyone in the night even had a web browser. Right. I remember when I first got the night.

18:00 Right.

18:02 Did Mosaic come out in 93?

18:05 Yes. And actually one of the very first graphical web browsers was Grail written by Guido in Python. If you go look up the history. Right. I think Guido missed being the first graphical browser for months or something. Crazy. There's a weird history in Python in terms of early web, but what this all led to. Right. And I remember, by the way, when I first got into the Internet, it was Usenet and Gopher. It wasn't even the worldwide web that was still AOL days back then. For those of you Telnet can date yourself based on that knowledge.

18:34 Eugenet and Gopher, yeah.

18:35 It was a different story, Michael.

18:37 It was a bit of a Sidebar. Things were so basic back then, but at the same time, there was so much like imagination for what could be, I think, because so much of it was unmaterialized. And you're like, I can see where this is going to go.

18:49 Yeah. Intended was even much more expensive in Germany. So I know that until I was lucky, I went to University in 2000. But before, when I was living with my parents, I had to pay the Internet by the minute. So local calls. So we're not free in Germany, you have to pay for them. And I know that like end of 99. I was lucky. It paid like several hundred a month to have like a persistent Internet connection. Persistent. But I used the only phone line we had, but I didn't have to pay by the minute. But I paid like super expensive. That's why lots of people I start actually using mailboxes or dial in mailboxes and like this concept that you were sending messages, you push the message and later night the mailboxes will call all the mailboxes. They do this Unix copy to copy the messages. Around the next day we receive a message from a friend who's living like in a different city.

19:53 Yeah, I remember. It was amazing. Like all the back ends that we just kind of sync up sort of this distributed like the PBS world and some of the other stuff. Yes.

20:02 It made a lot of sense back in the days to have all the useful bits and pieces in Python. So that's why the library was so big.

20:12 We have just Python code. It was easy, but we also have lots of CCO and C extension and these were really complicated to compile and build. So you have to make files to figure out which compilers and libraries you need. And building things for the Windows was super painful for the Windows people that they had to get the right compiler version.

20:34 They had to figure out their VC VARs bath.

20:37 And each Python version had its requirements for different Visual Studio versions.

20:43 Yeah. So I think that people get a sense of the time. Right.

20:48 At that time, it was amazing for Python to say as much as we can get into the standard library, it's going to be a benefit to people because if you need to parse CSS color codes, Hex color codes and it's built in. That's a bonus. Right. There's not the idea of pip and fast Internet and all those things. Right.

21:07 Or even disk details. Right. Like, as Christian was saying, compilation was hard enough as it is. So getting something in the standard library and ported over to CCODE for performance was a huge deal. So it's not even just distribution. It was literally just creating these things. But it's difficult.

21:22 Sure.

21:23 It just shows how far we've come. Right. That these were even problems that we used to have.

21:27 Yeah, it's pretty interesting.

21:29 Do you think if Python were designed today from scratch, this is the kind of the language you want. It would be packaged in the same way that it is now.

21:40 Would it have a large standard library?

21:42 No. That would be controversial.

21:44 I agree with Brett.

21:46 Yes. Which is why we're friends and why we wrote this pet that we're going to be talking about.

21:50 We just all be written in web assembly.

21:53 We have friends.

21:54 Thank you.

21:56 We went to a movie together at one of the Pythons of Portland, man.

21:59 Yeah. And we have friends on Nintendo Network.

22:02 Yes. On Switch, look at more recent languages like Rust and Go and stuff. They have a much more targeted standard library. They're able to lean on the community and people seem fine with it. And I think that's the key thing is we get to learn the less people have gotten to learn lessons from us. But we also get can learn. We would have been able if we started now to learn lessons from other people. And I think that'd be one of them is targeted center library that is very tight and very targeted, very poor, very stable, and that's much easier to maintain. Would probably be the good way to go because it makes getting yourself up and going a lot easier versus, oh, this isn't useful until I have this huge standard library, which as a burgeoning project would have been really difficult because you had to build a lot of code to make that happen.

22:46 Right. I do value having it there, though. At the same time, I understand why it would be smaller. I just think there would be some use cases that it would be less of an obvious choice to use Python. Like, for example, if I just want to script something on my computer, knowing that it has Python means it has all these libraries where as opposed to if you've got to start installing dependencies just to get your automation scripts to run like there's this bootstrapping stuff that has to happen.

23:14 Yeah. From a philosophical perspective, we actually don't have a definition of what the standard library is. Right. There is no Pep, there is no guidance over what the standard library is now meant for what it should or shouldn't have. It simply doesn't exist. It's always been previously based on Guido's opinion somewhat. And then now it's the string Council. It's somewhat based on just kind of consensus on the Dev team, and that's kind of what the steering Council will approve, more or less. But there is no guidance. Right. Should we make it so that, as you said, that simple automation script for managing your computer, should that be have enough in the standard library to make that happen? Should we be able to have a simple actp server? I don't know. Should we have Tk enter where is the line? And we don't have an answer right now. There is a discussion going on right now, actually over on Python Dev discussing this kind of thing, because there was a proposal to potentially rip out URL them. I think it was partially done by Victor Center to kind of not get a rise out of people, but to kind of spark a conversation. And some people are going like, okay, Euro Lib was written back in the day when OS did not necessarily ship with the network stack.

24:30 And when we could plug into would it be better to actually rely on the OS? Do we really have to have all of that code in there just to parse things and figure things out to make the right Http request versus just going to macOS or going to Windows or using Curl and Linux or any Unix platform really and just have it handle the request? I don't know, but this is the kind of questions we're getting into now and we'll probably get to we can talk about later. I have one of my infamous Brett's Grand plan things around library, and this all ties into actually funny into making that decision of what do we want the standard library to be. So we have better guidance for ourselves as to where it should go. That's what should or should not be in the standard library now. Not to speak. People. I don't know if this means we never deprecate anything that doesn't follow this policy that's in there now, but I would like to at least as a Cordev know what we want it to be for today compared to the cheap shipment model of useful Python code on the internet. Pre Internet, right.

25:32 Michael, you asked me about security bugs. And so Victor's proposal, Victor's thread on Python dev sparked by discussion I had with Victor on internal communication channels where I pointed out that your lips actually a place where we have lots of security bugs and even in very trivial things like turns out like parsing like in Eural, something like splitting up the protocol, the host, the path name, the suffix. This is not trivial. And the way our internal passer works, it's written for a more forgiving and more open world. But people also use the routines like to verify and validate potential hostile requests. That sometimes fails because we are too open, we are too nice.

26:30 This was one of the other reasons why the discussion spark. It's also a bit rated to WebAssembly. I think we can postpone it for later, but there is an interesting interaction with how WebAssembly runtimes environments work and what's not going to work with your lip.

26:47 Yeah. And the web assembly discussion also comes back to what Brett was talking about for sure just a minute ago. So defining what is the standard library? Do you want to come back as well to that later? But let's talk about your Pep. Which one of you wants to introduce Pep? 594 removing dead batteries from the Python standard library?

27:07 Christian is the original author. I just PM did to the finish line. Told the Christians.

27:12 Yeah. I ran out of steam and brushed it, pushed it helped me to push it with the finish line. Yeah.

27:18 So what's the idea here?

27:19 The idea here is to remove things that are in our personal opinion and maybe with some reasoning no longer like super required in the modern world. So there are especially some parts that cost us lots of time and energy the most relevant library for that is like NNTP, like so the library to interact with new servers that we got your information. Like Usenet, Usenet, yeah, use. Net. So since we don't have a server implementation of NTP, so the use center only the client libraries we need to use like actual new servers for testing. So we need to connect to some servers and do some testing. These servers are no longer stable. So we used to use libraries like sites like Gmain a lot or one French new server that was stable. But sometimes they work like we expected to work. They are sometimes just not available or the network connection fail or get issues with TLS connections and these issues we're having blocked our whole CI chain. So when you do pull request you run your test against Windows and Linux on XXX 64 and on also macOS. So like think four platforms, but once the progress is merged it also kicks off the whole billbot farm. They're like 20, 30, 40 different platforms that run the test host, commit anytime. One of the NMTP servers had a hiccup. We have false alarms and indicating problem somewhere which were not problems with the code or with the test, but rather with the infrastructure that was failing and it was one of the motivations. While I wanted to remove an NTP lib.

29:15 How often are people depending on that these days?

29:18 One, two who's maintaining it? But it's a really interesting list you all have of these are the things we want to consider to remove. There is somebody who understands and maintains it. Now, a lot of them know for that, right?

29:30 Yes. That's another aspect of this, right?

29:33 It was things that were failing our test suite because the thing we had to rely on just wasn't stable enough or available, but our parts were just simply no one wanted to step forward and say I will make sure that this keeps working. In modern versions of Python and compiler, errors are dealt with and I will deal with any feature requests and bug reports and all that. Because the standard library before this Pep had numerically more modules than there are countries in the world. Right. The standard library is that vast and I don't remember if I even delved into sub modules of packages. I think literally the top level names top level. So there is an aspect here of maintainability, of just simply there are only so many core devs to handle the influx. Hence why we have a 1600 pull requests that are currently open. And at some point you just have to kind of stop and go like okay, who is benefiting from us carrying this forward and how much of a detriment is it to the project to keep it going? And some of these things it was just a question of is this useful enough to the world for us to put the effort into maintaining it? And having to make a call on some of them, and sometimes no one stepped forward and sometimes some people very much stepped forward. I totally rely on this. And then it became a question like, okay, is it just you that needs it, or is it a large or decent enough size chunk of the community that still needed it that it warranted keeping around and continue to support it? Because I did a number crunch the other day of certain subdirectories in the Git repo. And if you look at just straight code between the standard library and the interpreter, 60% of that is standard library.

31:21 Wow.

31:21 How much does it influence your feeling and opinion about something, whether it has some C component or if it's just pure Python? Is it easier to keep a pure Python thing around that just as sort of higher level doesn't require as much nuance? Or does it not really matter.

31:38 At least for this Pep? I don't think we really took that view specifically.

31:43 The management is more just a side effect almost. But this was mainly, as the title puts it, dead batteries. So when Christian brought this list up, initially it was I don't think these are useful in modern times.

31:55 Less than oh well, this is also written in extension modules, which makes it harder to maintain because you can't ask an average Python developer to come in and help maintain it. You have to ask someone who knows Python and C extension modules to come in and help maintain it. That was never the motivation directly.

32:11 Okay, yeah, but knowing cross platform networking code and C is technically, I would say, harder than knowing standard Python.

32:18 Yeah.

32:20 This portion of Talk Python to me is brought to you by Fusionauth. Fusion auth is an authentication and authorization platform built by Devs for devs. It solves the problem of building essential user security without adding risk or distracting from the primary application.

32:37 FusionAuth has all the features you need with great support and a price that won't break the bank. And you can either self hosted or get the fully managed solution hosted in any AWS region. Do you have a side project that needs custom login and registration, multifactor authentication, social logins or user management? Download FusionAuth Community Edition for free. The best part is you get unlimited users and there's no credit card or subscription required.

33:04 Learn more and get started at Talk Python. Fm Fusionauth the links in your show notes. Thank you to FusionAuth for supporting the show.

33:15 Let's first talk about the status accepted.

33:18 Right.

33:18 So this is happening.

33:19 The first PR to document the modules being removed got committed last week, and now you'll get to see the curtain pulled behind how Python has developed. I'm going to publicly ask Christian to review my PR to deprecate AIFC. I know he's busy.

33:38 Maybe he reviewed it before the podcast. I don't know. But the first PR to actually deprecate AIFC, the first module in alphabetical order is now up. So now just be cranking through them until beta.

33:49 I feel like I saw in the release notes here that the Python 304 was out and this has deprecated various modules according to this Pep. Is that right?

33:59 Yes. So the steering Council made the decision that it was useful enough to backport the documentation deprecation. So Python 311, obviously, because that's going to raise the deprecation warning. But 310 and 309 also document that these models are deprecated because we didn't want people coming in in a company where they're on Three Nine at the moment start using these modules and then be surprised in two years time or what have you when they upgrade to 311 or later and suddenly, oh, this thing I've been using is suddenly deprecated.

34:33 Oh, this went back farther than 310 because Three Nine is still accepting bug fixes. Got it.

34:38 So that's what that means. It's purely documentation. The code has not changed at all. It is literally just if you go to the module index or these modules, it will just have a big deprecated term on it.

34:49 Yeah. Quick bit of nomenclature, real time follow up, Mr. Hypermagnetic. What is pure Python if not CPython? Christian, you want to sort of disambiguate the CPython for Python that has C. So in Python Python library.

35:05 We have models that don't have dedicated C models that just written in Python code.

35:12 There are other models that have a mandatory back end. And see if you look at the SSL model, which I maintain, the binding sealed material written in C, and the public facing SL model is FSA, then ads on top of the CBAC. And this we would consider as a non pure Python package because it requires C Code. There are also other models we have in the past.

35:40 Data structures like list and so on. Right?

35:42 Yes. All the core data types are written in cycling.

35:45 They are present always. So we'd be just talking about like things you would import and use.

35:50 Got it.

35:50 I see.

35:51 Then we have accelerated modules as well, like Daytime, where they're implemented twice, once in pure Python and once in C. So, like, Python will use the pure Python version or any platforms that can't compile the C Code for whatever reason. And then there's the full C version for performance. So pure Python just means the entire chunk of code for that module that you import is written only in Python code. There's no C Code involved directly. Obviously, when you import can transiently cause that, but for sure.

36:19 Yeah. Christian, what's the idea? Are we just going to when 311 comes out, all the modules are just yanked?

36:25 330.

36:28 What's the plan here? It's not so immediate or so abrupt, is it?

36:32 No, it's going to take a couple of years. So 311, you'll get like the deprecation warnings? I think that's a plan, Brad, to just emit depreciation warnings so if you would import a model and have the warnings model enabled to show you deprecation warnings, then you would get a warning pop up even in your CI, turn the deprecation warnings into deprecation exceptions into a heart failure. And then you see, I would fail and inform you that you're importing something deprecated. And this will continue in 312 and 313, the models are gone.

37:11 Right. Okay. So that's over basically two years, more or less two year period, because 311 is pretty imminent, right?

37:18 No. October.

37:19 So 313 would be October 2024. Yes.

37:25 Sounds far off now. I'm sure it's not in practice.

37:28 Yes. And to be clear, if for some reason the community really spoke out very vocally across a large swath of people, we might consider postponing, I don't think there's anything here we would not yank. But if people need for some reason more time to move, we might consider holding off. But we're fairly confident that most of these aren't. And already, honestly, the ones that people really care about are already getting copied and put onto PyPI, so people can totally still get the original code. And to be very clear, you're already using code from Python, which means you're already using the Python license, the Psf license for this code. So copy and pasting this code is totally fine. And we encourage it. If you need this code. Right. Because it's deprecated, it's not going to be changing unless it flat out breaks. Because of some change in Python that requires an update, this code is more or less after this deprecation lands, that code is not getting touched. Which means you could totally copy what's in there just prior to the deprivation or what's in 310. Honestly. Because once again, these modules aren't really being updated.

38:28 These are the ones that are being basically ignored and just dragging along anyway.

38:31 Exactly. So you could totally go and copy the code over into your own code base, paste it in, save it under the exact same name, and it should more or less keep working the same way you'll have access to maintain it.

38:43 Yeah. There's kind of two path forward for people who are like, no, I really need AIFC audio file parsing or whatever that is. One somebody might decide it's really important and they want the CPython version as a pip installable thing that you can then add back into Python with a dependency. That could be a possibility. Or as you just described, you could vendor it, which means just copy the file and you just have a copy of the source code internally and it's just part of your app now.

39:10 Yes, exactly.

39:11 I have seen as an interesting backstory. So when I posted my first draft of Pep, I got contacted by a lead Technic director from DreamWorks Animation Studios. They wanted to keep the model in CPython. So Python is a heavily used in the movie industry. So if you see any Blockbuster. It's probably powered by Python internally. Even so far that Python just won a special price for the animation awards. Danny, what's it called?

39:47 That's right.

39:48 Guido got that award, right?

39:49 Basically got the physical award and some core desert Queen, Christian got requested certificates. And I believe, yes, Christian is getting hit.

39:58 Oh, that's fantastic.

39:59 The Iworks award. That's the second guy who invented Mickey Mouse.

40:04 For example.

40:08 Walt Disney invented Mickey Mouse together. Fantastic.

40:11 That's a cool honor. And probably an unexpected one from working on Python.

40:15 Chris and I have been doing this long enough that I think I've mentioned this on the podcast when I first got involved as a core device Python. I still had to explain to people what Python even was right. Or if I was lucky enough they knew what it was, they just would go, is that the language where white space matters, right?

40:28 Yeah, the one that's weird. Yeah.

40:31 So it was a totally different time. And now we're being used to help plan flight paths for helicopters on Mars and handing the video of the lander that made that helicopter take off and processing images of black holes and gravitational waves.

40:50 I just had an interview JWST episode that I have.

40:56 No, I never thought any of this would ever happen. So, yeah, having blockbuster movies have their entire management pipeline for their assets written in Python, it's always mind boggling where Python is and what it's powering.

41:10 It's cool.

41:11 It's not just all your account videos and photos on Instagram anymore or videos on YouTube.

41:17 Yes.

41:17 It's both amazing and par for the course.

41:20 And I can't believe I just said it's not just Instagram and YouTube anymore. We've also been there long enough that people realize we power huge websites.

41:28 Lots of stuff in the world. No, it still blows my mind constantly.

41:31 Yeah. Youtube was originally written in Python thing. It's still lots of parts of YouTube are powered by Python.

41:38 Last time I spoke with someone there, it was. Sydney out in the audience has an interesting question. So there's a list of things that are deprecated here. Some that were threatened but kept. They were on the list and they just maybe not. Sydney asked, might there be further removals in the future?

41:55 Maybe.

41:55 Yeah.

41:56 So here's the deal. This Pep was done because Christian, I personally think rightfully so, thought we need to do a bit of house cleaning and get rid of some stuff that was just kind of just sitting there rotting in the corner and not being left. Some things got kept because to keep the controversy low, Christian was very conservative with the list. Basically, if anyone step forward and say, no, please don't take that out. That was a Cordev. It more or less just got left. Very few things are on this list that people still push back and said we're removing even if the court Dev wanted to keep it. That being said, as I said, earlier in this podcast.

42:28 Brett's crazy grand plan here was to help Christian get this Pep done, get initial clean done. I started a conversation over on Python committees about how to maintain the standard library. What does it take to add a module and to remove a module? Because that's actually not very clearly stated either.

42:45 It's been very just kind of open from my perspective. From the outside of what defines the standard module is just write only, like stuff only gets added and it's defined to be what's in the shipping version of Python. That's kind of an implicit definition.

43:01 Right. But how do you get something in there? Right? Did you know that graph flip was added by a couple of Cordevs just because they thought it was a good idea? Yes. No one asked or anything. That's how it's been historically maintained. It's very much just an open thing.

43:17 And there's a proposal to kind of make it a bit more structured, and you could argue more rigid, depending on your view of this whole process, more modern, maybe by suggesting that you need a Pep to add something because it's a shared cost to all the cordevs that we have to maintain it. You'll notice I have a slight theme here of maintenance costs.

43:38 I have a puppy for you.

43:41 One that's fluffier, one that I'm going to have to pay vet bills for clean up the two.

43:45 It might ruin your carpet, but it's also cute.

43:47 I've got a cat. I'm good.

43:49 It's not the joke. So just watch Brad's keynote from PyCon.

43:53 Like three or four years ago in 2019.

43:56 Maybe it was the first one in Cleveland. Whichever one that one is.

44:01 I can't remember 2018, I think.

44:02 I think it's 18.

44:03 Yes.

44:03 Once that's settled, my hope is once again come up with a policy of what that means for the standard library. Once we have that policy, there will probably be a discussion about what does that mean for the current standard library. Do we leave it as is? Do we maybe slowly transition over or not? I don't know. Some people like Lukas, for instance, have advocated never remove anything from the standard library ever again.

44:25 And literally just saying this is deprecated. We will never touch it or update it. It's dead, but it's in here, so we don't break the code.

44:32 Other people are way more let's strip it to its bones.

44:36 And if you need this stuff, we'll make it as a separate package or something. Some people talk about keeping as is, but separating the development process. So it's actually an externally maintained thing and it has its own release process and schedule and all that. The answer is, I don't know, but this is not the first time I've deprecated modules and had them removed. I was also in a competition with Fred Drake back in the two to three transition to see who could remove the most number of lines of code in Python, and I won, thanks to removing the compiler package. So I have been around long enough to say probably I just don't know when, just because I will never say never when it comes to a project that's 32 years old. But there are no specific plans right now outside of this Pep of deprecating or removing anything else from the standard library. So if that's the question, concretely, no plans. But philosophically, I am not willing to say anything.

45:28 So one of the things that stands out, let's talk about the modules real quick. But I want to kind of talk a broader thing because I think it's going to lead us down a path. Maybe just give us a quick overview of the highlights, let's say things being removed. I talked about this table here. It's under the deprecated modules heading on the Pep, and it has a module name when it's deprecated. Some of them are like deprecated in three, six when it's to be removed, when it was added, whether it's a maintainer. And Interestingly, I thought this was cool. There's an alternative that's new or better and maintained that you could just use, right?

46:07 Yes. If you look at the table, you'll know, some of the stuff has actually been somewhat documented deprecated all the way back to Python 2.03. You also have to understand some of the stuff got documented don't use anymore. A long time ago. We just didn't take the code out. And this is more of a push to actually finally remove the code and the documentation.

46:25 I see.

46:26 But yeah. So we have AIFC, which I believe is the audio format.

46:30 Correct.

46:31 We have Async Chat and Async Core, which were very early Async server things in the center of library that you shouldn't use, obviously, because it's on this list.

46:41 Yeah. An alternative is Async. Io, which was added in three, four.

46:45 Yeah. It's actually Asynchronous.

46:46 I think the sync check as in Core were added for Zoe back in the day.

46:52 I know that the Zoe used to use as in corresponding check a lot.

46:55 I don't remember what the hell audio app is.

46:57 Audio up is low level like conversion and math operations used by AFC Wave and some of the other sound models. So this just provides math primitives.

47:08 And CGI is literally what sounds like it's helpers to write CGI scripts.

47:12 That sounded like one of the bigger, harder to work with ones like there's no maintainer. It was designed at a different time.

47:20 Yeah.

47:20 And the funny thing though is a lot of packages still use bits of it because there's weird little helpers in there. But if you also go look at what the helpers are doing, most of them are either one liner or they delegate to something else somewhere else in the center library at this point. And they have horrible APIs now because they very much expect it to be CGI. So they're reading from environment variables and files to get the things to process versus passing them as an argument from SDD and SDD out.

47:48 It's pretty much hot code.

47:52 Yeah. So there's actual pushback on CGI, but I think when we pointed out like this just delegates to here or this is literally a one liner that I can paste in when you're in the discussion of what it does, like just copy and paste this register literally or just whatever. Most people I think we're okay with it. This is actually, I think one of the first modules to get put up on PyPI like a couple of weeks ago, CGI TV is trace backs. Pretty trace backs in your CGI code. I mean, once again.

48:18 CGI code you don't need one goes the other comes with it.

48:21 Yeah. Chunk, I think is literally just chunking data. It's literally just breaking up into chunks a little bit.

48:26 Like editor tools.

48:27 Yeah, I think so. If I remember correctly.

48:28 It's a format to distribute fault used by mailbox correctly.

48:36 Yeah. I will let Christian talk about crypt.

48:39 Yeah. Crypt is a binding to the Ellipse crypt function is used to password hashing around. The problem with that is that the only guaranteed algorithms available is horrible, and if you need some of the better ones, they're probably not available in your lib C. So there's like optional algorithms for password hashing and even there are problematic. So use one of these replacement ones.

49:07 They're much better. Yeah.

49:08 Like crypt or passlib or anything else doing it. Right.

49:13 Image here are a just a very limited approach to detect if you have a PNG.

49:19 Jpg, whatever based on the first couple of bytes of file.

49:24 The same with audio headers, sound headers. They just detect falls for mods but they only support a very limited setup file formats. And there are better libraries that are more efficient and support much more different file for us.

49:39 Yeah. So after image header, MSI. Lib, which helps with Windows MSI installers. Okay, why do we have this in the standard library kind of question. I think we used it originally to help write the installer for Windows, and I think we don't use it anymore, so it's no big deal. We already talked about NNTP Lib NISS.

49:57 Who still uses Nas Sun yellow pages.

50:00 Yeah, exactly.

50:02 Do people even know what sun is anymore? Right. We're getting to that point in our lives.

50:06 I think it's a synonym for Oracle.

50:08 Yeah, exactly. Oss audio Dev is just a wrap around an audio library. Once again, does that really belong in the standard library?

50:15 It was used by Linux before they had also before they had Pulse audio and pipe wire. So this is like from the 2000s.

50:24 No pipes library, more or less got replaced by subprocessors Unix pipes. Smtpd. I don't think we need to be able to run a server to send email and Python standard library. So that's why that's there.

50:36 Yeah. And there's PyPI alternatives.

50:39 Yeah.

50:40 Send header.

50:41 We talked about STWD it's the binding to the Etsy shadow file. So that's the place where Linux actually keeps the passwords for users.

50:52 But that's the wrong approach. So if you want to log in a user, you don't check the password. You ask the Pem stack if the user is allowed to log into a service is another audio format from Sunlight server.

51:10 This was specifically to be a server, not a client of Telnet.

51:13 It's like client library. Sorry, the client library. I don't think you'd be able to serve it, but actually I would have to open the code to read the code. Uuu that was used by UCP. So Unix to Unix, copy, chunking format, transfer, binary data, and Xdrlab, another sun library, that is the binary format used by NFS.

51:37 And remote procedure calls if you have some remote procedure calls for network fault servers from 1992.

51:46 Indeed, there's an interesting high frequency of sun being mentioned here.

51:51 Yeah.

51:51 I mean, Python used to run cleanly on Solaris right out of the box.

51:54 Yeah. And sun developed lots of Internet standards we still use these days.

51:59 Sure.

52:00 So hopefully all the listeners were shocked and had no clue any of these modules existed, and hence why we're getting rid of them.

52:06 Where's the threatened list? Down here somewhere. The modules to keep.

52:11 Yes.

52:12 So we got colorsys file input, get opt, and wave. Those are the ones that avoided being cut.

52:21 Colorsys, to be Frank, is just basic math. It's nothing special. But Cordev said I use it, I'll keep it around, I'll maintain it. Okay.

52:30 Parsing RGB color. Yeah.

52:32 Fileinput. I don't remember about file input. I think someone just said I use it. Get opt and op parse. Where once again, is it worth getting rid of versus Aug parse?

52:42 And is it why they use it enough?

52:46 There's just too much code out there right now to warrant ripping it out. So we decided it's easier to keep and Wave plays W-A-V sound files. And basically, it's used by education. It's a quick and cheap tool to show kids computers can do fancy stuff like make noise.

53:04 That's actually really nice if you got your turtle moved along. It could laugh when it gets to the end of its little thing or whatever, right?

53:11 Yeah, but I mean, that's another good point. Like, oh, turtle. Well, that requires a shipping Tk inter, which requires a shipping tickle Tk. And it's like where you draw the line.

53:22 But it's turtles all the way down. You know that.

53:25 Hey, and as someone who learned turtle back on an Apple II E with a good old green and black screen, I totally get it. But these are the questions we're starting to ask ourselves, right? Like, Where's the line?

53:36 Yeah.

53:37 All right, before we run out of time, let's have a broader philosophical conversation about this stuff. You said you have a big plan.

53:46 One of the things that I think is interesting about Python is where it runs in all these different places. I mean, you've got Mac, Windows, Linux, but you also got Raspberry, Pi's, you've got helicopters, you've got robots. And then you even have circuit Python and MicroPython where it's like really small. And I do think it would be interesting to say here is a subset, a portion of Python and a portion of maybe even the language syntax that if you program to this, you're guaranteed to be able to run it everywhere. Is there a way to agree? Maybe that compiles the web assembly and runs in the front end on a browser? Who knows?

54:25 So for any of you who read my blog, you'll know, I've been doing a long running blog post series on the syntactic sugar of Python, and part of the reason I've been doing that. And you'll find out more when I write the concluding the post about this. But basically I've been trying to figure out what I've mentally be calling the minimum viable Python. What is the core set of Python constructs that must exist to basically reconstitute all the other parts of Python in its syntax? And kind of if we were to start from scratch, almost, what would that look like? Now that's interesting from a language level, but from a standard library level that comes into play only in terms of support. Right. Like, we dropped this feature. Would that break things? Right.

55:07 Like name Tuple.

55:09 Right. Well, for example, does Tk enter need to be in the web assembly front end version? Probably not, right?

55:15 No idea.

55:16 I guess not. But maybe. Probably not.

55:18 Yeah. I mean, that's the tricky question, right. It's always one of these things of who are you optimizing for, who's the target audience? How do you define that? Right. Other than stuff that we have to have just to make CPython run, nothing technically has to be in there. Right. Some have said, oh, we should have enough to at least bootstrap in Pip. So if you can at least get pip or some installer in, you can then at least get yourself going and start installing dependencies. But as you mentioned earlier, it's nice to be able to also write that little automation script that helps maintain your system and not have to pull anything and have to create a virtual environment for every little script you ever write. But Where's the line on that? Yeah.

55:57 When I saw Python on Linux on just like a Bare server, I can apt to install Python three. But maybe that's not enough. Maybe I need to also install sometimes Python three Venv or it's like broken up into bits. Right.

56:14 The deviant question is a very hot topic with Christian in the room.

56:19 I personally argue against Debbie's policy on that and say Venv would be considered a core part. Right.

56:26 I think it's core as well. I wasn't so much picking on them as just like there are environments where it comes in pieces to some degree. Okay.

56:35 I need to find the right words to not get into trouble again. Or last time I'm careful. Somebody accused me to wage a crusade against Debian.

56:49 It's a philosophical disagreement, basically.

56:51 So Debian has the policies to split packages in smaller parts so they can have a minimal installation.

56:59 And they also don't like the way how we have virtual infinite surp provider because it's actually duplicating. Because we have like a zip file of binary wheels with compressed setup tools Pip, which contains vendor packages, but Debian wants to use their distribution provided version of Pip. So it is a conflict. Like we from the parts of the core developers. We think that the usability is more important here. So we want users to be able to use documentation instructions to use Python install extensions. This is more important for us. And this say a debate which had like multiple sessions between Debian engineers and the steering Council. There were several heated discussions.

57:53 There are just different philosophical detections.

57:57 So the key point is there are a lot of questions about like if you start to slice and dice the standard library into parts that you can install in groupings, for instance, is that true part of the standard library or is that just an optional install on pipe that the core Dev team just happens to maintain?

58:14 Right.

58:14 Because you can no longer rely on it. Yeah.

58:16 I mean, if you went down that path, it would almost be a little bit like the Anaconda story right? Here's the essence of what I got. But if I install what I normally install, I get more. There'd be like a minimum install and like a Fuller install, which I think. I don't know how I feel about that. Well, if it gets me Python on the front end, I'd have to write JavaScript. I feel pretty good about it. I'm starting to think I don't know.

58:38 For somebody it makes kind of sense. Let's go back to Tikinter. So Tikinter depends on tickle decay, which depends on your graphical user interface. So X Eleven and if Python would always require tickleTK, then you would have to install, I guess like 100 megabytes of libraries on every container image that has Python, because the whole X Eleven stack is very big. So you get your whole graphical use interface libraries down to the actual graphical server that renders all the output. It makes sense that most lenders distribute split up the tkenter with an optional package, but some distributors go bit further and split off packages that we consider to be core packages.

59:30 So the inner core that should be always available and working like daytime and two balls and stuff like that.

59:38 Stuff that has no third party dependencies, basically.

59:40 Okay, interesting. What about web assembly? You guys started actually building for web assembly, didn't you?

59:46 Yes.

59:48 Steve Dower pointed out that I don't remember the way to look it up. Super easy, but that there's a proper core Dev version of web assembly CPython.

59:59 Now it's recall easenhs me. I just posted chat with the chip wondering.

01:00:06 Yeah, basically what happened was at the core Dev sprints back in October, I started to look at seeing what it would take to compile CPython to web assembly. And I was lamenting in our build channel on our Cordev discord server. That all right. The build setup was kind of old and creaky and I needed some help to figure out what to do. And then Christian, working for Linux distribution, knows how all this stuff works. And so he started to answer my questions. And then I did some initial clean up and then Christian really got into it and just totally started to clean up our entire build process in terms of modules and such, because there's a setup file in the repo where you can specify what modules you do or don't want built in to Python. Both compiled it all into like a SL or DLL, compiled into statically into the binary or just completely left out. And Christian went through with some help with Ireland I think actually, and cleaned up a bunch of the whole structure so that we were using more of Pkg config and just made it just a bit more modern. And while he was doing that, Chris would bring up stuff about cross compilation, because that's another thing we do is you do a cross build, right?

01:01:19 You could totally build on your Mac right now, Michael four X 86 64 bit, even though you're running an M one.

01:01:25 Oh, interesting.

01:01:26 Okay, so it's just flagged.

01:01:28 Yeah.

01:01:28 But the way you get web assembly building that way is you just specify typically in scriptin as a C compiler, which basically is Claying, and you just specify and basically under the hood is more or less just specifying the right flags and everything to just make Claying build for a different CPU target. And then Christian was starting to talk about like, oh, well, for cross builds, could we do this and that? And I kept saying, no, it won't work for web assembly because of this and it won't work for websites because of that. And then Christian just looked into it and like, oh, I'll use this as a motivator. And then Christian just ran with it and Christian figured out all the problems and just got really into it. And it's been patching CPython's main over and over to the point that as of less than a month ago.

01:02:06 So for two weeks now we can run the entire test suite without any favors on web assembly inside. Nodejs, so you compile these a bit earlier? So when you build Python for WebAssembly, you have to not only tag WebAssembly but also like the runtime. So these things are called triplets so platform triplets. So you have like which CPU you target, which vendor and operating system, and additional sites like GDP and default target is Webb 32 and scripting. So you do compile to WebAssembly CPU instruction for 32 bits. And M scripting is the runtime platform, and M scripting can target multiple different platforms. So this is a scripting for the browser, but you can also compile it that it uses NodeJS back ends and NodeJS by filesystem access.

01:03:07 So we can actually have more permissions and more capabilities.

01:03:11 You can run threads, you can run sockets in a limited way, and you can access the file system.

01:03:17 So there was a process of running the test suite, figuring out why it's crashing. So we had in the beginning completely recrushing the runtime, not just itself, but there's like a virtual kernel layer written in JavaScript that provides syscall written in JavaScript to the WebAssembly model. It's a bit weird. So I was writing Colonel like code in JavaScript and the unit test code for that in C.

01:03:47 It's em scripting.

01:03:48 Very cool. Yeah. So this repel.ethanhs.me, which I'll put in the show notes, it's got my Mscripton version of Python 311 and I can do F string stuff. And the things right here in the browser hasn't been updated in a while.

01:04:04 It's been old. Need to talk to Ethan.

01:04:06 It's like six weeks old. That's old for the Web, I guess.

01:04:09 Well, I mean, we have a Cron job running on Ethan's github repo for this that runs nightly against Python. To buy that itself, to make sure it continues to compile, we need to get a build bot going.

01:04:23 Microsoft agreed to fund one via Azure, and I haven't talked to Christian about it yet. But we'll figure out getting a build bot going. And then part of one of the crazy grand plans around Python WebAssembly is we're defining an official platform support for CPython right now in Pep11. And my hope is we are going to get it so that Chris and I are listed as the maintainers of the Web Assembly support in CPython, and we'll get it listed as an official tier two, what we're calling tier two support platform, which basically means it's backed by two core devices at least, and a build bot. But the idea of tier one is something the entire core team supports is based on continuous integration.

01:05:06 That's really exciting.

01:05:07 Yeah. One of the extra nice things about WebAssembly is we could theoretically even get that to a tier one, because WebAssembly is an abstracted assembly language, but it's not CPU dependent. So we could actually compile it on GitHub, Actions and test on that just as equally as Mac, Windows, or Linux and have it still be fully tested and work on CI.

01:05:26 You can take this idea really far, right? Like suddenly we can start having executable code examples in the documentation for Python because we've got a runtime now we can let me make that work.

01:05:35 Yes. Because you don't have to worry about security or compute costs because it's in your browser, hack yourself and it's your compute.

01:05:43 Right. Selfishly for me at work when this works, I can start talking about shipping the node version of this with the Python extension for vs code or as its own extension. So you don't have Python installed. Install this other extension that's going to come with Python compiled the WebAssembly and you'll just have it or VScode.dev browser version. We could potentially start chipping that with VSCode. Dev and have the Python extension pull that in. And now you'll even have a runtime in the browser that ties into what I think is a nice editor. So there's a lot of possibilities here that we really want to get going with. And then when you start talking about Wazi right, then you start to get the edge compute. Christian just got that working. Like literally I think last week, I don't know where it stands yet.

01:06:26 Working again.

01:06:26 So the first version was like hacking and patching lots of things out.

01:06:31 Oh, yeah. All the piece redwork. Yeah.

01:06:33 And the new version week ago was using a library from a company is also working on Python support. They like Dubs and hacks.

01:06:43 Single store apps, right?

01:06:45 Single store apps. Yes. So it's even more restricted and limited compared to M script. Then you don't have any support like for threading, but you can't compile Python without at least the threading library headers. And they just had like look around for that too. So you can't spawn thread. So if you start trying to start a thread just failed, at least you have enough of the CIPA available to compile.

01:07:13 That's super exciting.

01:07:14 Just to be clear on that, it's not a completely new effort. There's actually a productive version of Python for representing in the browser called Payo Diet, which is also now used by Jupyter Lite. So Jupyter Lite is Jupyter notebooks on top of Pyodides. Yeah.

01:07:33 Jupyter lights looking really interesting.

01:07:35 Yeah.

01:07:36 And we actually met with the Paradigm team and more or less what we came to an agreement seems a little weird to phrase it this way, but basically an agreement where we will keep trying to make it so that the main branch and CPython can compile clearly to web assembly. And that frees them to focus on JavaScript API to tie into it and on getting the scientific stack compiled over to WebAssembly. Because you have to do some Hecky bits.

01:08:03 Right. Because you got to compile like map plot, Lib and stuff because it's got C in it. So you got to compile it over there. I don't know, some of those like NumPy. Definitely.

01:08:10 Exactly. Because traditionally they had to do all the patches and all the work every Python release to get it working. And Chris and I are just in a better position to be able to keep that up and going. So we just said let us handle that part. You don't have to worry about patching anymore. You can focus on the stuff that's more unique to WebAssembly and what you're trying when you're able to be good at over there. So that's kind of where we're dividing up the workload here. We're just going to keep main working and then they get to stop worrying about that once they start worrying about Python 311.

01:08:35 So one of the first things that Ethan and I did just Beto the patch stats that were developed by pilot developers and adjusted them and made them compatible so we can actually merge them and have them in the upstream code. So lots of their patches was just replacing things, but not a way that would work if you would build the same source code for standard drive. So they just hacking around and getting things working, which is totally okay for their approach. But you couldn't just take their requests and their much narrow use. So we had to make them in a way that this would work for standard private website.

01:09:12 Yeah, fantastic. Well, two things bring to mind for me when I hear this. One is does this mean I could write Electron apps and just have the slightest Shim that then just lets me run like a back end in Python JS side of that in Python, which would be awesome.

01:09:32 It should, because electronic chips with Node. So as long as you have access to the Node runtime. Yeah.

01:09:37 And you just get it on top of there, and then instead of writing all your sort of core logic of your app.

01:09:42 You'll have to shimout back to JavaScript appropriately and have those APIs which you might want Pyodite for potentially. But yeah, basically, yes, that door is open.

01:09:51 Yeah.

01:09:52 I don't know ElectronJS well enough to say, oh, I could totally just go do that. But I see that as a possibility. That would be awesome.

01:09:58 What we don't have probably won't add in the near future is any kind of bindings to JavaScript. So you can run like Python in your web browser as like an isolated process or web broker. But communication back and forth between the outside world in the Node js or browser world and Python that's provided by Python. But we don't have that right.

01:10:24 Very fun and great talk by Katie Bell. And she also helped us to kickstart the Port. She gave a talk at PyCon several months ago.

01:10:37 It was like ten minutes introduction, two minutes compiling Python to web assembly, and 28 minutes gets STD in working.

01:10:48 Std out working took like half an hour of our talk because that's surprisingly, surprisingly hard with WebAssembly.

01:10:56 We kind of made the decision that at least for now, we are not interested in trying to develop that FFI back and forth with JavaScript and having that opinionated API. And that's, as I said, kind of the agreement would come up with Pythagore right now is they already have an API. We're going to lean on them to provide that for now at least, and probably will continuously. I don't really necessarily see us changing this. And we'll just provide the lower level functionality of just the runtime.

01:11:21 Right. Yeah. But if you all built that, it's a great foundation for other people to run with.

01:11:25 Exactly.

01:11:26 And then the other one.

01:11:28 First of all, is this the talk that you're talking about?

01:11:31 Yes, that's the one.

01:11:32 Okay.

01:11:34 I'll put that in the show notes. The other one is the right URL.

01:11:41 Yes.

01:11:41 Is this Blazor project from the C#.net team?

01:11:44 Yes.

01:11:45 This is .net, the .net runtime as a front end framework running in the browser, which is pretty awesome. And it seems like there's just so many parallels that could be brought to the Python world that you don't have to do Angular. You can just do Pyblazer. I mean, that's not the name, but you know what I'm saying, it's closer to possible. And I think this is actually something that would be really neat.

01:12:10 Yes. The trick here, and I think Blazor does the same thing right now as well is from my understanding, Blazer has a WebAssembly compiled .net runtime that gets loaded into the client, and then that's how the execution, if I understand how Blazer works correctly. And so you could do a theoretical same thing with Python. Now, obviously there is size issues to consider here. Right. This is not something you necessarily on a really slow connection somewhere would want to pull down, because I think the runtime is it five Megs now, Christian, or is it three?

01:12:42 So the uncompressed wasm is like six megabytes gets down to four, and then you have the big bundle, the data file that contains the Python compiled files. It's another five, four, six MB.

01:12:57 It's big, but actually it's from small. Okay.

01:13:04 It's descended library, Python data, and Python weather. Yeah.

01:13:10 So it's three megs for the center library, 212 KB for the wrapper, and you already cached the wasm.

01:13:19 Yes, we can probably make a bit smaller.

01:13:22 So currently that comes like with elementary decimal model and all the hashing libraries compiled in. So we currently create one gigantic binary. There are ways to have side models like Chef libraries with wasm. It's a bit more complicated to get that right. And also it increases the size of the main binary, the main model a bit.

01:13:47 I didn't go into that yet. It's something I look into maybe in the future.

01:13:53 But as you're addressing, talking about earlier, Michael, about like slicing and dicing the standard library, if you're shipping the runtime, you can also shrink that down by just dropping all the parts of the standard library you don't need. Right. So if you run like module. Finder or some other script that's going to go through the standard library to figure out what you do and don't need, you could actually just compile your own version of Python CPython for WebAssembly only with the standard library. That you want.

01:14:14 Interesting. You can get it really small.

01:14:16 Right.

01:14:16 So which of these three Python JS Python data Python WASM has Ceval C in it? The wizardfold the Wasim one. Okay, so that's two Megs, and then plus the standard library bits. Right?

01:14:27 Yeah. So it's a little over five Megs in total for everything, which isn't huge, like an Internet project that you might need. Totally fine for electron apps, as you see it's.

01:14:38 It would be totally fine for a single page app. It'd be fine for Gmail.

01:14:41 Yeah. So they're going to be up and running constantly and not change. Yeah. It's within the realm of probably what Facebook's already downloaded onto your machine right now anyway. Yeah.

01:14:50 Just run an ad blocker. I'll do less.

01:14:54 All right.

01:14:54 I think we are well over time. But you guys, it's always great to talk to you. Yeah. Good work. I dream of PyBlazor happening, and I also saw a nice, interesting comment that triggered a thought it's progressive web apps are not really a thing we can do in Python very well because for a progressive web app, it's really got to be all offline in some meaningful way.

01:15:17 But if you could bring Python WASM down and use local DB, there might be some really interesting ways that progressive web apps become way more interesting to us as Python web developers.

01:15:28 Okay.

01:15:28 Now I have to confess, I have no clue what the progressive web app is. I don't know anything about your web development.

01:15:35 I got Python working on the browser, but I don't know how browsers work. Awesome.

01:15:38 I will give one that's great Teesdale. Michael. I had too much patience for getting this whole web assembly thing browser mobile because every phone has a web assembly runtime, thanks to all the browser.

01:15:50 Yeah.

01:15:52 Okay. Now I'm also very interested. I would rather have a mobile option than offline than a front end web option. So there we have it. Although these may well go together, honestly.

01:16:03 All right. Well, thank you both for your hard work on this prep. And then it's super interesting to see how it kind of ties back into this, like more focused run times and more places.

01:16:13 All right, before you get out of here, really quick last question I'm going to ask you. Just keep the one, since I kind of know the answers for the other. I'm pretty sure. Anyway, notable PyPI package you want to give a shout out to before we dip?

01:16:23 Oh, man.

01:16:25 There is one that runs web assembly, like something you can have, like Python, that will basically interoperate with any web assembly one. I can't remember quite what that was called.

01:16:34 Though, I think wasm in Time. Time provide has one.

01:16:37 There's a couple of people who've posted stuff to PyPI that let you load WebAssembly code and actually run them. And I think WASM time. wasm time is a WebAssembly runtime run by the Bycode Alliance and usually the most cutting edge of all of them. So if anyone wants to play with isn't looking for a runtime WASM time is probably a good one.

01:16:56 Python embedding of plugin.

01:16:59 You can ask Chris what his favorite editor is.

01:17:01 Yeah, the right one. The good one. He used a couple.

01:17:05 I use PyCharm, I use VS Code. I use them.

01:17:07 So depending on use them all. Yeah. You've got a whole survey of the whole spectrum.

01:17:12 That's awesome. Cool.

01:17:13 And then notable package. You want to give a quick shout out to before we get here?

01:17:17 It's better late.

01:17:17 I don't know the path of the standard library.

01:17:19 Yeah, the lack of certain things in the standard library. It's like an anti module. The fact that you guys are taking the anti gravity model is also fun. Yeah, sure. All right. Anti Gravity pip install anti gravity now maybe it's in the center of library. I'm going to give you all up here on the web assembly.

01:17:36 One that would not work.

01:17:45 So you can't run any processes.

01:17:47 I know for people to say I tried to import antigravity inside the Wazim Rebel. It didn't do it. Not yet.

01:17:55 I can do it. Yeah.

01:17:56 I don't know. Maybe you can. It should. They all should. All right, guys, thank you so much for being here. I'm excited to see this progressing. See it leading to good places.

01:18:04 Thanks very much, Michael.

01:18:05 Bye bye.

01:18:06 Bye.

01:18:06 See ya.

01:18:08 This has been another episode of Talk Python to me. Thank you to our sponsors. Be sure to check out what they're offering. It really helps support the show. Starting a business is hard. Microsoft for Startups Founders Hub provides all founders at any stage with free resources and connections to solve startup challenges. Apply for free today at Talkpython. Fm foundershub. FusionAuth is your authentication and authorization platform built for devs by devs. If you have a side project that needs custom login and registration, multi factor authentication, social logins or user management, then download Fusionauthcommunity Edition for free. Check them out at talkpython.fm/ Fusionauth/ Want to level up your Python? We have one of the largest catalogs of Python video courses over at Talkpython. Our content ranges from true beginners to deeply advanced topics like memory and async. And best of all, there's not a subscription in site. Check it out for yourself at Training.talkpython.fm Be sure to subscribe to the show, open your favorite podcast app and search for Python. We should be right at the top. You can also find the itunes feed at /itunes, the Google Play feed at /Play, and the Direct rss feed at /rss on talkpython.fm.

01:19:24 We're live streaming most of our recordings these days. If you want to be part of the show and have your comments featured on the air, be sure to subscribe to our region YouTube channel at Talkpython.com/YouTube. This is your host, Michael Kennedy. Thanks so much for listening. I really appreciate it. Now get out there and write some Python code.