Every year, the core developers meet to discuss and propose the major changes and trends in Python itself.
This invite-only conference of about 50 people happens inside PyCon in the U.S.
Because it's private, we rarely get detailed looks inside this event.
On this episode, we have Seth Michael Larson here to give us his account of the sessions and proposals.
It's a unique look into the zeitgeist of CPython.
This is Talk Python to Me, episode 475, recorded August 22, 2024.
Are you ready for your host?
You're listening to Michael Kennedy on Talk Python to Me.
Live from Portland, Oregon, and this segment was made with Python.
Welcome to Talk Python to Me, a weekly podcast on Python.
This is your host, Michael Kennedy.
Follow me on Mastodon, where I'm @mkennedy, and follow the podcast using @talkpython.
Both accounts over at Fostadon.org.
And keep up with the show and listen to over nine years of episodes at talkpython.fm.
If you want to be part of our live episodes, you can find the live streams over on YouTube.
Subscribe to our YouTube channel over at talkpython.fm/youtube and get notified about upcoming shows.
This episode is sponsored by Posit Connect from the makers of Shiny.
Publish, share, and deploy all of your data projects that you're creating using Python.
Streamlit, Dash, Shiny, Bokeh, FastAPI, Flask, Quattro, Reports, Dashboards, and APIs.
Posit Connect supports all of them.
Try Posit Connect for free by going to talkpython.fm/posit.
P-O-S-I-T.
And it's also brought to you by us over at Talk Python Training.
Did you know that we have over 250 hours of Python courses?
Yeah, that's right.
Check them out at talkpython.fm/courses.
Hey, Seth.
Welcome back to Talk Python to Me.
Hey, Michael.
Awesome to have you here.
I'm really excited to get a look into the zeitgeist of the core devs and the people building Python for us through the Python Language Summit.
Yeah, let's do it.
Let's do it.
So we're going to talk about the 2024 Language Summit that happened in Pittsburgh.
It's like an embedded mini conference inside of PyCon, which is smart rather than trying to travel all over.
But before we get into that and all those things, I know you've been on the show not too long ago, but for those who may have missed your introductions, you know, who are you?
What do you do for Python these days?
I'm Seth Larson, and I've been working at the Python Software Foundation for a little over a year now as the security developer in residence.
And so that means that I do a lot of stuff related to security just for the entire Python ecosystem.
That's CPython, PIP, packaging ecosystem, like outwardly facing things for PyPI, maybe not as much the internals.
I leave that for Mike Fiedler, the PyPI safety and security engineer.
And I maintain a lot of open source projects specifically in like the HTTP and internet space.
So like requests, your lib3, trust store, things like that.
Oh, awesome.
Yeah.
Thanks for everything you're doing there.
And how's the role working out?
I know this is one of you were the first person in this role, like officially, right?
Is that true?
That is true.
Yeah.
I was the first security oriented hire at the PSF.
It's been going really great.
I mean, I feel like we've made a lot of improvements and there's a lot of exciting stuff that I'm working on today.
And I don't know, I think one of the things that also got highlighted, because this role exists, it just means that more people at the PSF and in CPython core team and just in the Python ecosystem in general are talking more about security.
And like, that's just as important as the stuff that I'm doing day to day is that it's just, there's just more awareness of what's happening in security.
Yeah.
So I have two polar opposite thoughts here.
One is I'm really surprised how few significant issues there are in Python, CPython, you know, the interpreter, the runtime, the standard library, all that.
It's really rare that you get a red light splashing.
Oh my gosh, go patch your systems now.
Right.
There's sometimes really minor things like this audit trail is not completely followed under this condition, but that's not a, the pager goes off and you know, cause you got, it's now a race.
That's one thing.
So that's awesome.
Right.
That the other is PyPI, typo squatting, all the, all the stuff that makes Python extra good, the half million packages and other shady things people do.
And we're going to talk about this in the broader sense, not PyPI, but you know, in the open source space of like, well, what if, what if somebody took over a GitHub account for a little while or, or something?
Yeah.
So, and that, that's not on fire, but that's, there are battles being waged actively there.
I would say, right?
A hundred percent.
So it's a contrast, right?
There, I can speak with confidence that there is malware on PyPI right now as we speak, but like, yeah.
So that's, that's the case, right?
Um, I think it really has to do with CPython is an incredibly mature project in like every sense of the word, like governance, security, design, all of these things.
Right.
And like, there's just a huge amount of people and resources being, being work, like working on CPython at any one moment.
And like contrast that to the, you said half a million, you know, I tend to focus on the like 95% of downloads by totality.
Right.
Because there, there's a lot of projects on PyPI that maybe don't have the supply chain criticality of others.
Right.
But yeah, absolutely.
A hundred percent.
And when you, I think when you hone in on that smaller window of projects, it ends up being a much better picture than,
I think it's, I think it's a hundred percent good picture.
Honestly, there are legitimate bugs that people have to deal with.
Like, you know, maybe there's a Django release that says we didn't validate the CRF token in, in this particular form.
So you should update your Django.
Right.
And that's, that's just a legit bug.
That's not people attempting to do bad things.
I mean, I can't even take really any credit for how secure and mature CPython is.
I'm such a late addition in the life of CPython, and like PyPI projects.
And so it really is the community.
I think it is just, there's so much investment and so much love and care happening in all these projects that it does speak volumes.
But that, I mean, that doesn't mean that we need to think that things need to be perfect or that it's necessarily a bad thing to have a vulnerability in a, in a project.
Because there's projects that are even more mature than CPython that have vulnerabilities all the time.
It's totally normal.
I think the important thing is just like knowing what to do when that happens.
And that's where I come in.
Yeah.
I don't want to belabor this because you know, the security angle.
Wow.
Interesting to me and central to what you're up to is not the topic, but I do think it's interesting that, the white house recently came out.
I think it was the white house.
It said we recommend Python and a couple other languages.
We basically, we, we recommend memory safe languages, which did you see that post?
You must have, right?
I certainly did.
And I actually had a, a big part of recommending that to the white house.
So when the.
No kidding.
I had no idea.
Yeah.
So the, the PSF, we responded to the request for information that CISA put out, or the office of the cyber director put out.
Yeah.
I think a year and change ago.
And in that post, we recommended CPython and Python in general as a memory safe language and went into all the details about like, yes, it's written in C, but like the language itself.
What people will actually be programming in is Python.
And we also emphasized how Python is a bridge into memory unsafe languages.
And because of all the focus on performance lately, it's actually like sometimes in some cases more performant to keep the code written in Python as opposed to writing it in C.
And so we emphasize a whole bunch of stuff.
And so I think that that had a, a small percentage of, of the reason why that was recommended.
Put it, put it, put it on the radar and positioned it correctly.
Maybe.
Exactly.
Awesome.
Well, congratulations.
I was just going to say that's kind of interesting, but I'll join more to it than that.
There's a lot happening behind the scenes.
Yeah.
There is.
All right.
Shall we talk about this language summit thing?
Let's do it.
So, let's just, I mean, I gave it sort of a, a vague intro.
You wrote a nice blog post about it here on the Python blog.
Um, just give us a sense of what the language summit, what is it for?
Is this something is like an open space I can go to or not?
No.
So, so it is, it is a specific space for Python core developers, to use.
And so I think the, the whole goal of this is to get a bunch of core developers in a room
together to discuss things, and to kind of get some recommendations, some ideas flowing,
uh, maybe without necessarily like having the full formed thought, right.
Because like putting something out there, like completely radically changing the direction of
Python, that that's, that's, that's a lot to put that out just publicly, whatever.
And so this is like a place to collaborate for core developers and some special guests
because they're not everyone that's there as a core developer, but it is invite.
So you have to like apply to go and say why you want to go.
And then, yeah.
So it's not just folks who were, I can't remember if it was Sebastian Ramirez or Samuel Colvin
or something around the typing thing.
I remember some of those folks might've been at one of them because they're like, wait, we
can't change typing to be more performant where it completely ignores the runtime stuff.
Cause we have all these frameworks.
Right.
Yeah.
But the, the language summit, it, it's a whole bunch of different, like submitted topics.
People will talk and then there's discussion and some outcome.
Maybe there's no outcome.
Maybe there's like next steps.
Maybe we solve some problems within the time of the meeting.
And yeah, my job there was to, to actually like take down all the notes and write about
it and like publish these blog posts.
Cause one of the, one of the reasons why this is allowed to be like an invite only meeting is
that there is someone who is basically taking.
So I think it's a real careful balance.
Yeah.
And like, yeah.
And like what happened during the discussion so that the community can learn about what
actually happened without necessarily, you know, getting the rawness of it, I guess.
Yeah.
There's a real careful balance.
You got to strike between allowing the freedom to say whatever without public scrutiny.
But at the same time, you don't want it to be like, well, the Python cabal met.
They've decided.
Exactly.
They don't like, they don't like your idea or whatever.
Exactly.
Yeah.
It's cool.
So I'll point people up.
Obviously it's going to be in the show notes.
Point people at your writeup.
It's like a meta post, I guess.
Would you say that?
It's, it talks about, you know, sort of landing page.
Landing page.
Exactly.
Four, one, two, three, four, five, six, seven, eight, nine.
And then nine topics and presentations that were covered.
And then the lightning talks, which is also almost like another sub meta section.
So there's a, there's a lot to explore here and there's nice write-ups on each of these.
So where should we start?
And we want to kind of wrap up the conversation we were just having, because that was actually
a little bit of a topic at the language summit, right?
Yeah.
Security was definitely a topic at the language summit.
This, this was somewhat in the, in the recent light, like the light of XZ when XZ had happened.
Pablo, one of the release managers for CPython 3.10 and 3.11, I believe, brought this topic.
And it was basically discussing Python's like contribution and release and all of that model
in the light of XZ.
So like XZ, I'll just go over really quickly.
Yeah.
I'm sure.
Tell people about XZ.
Yeah.
I'm sure a lot of people have like heard of it at this point.
It was such a long game deal.
It was crazy.
So yeah.
What is the scary part?
What is XD utils?
And then what is the XC utils security issue?
Yeah.
So XD utils is a library written in C for basically processing archives of the XZ format, which is
just a compression format, like GZip, like any other compression format, Zotflee, Bratly, all
of those.
And so this library was maintained by a single person, big surprise.
And what is relatively little known, right?
Before all of this happened, I would say it was probably just adding features, fixing bugs
every once in a while, make a release and all of that.
And what ended up happening is this project was identified as a project that had very few
maintainers and also through a series of reasons had a linkage to SSH.
And so what ended up happening?
Yep.
And so SSH was...
If you can get into SSH and SSHD, then bad things are going to happen.
Yeah.
So the whole end goal of this entire operation was to get access to OpenSSH through like linking,
basically.
And so what ended up happening is a bunch of fake sock puppet accounts showed up after
this loan maintainer indicated that they were having a little bit of trouble maintaining the
project and like meeting user demands.
Like these sock puppet accounts show up like days before they were actually used and all kind
of in similar fashion, right?
And basically are pressuring this maintainer to either make them feel like they're not doing
a good job or to try to add a new maintainer.
In this case, Giatan is the pseudonym that this account used.
And what ended up happening is over a year of like legitimate positive contribution from this
account, this account was added as a like equivalent to a release manager, right?
Like someone who is approving PRs and merging things and is capable of accessing the actual archives
when releases happen.
And then a good chunk of time passes after that.
And there's a couple of little changes added that on their own are not an attack, but it
was preparing for an attack where essentially a small change was added into the release archives
that was not a part of the source tree that activated the entire attack chain.
And then when this started to like be pulled in downstream into things like Fedora, early
versions of Red Hat, this never went into any stable builds, but it was all the pre-releases.
This ended up getting onto people's machines and then was eventually discovered because of a very
small performance difference when logging into SSH.
Yeah.
And that was somebody who wasn't even looking, forgot the person's name, but they worked at
Microsoft and they were just on the Azure team or something.
And they, they just were like, why did this slow down a little bit?
That's weird.
And they're like, wait a minute, what is this doing in here?
Yeah.
It was on the verge.
And you know, what a long game.
One person, one account came along and was just, you know, I'm going to hear to help you.
I'm just going to make, I'm going to try to just become your best friend contributor.
And then another one is just abusing, like mentally abusing the people.
Like, why don't you just quit?
Why don't you get some more support?
And then like, who, well, let me reach out to some people who are helping me out recently.
And it turns out these are like two sides of the same coin.
Yep.
Exactly.
Yeah.
Shady.
Okay.
Luckily that got caught.
Cause you know, there's a lot of servers in the world that can be SSHed into.
And like, well, we got, you know, public private key encryption.
You can't break through that stuff.
Long as you don't use passwords, like you're going to be fine unless you're not.
So I received a lovely email on the day that this happened.
Report to the security response team for Python, because we of course use the XC utils libraries
because Python sports XC format as well.
And I, there was a, there was a lovely few seconds where I'm like, oh, this is either going to
be a fine day for me or a really bad day.
And it ended up being a fine day.
So that's good.
It's like, are we going to be canceling all our plans for next year?
Yeah.
Yep.
Am I going to have lots of questions to answer from like a concerned customers, users,
but it was fine.
Yeah.
I find this kind of stuff is a lot like everything's fine.
You relax.
Just work's going good.
Life's going good.
And then, you know, something's on fire just out of the blue and you have this, you know,
it takes your breath away moment.
Like, oh, does that apply to us as well?
Cause if it does, everything just changed.
And yeah, really glad I did.
This portion of talk Python to me is brought to you by Posit, the makers of Shiny, formerly
RStudio and especially Shiny for Python.
Let me ask you a question.
Are you building awesome things?
Of course you are.
You're a developer or a data scientist.
That's what we do.
And you should check out Posit Connect.
Posit Connect is a way for you to publish, share, and deploy all the data products that
you're building using Python.
People ask me the same question all the time.
Michael, I have some cool data science project or notebook that I built.
How do I share it with my users, stakeholders, teammates?
Do I need to learn FastAPI or Flask or maybe Vue or React.js?
Hold on now.
Those are cool technologies and I'm sure you'd benefit from them, but maybe stay focused on
the data project.
Let Posit Connect handle that side of things.
With Posit Connect, you can rapidly and securely deploy the things you build in Python.
Streamlit, Dash, Shiny, Bokeh, FastAPI, Flask, Quarto, Ports, Dashboards, and APIs.
Posit Connect supports all of them.
And Posit Connect comes with all the bells and whistles to satisfy IT and other enterprise
requirements.
Make deployment the easiest step in your workflow with Posit Connect.
For a limited time, you can try Posit Connect for free for three months by going to talkpython.fm
slash Posit.
That's talkpython.fm/P-O-S-I-T.
The link is in your podcast player show notes.
Thank you to the team at Posit for supporting Talk Python.
One of the talks was Python security model after this issue, the XC utils backdoor.
Tell us about that.
Yeah.
So this entire talk was essentially just overviewing like, hey, is this possible?
Is this possible for CPython to be impacted by such an attack?
And I mean, the answer is yes.
It really is.
Because if you have accounts that are willing to put years of effort into contributing good
code to CPython, right?
Like that, that is enough to become a core developer likely.
And if you're a core developer, it means you can merge PRs.
It means if you, I mean, if you get two core developer accounts promoted to this level of authorization, you can merge your own PRs with review, right?
Like we're the big focus on this talk was like, okay, how, how can we prevent this?
And if we in, in the ways that we can't prevent it, how can we be ready?
And kind of like a discussing whether or not we're ready at this point.
And I think the big consensus was that if we were to discover something like this that had already been merged into Python or had been released in Python, that we would be okay to be able to, to get it to like, get the word out.
Like that sort of infrastructure already exists.
And we're not too worried about that.
Like we're a CNA.
We can create a CVE really quickly.
We can get like the announcements out really quickly.
We can get releases out really quickly.
So like in that way, the reactive sense, we're okay.
In the proactive sense there, which is the more important one, but it's also the harder one.
We, yeah.
Because when nation states hire people to say the next three years of your job, it's, it's almost like, it's like a team, like a spy.
Yeah.
For multiple years.
It's like the CIA or MI6 or something, you know, on the code side, you know.
To be honest with you, it doesn't even need to necessarily be like nation state level stuff for this to happen.
That's true.
Because vulnerabilities in popular pieces of software are very, very lucrative.
You can sell them and people make a lot of money on selling vulnerabilities to projects.
But so why not grow your own, right?
Yeah.
So.
Go ahead.
One of the things I think is great is there's a really long release cycle.
Yeah.
Right.
And like a staged rollout.
So I don't know how many people jump in and install alpha one of some Python, but it's, it's pretty limited and it's not going to make it.
No, no.
Of course.
I know.
I see you raise your hand.
But I think that's going to be a good thing.
I think that's going to be a good thing.
update channel.
I think that's going to be a good thing.
I think that that helps in a way.
Obviously, people that are living on the edge, maybe they're the more valuable targets, but I mean, I'm not going to be the one to encourage that.
Yeah, I don't think that I think the biggest defenses against this, and this was what was discussed there, was trying to push things to be in the open.
Actually, in a way, open source is uniquely able to respond and be defended against for these sorts of attacks.
Because if this were to happen in Windows, for example, would we have had the almost immediate being able to debug what the actual attack was, how long this had been going on, what patches were bad?
Like that sort of visibility into the source code is something that was really important in being able to actually track this thing down.
Yeah.
And so having test files and binary files not checked into source code and instead generated.
So one of the parts of this attack that allowed it to go hidden for so long and be checked into source code was that almost all of the attack code was hidden extremely well in a binary file, which made it so that code reviewers could-
Some of the test binary elements.
Because if you've got a compression file utility, you've got to have compressed files for your unit test, right?
Mm-hmm.
Exactly.
So it was basically these files were checked in and there's just huge binary blobs that you can't actually get your eyeballs on to review.
We talk about like lots of eyeballs make for shallow bugs.
Well, if the eyeballs can't see the bugs, then you're not going to find them.
And so we talked about like removing binary files from the source code or making sure that all the binary files that are generated have like a script that allows them to generate at any time and things like that.
So is it one of the changes recently?
I can't remember if this was on IPI or if this is a GitHub thing, but allowing GitHub to be the thing that publishes directly builds the wheels and uploads them to PI PI rather than somebody downloading the code, building them and uploading it, which obviously that's a opaque step there.
Yeah, so other like other things that tie more strongly these whatever release artifacts are actually ending up on people's machines to the source code.
So that's I mean, I would call that build provenance.
There's a whole bunch of different frameworks that that works under.
But yeah, build provenance being able to tie in an artifact that's installed in your system back to the actual source code so that when you are evaluating that that artifact on whether you want to install that on your system or deciding whether to upgrade or whatever, you can look at the source code instead of writing at this like compiled binary.
That's something that I really want to focus on for like PI PI in the future.
But yeah, we're not there.
Awesome.
Yeah.
When you because when you look at a project, you say, well, let me see what the releases on GitHub.
If you know that literally that was the thing that compiled or got built and then that's what's on PI PI.
That's a different forensic analysis than.
Exactly.
Well, it's somebody's machine.
What it ends up nice like being nice is that it and this is the tough part is that I feel like a lot of people's behavior, which is to go on GitHub and say, you know, I'm going to get a lot of the work.
I'm going to get to go on GitHub and look at the diff between like tags.
That's what a lot of people do, but that's not actually what you should do.
You should be looking at the diff between the artifacts.
That's the thing that's actually installed on your machine, but that's way harder to do than looking at the tags.
So exactly.
We just crowdsource it.
We're all crowdsourcing it.
Like wait for the way we are.
I know where we are.
All right.
We're just related to this while we're still on this topic.
You know, you talked about the somewhere.
There we go.
There's also a big news around CVEs, which are official vulnerability numbering.
So they're referenced through all those cybersecurity talk and stuff.
Right.
You can describe it better.
But so big news is that the PSF now, and you alluded to this, is now an official numbering authority.
So rather than saying there's a problem with Python, who is going to sort of officially call this out and write it up and so on.
Like you guys can do that directly now.
Right.
Yeah.
So like CVEs are basically it's a set of identifiers and records that show what's like a bunch of metadata about vulnerabilities in software is what it is.
And it's only one system.
There are a bunch of other like vulnerability databases, but CVE seems to be the one that everyone uses or references.
And so what being a CVE numbering authority gives us is it makes it so that someone at the PSF can like operate the CVE UI and workflow and all of that to say like, oh, we want to create a new CVE ID on behalf of the Python team or on behalf of the pip team.
And what that ends up meaning is that because we are part of the process instead of having to go to some other entity.
So like Mitre or Red Hat or, you know, Microsoft, there's a whole bunch of CNAs.
There's a there's over 100 now, I think.
Instead of going to someone else that, you know, isn't as well versed in Python or, you know, our release schedule or any of those things.
Right. We get to inject the knowledge that we have about Python into all of these records, into all these advisories.
And it makes it so that we don't actually have to talk to someone else to be able to handle a vulnerability and to end.
Right. So like before you would potentially have a reporter going to talk to someone else and getting a CVE ID and then they would come talk to us.
And by that point, like it was it's hard to like make a determination and there's a whole bunch of things have already happened.
And maybe there's messes that need to get cleaned up to make sure that it's not confusing.
So by owning the entire process, we're able to make sure that things are as little confusing as possible.
Like what actually needs to be done for users when we publish these things?
Yeah, that's great. I want to move off the security angle here because I know there's so much more to talk about.
However, you know, but you guys do it. Have you considered or ever run any sort of bug bounty program?
We don't have a bug bounty program right now. I mean, the hard part with the bug bounty program is it takes money.
So if if you would like to see a bug bounty program happening at the PSF, get in touch with the PSF. Send email.
Yeah, I think incentives are really aligned there.
There's a lot of companies that have this tooling at the center of their data center.
So maybe.
Yeah, maybe.
Maybe. Maybe. Maybe we can make it happen.
All right. Next up, the REPL or the PI REPL for the Python PI REPL.
What's the deal with this?
Yeah, so this was a a talk that was given by a couple of different core devs.
I think this included a bunch of people. Pablo, Lukash and Lissandros all gave this talk and it was about, hey, this new REPL that's coming in Python 3.13.
Here's all the cool stuff that it can do and how it makes the usability so much better for people.
And they demoed a whole bunch of the new features, which was really exciting.
There was lots of applause showing off a few of these like little little features.
And I think that the the other side of it is like because this new REPL is written in Python and not written in C, it lowers the barrier for contributions and maintenance drastically.
Before the REPL was like super entwined with like the parser and all of these other really low level details of Python that a lot of people probably didn't want to get involved with if they didn't have to.
Versus this where it's this completely separate and much more easy to contribute to piece of software.
Yeah. And did this come from the PyPy project?
Yes, this was PYPy. And I think that there's been some some back and forth contributing back, contributing forward, all of that, which is also really great. Right. Having one REPL shared between two different implementations.
Yeah, that's great. Just working and working better together. More people working on it.
I always caught PYPy because some people call Python packaging index PyPy.
But that's also this other thing. So anyway, it's a part of every one of my days. Right.
I'm sure that I because a lot of the times, you know, a significant percentage of my work as a security person is being in working groups that are not Python related at all.
And yeah, there's a lot of PyPys flying around. Yeah.
You talk about NumP being on PyPy and you're like, OK, hold on.
Hold on. Could be two different things.
There's a lot going on here.
Yeah. So this is really interesting.
I haven't really played with it much.
I honestly don't spend a ton of time in the bear Python REPL.
Like if I'm I'm REPLing a lot of times I'm in the JetBrains sort of enhanced REPL that's inside PyCharm, something like that.
But and I think partly because there was a lot of challenges with the bear Python REPL, right?
There's no autocomplete. But worse than that was if you've got a five line function and you want to edit it, you've kind of got to go to the top heart hitter.
Like it's it really was hard to work with blocks of code.
There's no color, things like that.
Yeah. Color is the standard for you don't have color in your terminal at this point.
Like even 20 basically given up at that point.
I have an emoji and you don't have color. I mean, emojis.
Got to have like the rainbow prompt, maybe like the logo of the thing that starts a starship.
Oh, yeah. I was well. I didn't even consider the ASCII art and possibly the ASCII art in color.
No, seriously, though, I do think it sounds like a minor deal, but just the readability of having highlighting and stuff.
Yeah. Syntax highlighting is huge. Syntax highlighting is like really huge.
That's not a part of the current REPL, I don't think.
But like it becomes much more possible because this PyRipple exists.
Yeah, exactly. Yeah.
I think that like the biggest thing, yeah, like the whole blocks of code.
I just remember the demo of them showing like, oh, you have like five lines.
You have to hit up, up, up like four times hit enter.
It's just like, oh, my God.
If you mess it up, you got to start over.
You got to start over.
And you're just, you're just sad.
You just contemplate putting it in a file instead of doing this in the REPL.
Exactly. That's why I'm not in there.
I avoid being in there because it's hard.
This will be really, really great for people that are just starting Python journey.
Because I think that a lot of people learning and starting off will use the REPL straight up.
Yes.
Instead of an IDE.
Like having this, there was a big focus on like teachability and documenting it and making it work the same.
Like if you actually like read the post, like what the discussions were about for like everyone is basically totally in favor.
They loved it.
But they wanted to make sure that this was going to be like a consistent experience.
Specifically, like Carol Willing had this big point about like having a consistent experience being really important for teaching Python across different operating systems.
So.
Yeah.
And something a little bit, a little bit better than up, up, up five times and don't get it out of order.
Exactly.
Yeah.
So I guess one of the, I don't know if this was discussed, but one of the challenges of this, I think is it requires, and it's not necessarily bad, but just a challenge is I think it requires the new windows terminal rather than say CMD.exe, the older style.
So it just works out of the box on macOS and on Linux, but on windows, you got to be a little careful about how you run it.
Is that right?
So I actually don't know what the current status of all of this is because the time has marched on since these blog posts have happened.
Yeah.
There has been a lot of work done on the windows side that the current team, like the team that presented this didn't have any windows experience.
And so they didn't know really how hard it was going to be.
I think that there's been a lot of strides in the windows side of things.
So I think the situation's better.
I don't know offhand if CMD that exe is supported or if it's just the new windows terminal, but I, yeah, I think it's fine.
It was sitting in windows terminal.
Like people need to be using that thing.
Anyway, it's true.
It's like opening up on your Mac and just having like the bare white bash, I guess it's Z shell these days, but just the completely, you know, non fixed font.
Like, what is this thing that you are running?
Like, you're like, the terminal is horrible.
Like, well, that thing is, but you know, you could make it a lot nicer by the way.
And, you know, it's, it's similar, trade off there.
Right.
And windows world.
So, okay.
Exactly.
Interesting.
Next one is, should we, should we adopt?
Should we adopt calendar versioning?
We're beyond zero verse.
So that's really good.
But there's been a, a reluctance to have Python four, but we've got three 12, three 13.
Are we just going to have three dot 128 or should we come up with something else?
Right.
What is calendar versioning and should we adopt it and how many digits should it have?
Yeah.
So this was, presented on by Hugo, who is the new release manager for Python three 14 and 15, which maybe that's a little presumptive saying those numbers.
Um, cause if this goes through, that would not be the case anymore.
Um, he's going to work himself straight out of a job here.
What's going on?
Yeah.
Uh, this is, this was, there were definitely jokes about like, this is just your attempt to, to get out of being the release manager for these releases.
Um, but yeah, that, so Hugo proposed this, what this is kind of like a pre PEP feeling out of how this situation should be.
And like trying to pare down the options, I think was Hugo's biggest, biggest question or what should we do it?
And should we pare down the options?
Cause there's a million different ways we can do calendar versioning.
Um, and yeah, I think if you scroll down, there was like a slide that had just, you know, every single possible calendar versioning possibility for Python and all the different languages.
Um, but yeah, counter version is like really common for programming languages and other things that are similar to Python.
Um, and so this was basically like, Hey, we have this yearly release cycle that is been working for a while and we're probably going to keep doing it.
Should we, it's worth pointing out for people who don't know that it used to be 18 months.
And so the calendar version, it would get a little out of phase or something there, but now, now that it's yearly in the fall, it really lines up perfectly.
Yeah.
And so this kind of assumes that we're going to keep doing the yearly thing, which I'm fine with the yearly thing, but yeah, as, as long as we kept the yearly schedule, it would look like.
Like the release year would line up with whatever the, so it would be like the ended, the one that was like most agreed upon was like three dot.
And then a two digit year, or what would end up becoming a three digit year when we roll over to a hundred, assuming that Python's still using them, you know, a hundred years.
Um, but yeah, so like, that was kind of like the one that was most palatable to core devs or people were most excited about.
And I think the big reason why switching to like a Calver year was interesting is that we have this thing called like support lifetime.
So like how long is CPython supported?
How long do you get security fixes?
How long do you get bug fixes?
Um, and so being able to do math is easier.
Yeah, exactly.
Like, let me put this out to the audience.
Is Python 3.8 supported or is it not supported?
I don't know.
You gotta do, you gotta do math.
You gotta think about it.
Um, yeah.
Um, yeah, so three, seven just recently dropped support, right?
Which is crazy.
Cause that seems like a pretty new version of my mind, but I totally make sense.
Uh, but if you just knew it's supported for how many years, six years, five years, it's five years for security releases, I believe.
Yeah.
So then you're like, it's 2025.
So, you know, 20, 20 out becomes a lot easier.
And there was also say like, do I have the current one, right?
If you're not tracking it super carefully, like three 11, is that the latest?
Like, I don't know.
I only use Python every like once a month.
What was the 2023?
Oh, I see.
Well, I mean, that's not the latest one.
Okay.
Yeah.
So it was just an interesting conversation about figuring out what the best potential option
was.
And then Hugo ended up creating a pep.
And I think that's being discussed right now.
So cool.
Why not 2024?
Why 24?
Because that feels, I don't know.
It just feels like you've just point shifted what you're doing now rather than, then really
clearly.
Because, you know, as a new person coming in, you don't see that go, it's 24.
So it must be 2024.
And unless you really like put together the calendar.
But if it's a dot 2024, you're like, I bet that's the year, you know?
Right.
Yeah.
And then eventually in the not too distant future, there will be a Python 3.24.
Exactly.
Yeah.
We'll see.
So there's going to be a PEP around this, you say.
In fact, it says right here, PEP 604, right?
I think it's pep.
What is it like?
Well, you know, that's just the yearly announcement.
That's the yearly cycle.
If you scroll down all the way to the bottom, it'll, I think it's like peps.
Yeah.
There's a lot of peps.
Drafting.
It just says drafting a pep.
Oh, give that a click.
I'm pretty sure there's a number 2026.
There we go.
Okay.
Yeah.
Oh, so that's going to be a while.
So they released this pep.
Well, so the, I'm just kidding.
The most, the most important part of this discussion was that the, the Python version 3.14 be, be preserved.
Python.
Yeah.
Well, it wasn't allowed for three, three 14 to change it.
Yeah.
The only thing that I can think of that you would have the two digits is that there's a lot of code and regular expressions and junk out there that checks for that.
But you know, if we talk about some of the other stuff out there, like that's a pretty minor change.
Like for example, free thread Python.
Yes.
Free thread Python.
It's here.
It is here.
Sort of.
You know, what actually really surprised me is that when I saw this PEP come through, was it 702 or something like that?
It said, we're going to allow free threaded Python, which I'm going to have you explain for folks in a moment, but you're going to have to have a special build of it.
And I thought, oh, well, that means if you want to play with it, you're going to have to build your own.
But I noticed that the installers now give you an option for it.
Yeah, they do.
Yeah.
The installers.
Side by side install.
Right.
Yeah.
That gets put onto your actual like.
Yes.
Python T is what you type instead of Python.
Yeah.
If you want the free threaded one.
Yeah.
It.
I mean, free threading is here.
I mean, there's options.
If you're compiling yourself, you just enable some options.
And I think I go over that in the actual blog post to the options that you actually use to try it out.
And yeah, it free threading essentially is it's a way to remove the GIL and move to a different like reference counting model object counting model.
And which is quite exciting for a lot of people, but it will what it will end up meaning is that a lot of the packages that are written in C or that are relying on CPython APIs will have to get either, you know, tweaked a little bit to like use these slightly different C APIs to make it so that they play nicely with having no
no GIL enabled and with the new memory management.
Yeah.
It's super exciting.
It's a variety of changes in the ecosystem basically.
Yeah.
Just because you have threads doesn't mean you get perfect scalability across the cores.
Can't remember who wrote this article.
Is it Simon Wilson?
Maybe.
Who did some.
Yeah.
I'm pretty sure Simon Wilson wrote one that said, look, we're going to take an algorithm that can is kind of embarrassingly parallel and parallelize it.
And it turned out to be something like 50% gain per core.
So it was like he had eight cores and it was four times faster with three threaded Python than without, which is still, if you can get your code to run four times faster, that's still really good.
Right.
Yeah.
It's going to have, like you said, I think it's going to have an interesting requirement put on all the people building packages.
Right.
Yeah.
And I know when I hear people say, I think maybe you just said like, oh, it's going to be the C extension packages that are really going to have to deal with it.
They, they'll have to do locks in their thing.
I think even in the Python code, there's certainly algorithms that have multiple steps that they'll get some data here.
They'll work with the data.
They'll make some changes.
Then they'll put the data back in the same place.
And even that would be subject to a race condition.
Yeah.
I think we're, you know, I've in long in the past did a lot of C++.
I did a lot of C# and in communities like that, people are like always focused.
They're like, always kind of crazy about two things, memory and threading.
Right.
And we just don't do that in Python.
We just, I think we have just leveraged the fact that the Gil gives us kind of enough core screen granularity, the execution of our code, that it's just not something we hit a lot.
And we don't try to do a ton of threading because it doesn't work all that well.
However, this, this could expose lots of stuff.
This could put a new focus on that.
Yeah, definitely.
Yeah.
Just having more people using threading with Python that that's going to be huge for finding thread safety issues.
Yeah.
It's, it's just really exciting.
I think that, and there's another blog post, a completely separate one that talks about like the C API.
And there was some mention about like free threading and evolving the API so that it's a lot easier to use from a three, a free threading perspective.
Uh, so like there's a ton of work happening in here to make this as easy, hopefully brief kind of split in the ecosystem and then have it converge together.
And then we're going to, we're going to, we're going to, we're going to have a way that if this is really not working out, we can go back.
But if it is working, we need a way that we can actually land this thing as the default.
Yeah.
Right. Right. Right. Right. And the PEP discusses. This is like, we're going to, we're going to see how it goes, which is really interesting.
Um, but I think it's not breaking in the sense that you can't still run Python three with the thing that you've got.
You just might not be able to enable this free threading aspect of it for some time.
Whereas from two to three, it's like, you cannot run this library on three period.
There's no scenario which this is going to work because it needs to take into account this and it doesn't.
And so it's out.
So I feel like there's more time and space to evolve it.
And you could say, well, in this space, you know, in this data science section of the world, we use these seven libraries and we're going to work and make them compatible so that we can get way better performance.
Or, you know, we're going to work to make sure that FastAPI and Pydantic support it really well so that we can scale our web servers better.
Yeah, no, this will be huge for, for like web and data.
I think that a lot of people are excited for this for a really good reason.
Yeah. Yeah. I totally agree. I totally agree.
Okay. So this is a big deal.
It's coming in three 13, but you've got to run Python T for now.
Yeah, it's a three 13, but it's also available in the pre releases.
The first release candidate for three 13 is out.
So give it a test. If you haven't given it a test, give it a test.
Yeah. Very cool.
All right. What, what one do we want to talk about next?
We got just a couple more minutes to cover.
We've got, what about Python and mobile?
I think that one's, I know there's the black swan talk that Keith Russell McGee gave and Carol willing also sort of shouted out.
Like there's a couple of places that are really, really important computationally in the world that Python kind of isn't.
Yeah.
We should have it there.
And those number one has got to be mobile.
Yeah.
Mobile and front end for me, mobile and front end are the two.
And like a far distant behind that is like, could I get a single binary out of my app that I can give to someone?
That's a different, that also is in there, but it's, it's like not as important as, Hey, I want to, I want to build some mobile apps.
Can I use, you know, I want to learn that with an easy language.
Can I use Python?
Like, ask me something else.
Yeah. Yeah. Right.
Like next question.
Yeah. So this was, this was a, it's almost like a, a big status update on where Python is in the mobile space, which is really exciting because they've made a ton of progress on getting like actual tiering of support for these platforms.
Um, so if you don't know, Python has a like platform support tiers where it's like tier one is like x86 Linux, right?
Like that's a, you know, 90% of PI PI downloads are, are that like, yeah, probably want to support that one.
Uh, and then it's things like macOS, you know, x86 and arm and all of that.
Right.
Um, and then lower down there's tier two, which is, you know, the platforms that they have people that are interested in them.
But if those people were to go away, then we wouldn't actually have a way to support them.
And tier three is like even more so.
Right. so having tier three support, for Python, for both Android and iOS for three 13, like that's super exciting.
It means that these things are getting actively tested.
Um, there's like integration testing on real platforms and that there's people that care about it that are fixing bugs.
And this is exactly what you need, to get your platform supported.
And so this is all being provided by Anaconda funding this project and be aware.
Okay. Yeah, that's right. They are, you know, be aware and Keith has been on this for a long time, but in a kind of come along and put more time and energy behind it, in terms of funding and people as well.
I'm not sure, but certainly in funding, that's awesome.
Mm hmm. Yeah. So I think this was, it was both a status report and also kind of trying to figure out how these sorts of platforms can get tested, more easily and like actually not having constant breaking because these platforms are so different from, you know, what almost every other core developer is using to develop a Python or a lot more limited in terms of capabilities, and like locked down in a security perspective too.
And they have no regard for backwards compatibility. I, I got, I, you know, I have mobile apps for the talk Python courses that are in both iOS and Android's app stores.
And I'll get messages like, Hey, dear developer, if, we see that you're built against three year old APIs.
If you don't rebuild and republish your app in the next six months, we're taking it out. The last one I do this for was Google. I'm like three, three years. Okay. Can we, no, we can't get any better compatibility than that. Like I just got to keep re-uploading the same thing. Even if there's no changes. Like, so, you know, that's just a different mentality of like, ah, we changed all that. We don't like that anymore.
Yeah. Luckily I'm, I'm actually not sure how affected Python in particular is by things like that. Cause that's like utilizing APIs, like mobile SDK APIs versus like the operating system of the phone, which.
Yeah. Right. Like people would build apps with Python and then they would be subjected to these emails. And it's not even that I was necessarily using any of those APIs. It's just like, we see you're compiled against the wrong version. So try again, you know?
Yeah, no, the, yeah, the, the difficulties that I've, at least from, from this talk have, figured out is that like these platforms are just a lot more locked down. So like a lot of system calls won't be available that the test suite like assumes are available always.
Sure. It's almost like a circuit Python sort of deal, but not that extreme.
Yeah. It's like somewhere in the middle and figuring out how to all work together happily and develop on this similar code base that has all these different target platforms.
Yeah, absolutely. Absolutely. Awesome. Well, I'm, I'm really excited. I'm all here for it. If, if three years ago, I think it was when we started working on those mobile apps, if I could have used Python in a really solid way, a hundred percent, those apps would be built in Python.
But just, there's so many, so much tooling and stuff around that. You got to create a signed APK before you upload. There's a lot of stuff going on there. And so, hopefully they, they get that. That would be a game changer.
And just, you know, it's not on, it wasn't here. Almost surprised me that it wasn't here, but front end stuff, WebAssembly, PySerscripts, Pyodide, all those things I think are in that same realm.
Although they can just kind of ship stuff to the web because there's no gatekeepers, but still.
Yeah.
Was that mentioned anywhere during the summit that just didn't make a post?
No, Wasm was not, there was no topic about Wasm specifically at, at this language summit.
Yeah, sure. I think there was the previous year.
Previous year there was. Yeah.
Should we make PDB better? Does it matter? Are people using PDB? What do you think?
Yeah. So this, this was all about PDB is Python's debugger for people that don't know. if you've never used it, it lets you kind of like drop into set a breakpoint in Python and then drop into that exact spot with all the context and everything, which is really.
Right. At a lower level. Yeah. At a lower level than a VS Code or PyCharm.
Right. Exactly. Yeah.
Like seeing all these like super internals of Python, if that, if that's something that you really need. Right. and so this was a talk, that was mostly about, okay, we're, we have PDB, but now we have all of these new models like free threading and all of that. And also we're being a little bit held back by backwards compatibility. there's like a specific, really specific point where, because of backwards compatibility reasons.
And PDB is a part of the Python standard library. It becomes difficult to break backwards compatibility, even if it would mean you get a bunch of really good stuff out of it. you can't always do that because people are depending on it. And I think that the, yeah, the recommendation was maybe we should develop this outside of the standard library so we can, you know, be, yeah.
Break backwards compatibility if it's not necessary and, and make it so that we can support multiple versions instead of just having it be per version. And yeah.
Yeah. Yeah. That's, that's a good idea. That's exactly what I was thinking. Cause you know, there's the whole dead batteries talk. Like, does this still belong here? I'm not necessarily thinking this should not be in Python, but you know, yeah.
Yeah. Yeah. Something broken out maybe, but take that exact code, break it out, but you know, enhance it kind of independently.
Yeah. And I think the concern from, from some people in the room was that, oh, if we break this out onto PPI, then it would potentially mean that it would not get the same level of contribution that PDB sees because it's part of Python. Right. Sure. And I mean, totally valid in my opinion too. Like being a part of Python is a huge, like blessing of like, yeah, this is something important. Right. Um.
But I, I think that there's, there's other ways to signal that that's something important. Like if you look at like my pie, my pie is underneath the Python GitHub organization. And so maybe something like that, right. Where it's this tool that is very actively used by core developers for development. And it is a little bit more,
more official than, you know, just some random person putting something up on pipe. Yeah. This is core developers supporting this. And black is that way too, I believe. Right. It is. Yeah. So maybe something to signal just a little bit more of an official. This is a core developer tool. Here's why you should contribute to it instead of just, you know, a random project on pipe. Yeah. Which definitely wouldn't be in that case.
It would not. It would definitely not. All right. How about, how about a quick review of maybe some, some of the lightning talks? Yeah. Any of these stands?
out. You know, obviously Rust and Python is, seriously a one. yeah. Emily's talk was, yeah. I was going to say, Emily's got a good one. Emily has a really good one because, and this is like, it's, it's almost meta, right? Because lightning talks, are not submitted ahead of time. You actually have to submit them during other people's talks. like to the list that you want to talk about this and then put together some slides really quickly.
So yeah, these talks are pretty impressive in that way having your minutes, but the Emily's talk was about, it was kind of like wrapping up a theme that was being heard multiple times over the course of the language summit.
Um, but obviously this is a problem outside of the language summit too, which is that when someone goes to make a prototype for a pep, they are given at least today, not a whole lot of support, for doing that prototype because it's basically like, oh, we think that this should be developed outside of the standard library.
Uh, initially, right. Like that's a really common, determination that the steering council comes to. and so being able to have kind of like a, a standardized way that people do a PEP prototype outside of the standard library. So things like creating a repo and like having all of this existing infrastructure set up and, maybe even hosting it under the Python.
So to the Python GitHub organization to give it some like air of officiality of like, yeah, this is something like really big is happening here. It's not just like someone in a corner writing something right. Like giving some more grandiosity to, to the work that's being done and not just kind of saying, oh, go away.
Like that is a question. Right. But that's kind of how it can land sometimes.
Right. And maybe setting up people for success, at least this is what we're going to expect from you. If you go through this process, then you've got, you're further down the pipeline of having that conversation for a pep.
Yeah, definitely. And like, if you're wanting to write something that is for Python, you know, you probably don't necessarily care about like setting up these exact workflows for publishing to PyPI. Like that's just a whole bunch of things that are in your way to actually being successful. So having that all be figured out already ahead of time for you makes things a lot easier for you.
Yep. Yep. Yep. Yep. Let's, let's finish out with Yuri Silvanov's presentation, efficient data sharing between sub interpreters. And it's interesting because we talked about free threaded Python, but the year before the big news was sub interpreters and Eric Snow's work. And those are not directly competing type of things, but in a sense, they're kind of competing.
Yeah, they're definitely competing for being like the, the model of how to do efficient, you know, parallelism in Python.
Yeah. Yeah. How do we isolate the stuff so that we can avoid the guilt and we take it out and add different algorithms or do we just make copies of the interpreter and run them in isolation? But then you have this data sharing issue. I can't just share a pointer easily. Right. So what's this about?
Yeah. So Yuri basically came with, and this was also a, if you want the extended version, Yuri also gave like an actual PyCon talk about this library that he's developed called MemHive.
And then, yeah. What's it called?
MemHive. Like M-E-M-H-I-V-E. Yeah.
All right. Awesome. And just for everyone listening, just this week, last week, recently, all the videos of all the talks are now available on YouTube.
So it's been a while coming, but you can go watch it now.
Exactly. So go watch them all. If you, if you missed out on my talk, go watch them.
But yeah, this, so this library in particular is, it's basically a way using immutable data structures.
There's this immutable data structure called an H-A-M-T. I actually don't know what it's short for, but it's a hash, hash array map tree.
There we go. It was in the, I wrote it down.
And it's essentially like a way to have this tree that can be passed around and shared without like worrying what the other processes, sub interpreters are.
They're not processes.
The other sub interpreters are doing to this data structure.
So it enables a more efficient and safe way of sharing data.
That's kind of like in a tree structure.
And I think one, the demo that he ended up giving was about a dictionary like data structure where, you know, you have a million keys and a bunch of sub interpreter workers working on that data.
And they're able to, because it is using this immutable data structure, the modifications and changes are all safe, but it's also like super scalable and performant.
Yeah.
The thing about parallelism and multi-threading is if it's immutable, you can have many things as you want reading from the same memory.
It's only when they start writing, does it matter?
So.
Exactly.
Yeah.
This, this like has a way, a mechanism to capture the rights in a way that is safe so that like the current one can see what has been written.
And then the other ones aren't affected because their copy is not changed.
Okay.
That sounds very interesting.
We talked about the coming compatibility matrix of free threaded Python.
This won't have that issue, right?
This operates in the every version of Python.
Yeah.
So this, I would assume that this sort of module would be able to say like, I am ready for a guild free world.
So that's like the mechanism that I believe CPython has, has adopted for saying that your C module is ready for not having a guild.
You actually have to opt into it.
And then that module will, will be allowed to run in a free threaded Python.
Yeah.
It's something I recently learned is there's separate wheeled builds for free threaded Python as well.
Yeah.
That's, that's interesting.
Yeah.
It's its own.
I don't know exactly the phrase for it, but yeah, its own wheel tag platform target or whatever.
Yeah.
Like free threaded gets appended to, you know, macOS arm 64 or whatever.
Exactly.
Yeah.
Awesome.
All right.
Seth, this has been great.
How about some parting thoughts?
Let's close this out.
Let's just take aways from, from the whole experience.
Yeah.
I mean, the language summit is lovely.
One of the things that's like most important to me is like this whole aspect of storytelling.
And so that's why I felt really, really happy that I was invited along to, to be able to tell
these stories to all of you.
And I think that having all of these different narratives all in one place of all of these huge
themes about what Python is going through all at once, right?
Like it's really incredible how many different things are happening in Python all at once.
Because like sometimes when you're focusing on just one or just two, you know, you don't
have this huge context of, wow, Python is changing in like at least 20 different ways all at once.
And we're somehow doing really, really well.
I would say like, yeah, I have no doubt about any of any of these huge changes that Python
is going through, like to, to take it in the wrong direction.
Like I'm feeling hopeful and excited about all of them.
So it's an exciting time.
Yeah, I am as well.
And it is really tricky to get a picture, a holistic picture of, of the progress.
Cause there's a lot of different groups doing different things and there's no one person's
or one company's job to get somebody to come and tell that story.
So yeah.
Thanks for giving us the insight here.
It's been awesome.
Yeah.
Thanks for being on the show and I'm sure we'll have you back soon.
Yeah.
Sounds good.
Thanks for having me.
See ya.
This has been another episode of Talk Python to Me.
Thank you to our sponsors.
Be sure to check out what they're offering.
It really helps support the show.
This episode is sponsored by Posit Connect from the makers of Shiny.
Publish, share, and deploy all of your data projects that you're creating using Python.
Streamlit, Dash, Shiny, Bokeh, FastAPI, Flask, Quarto, Reports, Dashboards, and APIs.
Posit Connect supports all of them.
Try Posit Connect for free by going to talkpython.fm/posit, P-O-S-I-T.
Want to level up your Python?
We have one of the largest catalogs of Python video courses over at Talk Python.
Our content ranges from true beginners to deeply advanced topics like memory and async.
And best of all, there's not a subscription in sight.
Check it out for yourself at training.talkpython.fm.
Be sure to subscribe to the show, open your favorite podcast app, and search for Python.
We should be right at the top.
You can also find the iTunes feed at /itunes, the Google Play feed at /play,
and the direct RSS feed at /rss on talkpython.fm.
We're live streaming most of our recordings these days.
If you want to be part of the show and have your comments featured on the air,
be sure to subscribe to our YouTube channel at talkpython.fm/youtube.
This is your host, Michael Kennedy.
Thanks so much for listening.
I really appreciate it.
Now get out there and write some Python code.
I'll see you next time.