#179: Python Language Summit 2018 Transcript

00:00 The Python Language Summit is a yearly gathering of about 40 to 50 developers from CPython and other

00:05 Python implementations and related projects. The summit is typically held on the first day of PyCon.

00:10 Many of the decisions driving Python forward are made at this summit. On this episode,

00:15 you'll meet Marietta Wijaya, Lucas Lenga, and Brett Cannon, three well-known core devs,

00:21 to walk us through the major topics covered in this year's summit. This is Talk Python to Me,

00:26 episode 179, recorded September 26, 2018.

00:30 Welcome to Talk Python to Me, a weekly podcast on Python, the language, the libraries, the ecosystem,

00:49 and the personalities. This is your host, Michael Kennedy. Follow me on Twitter, where I'm at

00:54 M. Kennedy. Keep up with the show and listen to past episodes at talkpython.fm,

00:58 and follow the show on Twitter via at Talk Python. This episode is sponsored by Linode and Rollbar.

01:04 Please check out what they're offering during their segments. It really helps support the show.

01:08 Marietta, Brett, Lucas, welcome to Talk Python.

01:11 Hello.

01:12 Hi.

01:12 Thanks for having us.

01:13 Yeah, it's great to have Brett and Lucas have you guys back. And Marietta,

01:17 welcome to the show for your first time. I'm excited to have you here.

01:20 Thank you. Yeah, I'm excited to be invited here. Thank you.

01:23 Absolutely. So we're going to talk about the Python Language Summit, which I think is

01:29 not that well known and really, really important. And so I'm really excited to talk about that show.

01:34 But I want to give you all a chance to just introduce yourself for folks who maybe didn't

01:38 listen to some of the other shows or don't know you. So Marietta, let's start with you. Just,

01:42 you know, who are you? And what do you do day to day real quick?

01:46 Yeah, my name is Marietta. I work at Zapier, building integrations for various apps. I work

01:52 in the partner engagement team. So basically, we're building more tools for our partners.

01:58 Nice.

01:58 For CPython, I build lots of bots, lots of automations as well. So, yeah.

02:04 And you're on the core developer team, right?

02:05 Yeah.

02:06 You're one of the core developers? Very nice.

02:07 Yeah.

02:08 Lucas?

02:08 Hello, I'm Lukasz. I work at Facebook on the Python theme there. So we make sure to

02:14 be using the latest and greatest that is available in a safe way and in a way that sort of enables

02:21 us to use Python in our increasing scale and whatnot. I am contributing to Python, but you

02:27 probably don't know this. But what you do know is that I am the creator of black. So this is what

02:32 sort of everybody, I don't know, for some reason always knows about me, but not the other things.

02:38 I do write peps from time to time. In fact, I'm working on one right now. But you'll probably also

02:44 not know about this because that PEP is very boring. It's just about how we're going to make decisions

02:49 in the future. We'll see if it gets selected.

02:51 I think that's actually a pretty big topic, actually.

02:55 Oh, yeah, it is. There's a few sort of, I don't want to say competing, but well, they are competing

03:00 in a sense. There are a few possible models of governance that we can end up with. And the one

03:07 that I'm working on is revolving around the community. It's assuming that there is not going to be another

03:15 benevolent dictator and there's not going to be a very small council or triumvirate or however you call

03:22 it. We're going to sort of look up to the best, like C++ standard comedy. Ah, no, kidding. Well,

03:28 there's other projects that actually use this model like Rust, like ECMAScript and whatnot.

03:34 And Python actually is enough. Well, it's mature enough that in my view, this model would actually

03:42 ensure that we are addressing the interests of the widest sort of number of our users that way.

03:49 Yeah, I think I read the PEP that you're working on and that's really great. I like the

03:53 sort of exploration of the other options that are out there. It's really cool.

03:57 Cool.

03:57 You're also the release manager for 3.8, right?

03:59 Yes, I am. So far, it's going to look at as the most interesting release because it has a very,

04:05 very controversial PEP implemented in it. And at the same time, a rather boring release because now we

04:12 cannot make any other decisions. So there's probably not going to be any more sort of fundamental

04:16 features in it.

04:17 It's quite ironic, isn't it? All right. Very, very cool. Brett, how about yourself?

04:20 Yeah. So my name is Brett Cannon. I work at Microsoft as the dev lead on the Python extension

04:26 for Visual Studio Code. In terms of my Python contributions, I've been a core developer

04:30 since April 2003. So I think that's over 15 years at this point. Unlike both Wukash and Marietta,

04:38 I am not writing a governance PEP.

04:40 Although your name has been thrown around and related to that I've heard.

04:43 Yes, that's true. But yeah, another reason why I'm not writing a governance PEP.

04:47 Perfect. So Brett, why don't you kick us off with our main topic and just tell people what

04:52 is this language summit that you guys have every year?

04:56 Yeah. So basically, we came to a realization at PyCon, what feels like eons ago, that we didn't

05:04 really ever have a chance to sit around as a development team and talk things out. And some

05:11 people would think the sprints were that option. And they were, honestly, early on. But what happened

05:17 was, is as Python's popularity grew and PyCon's own popularity grew, we had more and more people

05:22 coming to us during the sprint saying, hey, I want to get started and contribute. Can you help me out?

05:26 And basically, it's made it such that the sprints at various conferences are not a place for us to

05:32 actually get work done, per se, as a team. So in order to make sure that happened, we carved out a

05:38 day. It always happens at PyCon US during the first day of tutorials, where we get together in a room

05:43 all day long and basically discuss things. Originally, it was kind of a round table-y kind of thing. It's

05:49 slowly grown into a more structured. People come in with a presentation, typically with a question,

05:55 or some specific reason why they're coming to talk about it and present to the room of core developers.

06:01 And basically, it's just a good way for us to kind of hash out or have discussions about where we see

06:06 things going. If we need a decision made, it's easier to make a decision in the room versus trying to do it

06:10 over the mailing list, etc., etc. And I think it's been useful. Lukash and Marietta can answer that since

06:17 they've been doing it far, as long as I think they've been core devs, and they're going to be leading it

06:22 starting next year. So they can have more insight on how they see it going forward.

06:26 Yeah. Sounds good. Do you guys want to add a little bit to what Brett said about the language summit?

06:30 Sorry, you said you guys. You mean you all, right?

06:32 Yes.

06:33 Lukash, perhaps you can say more just because I've only been to two language summits.

06:38 Yeah. So there's several language summits. In fact, we don't always do the European one at

06:46 Europython. The one in the US at PyCon US is, as far as I can tell, an annual event, like as far as I remember.

06:55 So the first one that I actually witnessed was either in 2008 or in 2009 at Europython. So I had a totally different

07:06 sort of view of what that means because the European one was run by Michael Ford, who had a really sort of, you know, like, well, I don't want to use the word, but it was a really happy, hippie approach to like, you know, let's just come on in and, you know, discuss things as we go.

07:29 Oh, you know, everything sort of organically grew, like, you know, people overflew with sort of ranty comments, but it was a very lively conversation.

07:37 The language summits in the US on PyCon US have to be structured a little bit more because there's just way more of us in the room.

07:48 So there is an agenda, there are talks, as Brett said, and whatnot. What I found, like, really amazing about this is that people who are really some of the most active contributors and some of the most senior contributors to Python and related projects are there in the room. So it's, it's something that even if you want it, you just can't buy.

08:10 Yeah, it seems like such an amazing gathering of people. So do you have to be a core developer to attend? Or can anyone attend these?

08:17 So, so far, at PyCon US, like, the rule has been as follows, like, if you are a core developer of CPython or any other alternative runtime, you are automatically in.

08:29 If you are interested and you're in town that day, you can just sort of respond to the invitation and there's going to be a spot for you.

08:39 There's also a few other ways in which you are, you can be present at the summit.

08:45 If you are invited by a core developer to talk about a subject that is of interest to us, then obviously, you can get in.

08:53 Or if you are a member of a project that is written in Python of some, like, notability, right?

09:03 Like, historically, we've had people from the Twisted project, we've had people from the Mercurial project and whatnot.

09:10 Sometimes the scientific stack and whatnot.

09:12 So, definitely, there's some form of deliberate exclusivity to the event.

09:19 But the point of it is mostly that we really have very little time in which we can actually see each other face to face and discuss things that are just way easier to be discussed when you see that other person.

09:34 So, we don't want this to devolve into sort of bike shedding or, you know, like, story of my life sort of event for any random, you know, attendee of PyCon.

09:44 Yeah, that makes a lot of sense.

09:45 Because this is for you all to get together and really be productive and move the language and the runtimes forward, which I think is great.

09:54 I guess the way I'd like to do this is let's just go through some of the various sessions.

09:58 There were quite a few sessions.

09:59 And there's a really nice write-up, actually, on lwn.net, which I'll link to the main write-up and then all the sub-articles about the sessions.

10:08 And maybe we can just start there.

10:11 So, I'm not sure which one of you is most familiar with each session.

10:14 So, I'll let you all jump in.

10:16 So, the first one I want to talk about, there was something with regard to sub-interpreters in Python.

10:22 And I thought the whole concept of a sub-interpreter was pretty interesting.

10:25 And what was the story this time?

10:27 I can take that.

10:28 So, that was presented by Eric Snow, teammate of mine on the Python extension.

10:31 And basically, Eric was kind of giving a status update of what he's trying to do, which is basically CPython itself currently has the concept of sub-interpreters.

10:43 It's used in the Apache web server, actually, for, I believe, ModPython.

10:48 I don't think – I'm not sure if ModWiz uses it.

10:51 But basically, it's a basic way to run multiple interpreters in a single process.

10:57 But it's never really been built out.

10:59 And so, Eric viewed this as a potential opportunity to deal with a concurrency model by trying to make sub-interpreters a first-class citizen of CPython itself.

11:11 So, the idea is basically you will potentially eventually have a module in the Sard library that will allow you to actually create other sub-interpreters in the process, send data across to the sub-interpreters to then be worked on, send results back, and basically give you kind of a message passing style of concurrency.

11:30 Is it a little bit like multi-processing but not as heavyweight in the sense that you don't share the data structures as much?

11:37 Correct.

11:37 Yeah, exactly.

11:39 And a lot of Eric's work has been actually trying to tease out a lot of the global state that Sun's built over the Earth by accident and trying to compartmentalize them better so that it's very obvious what is connected to a specific interpreter.

11:54 So, regardless of where the sub-interpreter work goes, it's been nice to kind of clean up that code to help centralize all the data structures and such.

12:02 Yeah, that's really interesting.

12:04 One of my first thoughts when I saw this, I mean, you were talking about in terms of concurrency, which is really interesting.

12:09 My first thought went to compatibility, right?

12:11 Could we take something that runs in Python 2 and somehow get it to stay closer to other code?

12:18 Or is there any thought of that?

12:20 Or is it more of a concurrency thing?

12:21 It is very much a concurrency thing from our perspective.

12:24 You Dropbox just did a blog post yesterday about how they transitioned from Python 2 to Python 3.

12:30 And it sounds like they somewhat went down that road in their custom Python version of CPython.

12:37 So, it sounds like it'd be definitely a potential way of doing it.

12:41 But I think from Eric's perspective, with Python 2 hitting EOL January 1st, 2020, the amount of time he still has to put in it to make it work isn't going to be worth his while to look into that.

12:52 But I'm sure someone could potentially make that work with a lot of effort.

12:55 We don't want to encourage that behavior, do we?

12:57 No.

12:58 This portion of Talk Python to Me is brought to you by Linode.

13:02 Are you looking for bulletproof hosting that's fast, simple, and incredibly affordable?

13:06 Look past that bookstore and check out Linode at talkpython.fm/Linode.

13:11 That's L-I-N-O-D-E.

13:13 Plans start at just $5 a month for a dedicated server with a gig of RAM.

13:18 They have 10 data centers across the globe.

13:20 So, no matter where you are, there's a data center near you.

13:23 Whether you want to run your Python web app, host a private Git server, or file server, you'll get native SSDs on all the machines, a newly upgraded 200 gigabit network, 24-7 friendly support, even on holidays, and a seven-day money-back guarantee.

13:38 Do you need a little help with your infrastructure?

13:40 They even offer professional services to help you get started with architecture, migrations, and more.

13:47 Get a dedicated server for free for the next four months.

13:50 Just visit talkpython.fm/Linode.

13:53 How about the next session that you all covered was modifying the Python object model.

14:00 This one came from someone outside of the core developer team, right?

14:04 That is somebody that I actually invited to the Language Summit.

14:08 It's Carl Shapiro, who works currently with Instagram on making Python sort of handle the next billion users for the social network.

14:20 We've seen some tremendous growth over the years on Instagram.

14:24 We are extremely grateful to have this runtime and be able to run the social network on it.

14:31 In fact, in the year where we switched from Python 2 to Python 3, we've seen some crazy feature growth and user growth at the same time.

14:40 And we actually switched the runtime version, like, you know, just sort of all in parallel.

14:45 So that was pretty amazing.

14:46 Yeah, I think that whole transformation is really a case study in how large organizations should do it.

14:53 You guys did an awesome job there.

14:54 Yeah.

14:54 So this was a massive project.

14:56 There's a bike on 20, what is it?

14:59 17?

14:59 I think it was 17, yeah.

15:01 I think, yeah.

15:02 Like a keynote about this.

15:03 So I highly recommend it.

15:05 It was a fun journey.

15:06 However, like since then, like we've been seeing that like we are paying some of the price for things that we don't have to pay the price for.

15:15 But at Instagram scale, like, you know, we're running many, many servers with the Django processes that actually serve users.

15:23 So we observe that some of the things that the CPython interpreter is doing can be optimized the way.

15:30 Or rather, we can change the interpreter in a way where those things can be sometimes pre-computed before runtime.

15:38 Like they can be done like circa compile time, if you know what I mean, like for Python, right?

15:43 Like obviously we are running like out of BYC files, which are, you know, created by the interpreter.

15:49 So whatever we can do there, like, you know, instead of every time that we start up the process would be nice.

15:56 In particular, Carl has a lot of background on various implementations of virtual machines for dynamic run time, dynamic like languages.

16:08 Like for example, I don't know, like R2VM Dart and whatnot.

16:13 So he has plenty of experience to know like where we could actually find pieces that we can optimize.

16:19 And in particular, he presented ways in which, for example, we could use an observed fact that most objects in Python don't actually change their shape during runtime.

16:34 So what that means in regular language is that when you're creating an instance at runtime in Python, Python doesn't have any expectation currently whether this instance is going to grow or, you know, remove attributes later on in the lifecycle of the object.

16:54 It's very dynamically sort of built in the sense that you can just change those things as you go.

17:02 You can add things not only to the objects.

17:04 You can add things and remove things from the classes themselves, you know, like freely, like whenever you want.

17:09 But in practice, what is happening is that a large majority of objects have all their state created and they're done their init methods.

17:20 Right.

17:20 You could almost use something like slots and like lock them down and make them more efficient.

17:25 But people don't do that.

17:27 They're not even actually recommended to do that, right?

17:29 Yeah, so slots, in fact, is a very sort of implementation of what we would be after.

17:35 What we would be after is more akin to the concept of hidden classes in JavaScript that V8 and a bunch of other runtimes implement.

17:43 What that means is instead of storing your attributes on the object as a hash table, right, like in the sense that you can just add things and remove things,

17:56 but then the cost of retrieving every attribute is rather high, what you can do is you can understand what the shape of the object,

18:05 the shape meaning efficient data structure in memory so that it's no longer a hash table.

18:10 It's simply consecutive array that you can just index, which has, you know, very nice property of, you know, having way faster access at runtime.

18:20 And in fact, we experimented with stuff like this at Instagram.

18:24 That actually did win us some nice, you know, improvements in performance.

18:29 So that change was suggested as something that could be, in fact, implemented in a future version of Python.

18:35 That's super interesting.

18:37 Do you think some of these changes are going to make it into the Python, into the Python runtime, into CPython in the future?

18:43 So definitely we need to be careful about, like, keeping Python what it is, right?

18:48 So we don't want to become a bit more performant, but way less dynamic.

18:54 Like, Python's entire success is built around the fact that it's very hackable, right?

18:59 Like, you can change things as you go at runtime.

19:02 And a lot of our day-to-day work is actually built on top of that frame of mind.

19:09 Like, for example, like, if you're mocking things in Python, right?

19:13 Like, during your unit test and whatnot.

19:15 Like, this is a very dynamic feature.

19:18 So, like, taking stuff like this away could potentially hurt, you know, our users.

19:23 So we don't want to do that.

19:25 Right.

19:25 A 10% increase in performance is not worth the loss of unit testing capabilities or massive re-architectures.

19:33 Absolutely.

19:34 So what we want to do instead is for the kinds of things that don't sacrifice user-visible functionality,

19:41 we want to have optimizations that actually let us do things faster.

19:46 Do, like, do less work.

19:47 If you can do less work, everybody wins.

19:50 However, we also have another constraint in CPython that is maybe not always visible,

19:56 which is that the runtime that we maintain has to be maintainable for the team that is actually in it for the long haul, right?

20:06 So if you would drop a much more performant piece of runtime, you know, on our labs,

20:13 we would thank you, but we would be also rather concerned about, what are we going to do with it now, you know?

20:20 If this is a very brittle and complex piece of code, we need to take into account that we need to release the next version of Python

20:29 and then the next version of Python, and there's going to be different platforms that we're going to have to run on.

20:34 In fact, just today, I fixed a bug in black where people noticed that, oh, we cannot run on Android phones.

20:41 You're like, I didn't test it on Android phones.

20:44 I mean, seriously, like, are you formatting Python code on Android phones?

20:49 Like, that didn't occur to me as an important feature, but people really have needs like this, and those needs change in time and whatnot.

20:58 So the point I'm trying to make is that we do put a lot of weight to making sure that we understand the piece of software that we're maintaining, right?

21:08 So there's definitely some compromise that we need to do there.

21:12 We cannot bring in another million lines of code, you know, like from some benevolent contributor and then be left with maintaining that thing.

21:22 So yes.

21:22 This goes all back to Brett's thing about open source expectations and, you know, giving away puppies and things like that.

21:28 Absolutely.

21:28 But it sounds like there's some really interesting possible performance benefits.

21:32 Yeah, that's cool.

21:32 The next one up is the galectomy.

21:35 Larry Hastings' galectomy.

21:36 The attempt to remove the GIL.

21:39 Marietta, you want to take this one?

21:40 What I know is it doesn't seem like it's going to happen, but Larry had some issues with it.

21:47 And I think he was looking to get inspiration from Carl's previous talk.

21:54 But last time I checked from Larry, it doesn't seem like it's making any progress.

21:58 Yeah, I think that's kind of the summary of this whole talk is like, I've been working on it, but it's not really going anywhere, right?

22:06 I mean, the sub-interpreter thing that Brett mentioned, part of that attempt is to break free the GIL.

22:10 We have multiprocessing.

22:12 We have asyncIO, which is not really subject to the GIL because of the way it works and things like that,

22:17 at least if it's IO-bound, what you're awaiting.

22:20 So I don't know, I'm not sure it's as necessary as it used to be, but it sounds like it didn't really go anywhere, right?

22:27 No.

22:27 That's too bad, but it's good to, I think it's a cool project and I really hope he makes progress, but it doesn't sound like it, right?

22:34 Right.

22:34 All right, so that was a pretty short update, but that's cool.

22:37 Why don't you take the next one as well?

22:39 Because did you actually do this presentation?

22:41 Yes, so I proposed that we started using GitHub issues instead of Roundup,

22:47 which is bugspython.org.

22:49 So BPO bugspython.org is an instance of Roundup that we've been maintaining

22:55 and it's been working for us, but I think our workflow could be better if we started using GitHub issues.

23:04 Yeah, and Brett actually had helped drive the project to move CPython code.

23:10 So the code is on GitHub already, but the issue tracking had been somewhere else,

23:14 and so you were like, well, why not just in GitHub where it is, right?

23:18 Exactly, and everything else on Python uses GitHub issue trackers except CPython,

23:25 like our peps in GitHub, our dev guide workflow issues, everything is in GitHub already.

23:32 So I think it's good to start to explore why don't we start using GitHub for CPython as well.

23:40 I think it's a great idea.

23:41 I think it's absolutely good thing that CPython code is on GitHub.

23:45 I know it's just a source control system, but I think it opens up people's willingness to participate and interact with the code way more for being there.

23:54 Yeah, and personally, I find it always odd that I have to jump from one interface to another,

23:59 like looking at the issues in Roundup that looks very different, and then I have to go to GitHub to create my pull request.

24:06 Like for me, it's distracting.

24:08 Yeah.

24:09 So I think it's one benefit that we get this unified experience, not having to jump from one place to another.

24:17 And in fact, I've started writing a PEP for it, PEP 581.

24:21 So that's our plan for starting using GitHub issues.

24:26 And we've got to discuss more about this during the sprint a couple weeks ago.

24:32 Yeah.

24:33 Ezio actually went around and asked all other core developers who attended whether they would like to start using GitHub issues.

24:42 And most people are okay, like plus zero, at least not totally opposing it.

24:50 So I think we might, there's no decision yet because we don't have PDF to pronounce on this.

24:57 Yeah, that's another session we're coming up on shortly.

24:59 Yeah.

25:00 Brett, since you were so involved in GitHub, what do you think about this?

25:03 The main reason when we moved over to the Git repo, we didn't move the issues, was twofold.

25:08 One was moving the repo over itself was a good amount of work just because we were moving from McEarrel to Git on top of hosting and all that.

25:17 But also there were some initial knee-jerk pushback, like don't change too much underneath me.

25:22 And so I basically just only had so much energy in the world because, I mean, the GitHub transition took two years.

25:30 Like people don't realize this.

25:31 It took about a year of discussion and a year of actually making it happen.

25:34 So I can only imagine how much more time I would have added to do it.

25:38 And how much potential pushback, right?

25:40 Well, exactly.

25:41 And I just only have so much time in the day to deal with pushback.

25:45 And I got buy-in on the Git stuff, so I decided to not press my luck.

25:48 I think there's definitely possibilities for having an improved workflow.

25:53 Like a lot of the work Marietta has shown the team is possible using bots through Ms. Islington and the stuff she and I have done with Bedivere.

26:00 Really show that a lot of workflows can get automated and made fairly cleanly.

26:05 I do know there are some core developers, especially ones that have been around for quite a while, who are kind of attached around up in terms of certain feature sets.

26:12 So my suspicion is if we can add the missing features that they have latched onto on Roundup and somehow mirror them as appropriate or find an alternative that gives them the same result that they're after.

26:25 I think it's definitely a good idea.

26:28 And in general, I'm for it because I live in GitHub already for work.

26:33 And as Marietta said, everything else is over there already.

26:35 So I'm definitely at least a plus zero to a plus one on this.

26:39 Perfect.

26:39 That sounds good.

26:40 So, Lucas, the next one was shortening the Python release cycle.

26:45 I suspect that might have had some input from you.

26:48 Yeah, cool.

26:48 Tell us about that one.

26:49 I still do have this idea to attempt to talk about this on the next Language Summit.

26:56 And maybe we're going to get somewhere there with it.

26:58 But for now, I postponed it at least for 3.8 because currently there is nobody to work with on actually making a decision about it or not.

27:09 Right.

27:11 So what are we trying to decide?

27:12 What was the idea about?

27:14 Well, Python has a release schedule that is currently 18 months.

27:19 Right.

27:20 So every 18 months, every year and a half, we're going to get a new major release of Python.

27:25 Like, you know, 3.7 was just out mid-year now.

27:29 So, yeah, like, you know, add 18 months and you're going to get a new release of Python late next year.

27:36 That'll be exactly when Python 2 goes out of support, right?

27:39 Pretty much.

27:40 Like, almost, yes.

27:41 Like, I'm pushing this, like, a few weeks here and there so that we can, you know, have some more interesting, like, you know, cadence in terms of, you know, doing sprints during Python US and maybe having, like, our annual course sprints somewhere else.

27:56 So we try to have those events productive.

27:59 So, like, we're not super tight, like, to actually have, like, the release exactly every 18 months.

28:05 Like, we always have some leeway.

28:07 You know, for example, when we released Python 3.6, we had a course print at Facebook, which was just coinciding with the beta 1 release.

28:19 So, like, we're not stabilizing the release.

28:33 So, originally, that beta was supposed to happen, like, the week before the sprint.

28:40 But that would make the sprint very, very boring and rather sad, you know?

28:45 So, I talked with Ned, who is the release manager for 3.6 and 3.7, about just pushing it just a week and a half into the future so that we can actually sort of rally up during the sprint and finish up all the fancy features that maybe are just almost ready but not quite.

29:05 And, in fact, as far as far as I can tell, that was the most productive week in the project's history, even up to today.

29:12 We're trying to do this every time pretty much now.

29:16 But why shorten the release cycle?

29:19 Well, the problem with having a release every 18 months is that it builds up this mode of work where most of the time spent on the release is just vague ideas, like having things implemented and left, you know, in a state of there are things that need still to be done.

29:37 And then the few weeks before the beta cut is sort of aggressive sprinting on making sure that we actually make it in time to, like, meet the deadline and have our feature shipped.

29:50 But this doesn't really create a very healthy rhythm of development in my experience.

29:57 And actually, working at Facebook, I learned that, you know, releasing early and releasing often, like, works very well.

30:05 It does have a lot of good features, like, it does decrease the size of our release, right?

30:12 Like, if we released, say, every six months, that would mean that Python 3.8 and 3.9 and 3.10 would have way fewer differences between them.

30:21 So migrating between those releases would also be easier for the user.

30:26 There is a price to pay, right?

30:28 Like, now your tox matrix on your open source project is way bigger because there's more versions of Python.

30:35 It pushes more work onto the release managers and so on and so on.

30:40 So definitely something that people need to agree with.

30:44 But I still believe that we would be better off having smaller releases that are released more frequently.

30:50 That sounds good to me, especially the year thing.

30:52 You could schedule the release to be in the fall or something.

30:56 So you always have at PyCon sort of a sprint before the beta period closes.

31:02 Things like that, right?

31:03 For example, yes.

31:04 Now it's kind of out of phase, basically.

31:07 Every other year or something weird like that, right?

31:10 So yeah, I think it makes a lot of sense.

31:11 It makes it pretty predictable.

31:13 The major tech conferences are in May and June.

31:16 We know that Python releases in the fall.

31:19 I don't know.

31:19 Something like this, right?

31:20 Like, that's just how it might go.

31:22 This portion of Talk Python to Me has been brought to you by Rollbar.

31:27 One of the frustrating things about being a developer is dealing with errors.

31:30 Relying on users to report errors, digging through log files, trying to debug issues,

31:35 or getting millions of alerts just flooding your inbox and ruining your day.

31:40 With Rollbar's full-stack error monitoring, you get the context, insight, and control you

31:44 need to find and fix bugs faster.

31:46 Adding Rollbar to your Python app is as easy as pip install Rollbar.

31:50 You can start tracking production errors and deployments in eight minutes or less.

31:54 Are you considering self-hosting tools for security or compliance reasons?

31:59 Then you should really check out Rollbar's Compliance SaaS option.

32:02 Get advanced security features and meet compliance without the hassle of self-hosting,

32:07 including HIPAA, ISO 27001, Privacy Shield, and more.

32:12 They'd love to give you a demo.

32:13 Give Rollbar a try today.

32:15 Go to talkpython.fm/Rollbar and check them out.

32:20 Next one is about unplugging old batteries, right?

32:24 Python is said to come with batteries included.

32:26 That means it has rich standard library and all sorts of stuff you can use.

32:29 But every now and then, those go out of fashion.

32:32 People stop using them.

32:33 Like, what do you do then, right?

32:34 Do you still have to maintain them?

32:35 So who wants to take this one?

32:38 I'll take it because it's been...

32:40 I've been rummating on this in the back of my head for...

32:42 It feels like a decade now.

32:44 So this was a presentation by Christian Hamies.

32:50 I'm going to butcher Christian's last name.

32:51 I'm so sorry.

32:52 Where he basically has a rough draft of a PEP where he has suggested some modules that we can potentially remove from the standard library.

33:02 And we've done this once before in a large movement when we went from two to three where I actually personally went through and got a bunch of modules deprecated and removed.

33:13 And the main reason we do this on occasion, some things just turn out to not be useful anymore.

33:20 For instance, a good example is moving from Python two to three, we got rid of the gopher lib module because who runs gopher anymore?

33:27 Heck, who even knows what gopher is, right?

33:31 Yeah, gopher, that was one of the originals.

33:33 They tell.

33:33 I know what that gopher is.

33:35 But the key thing here is there is a burden of cost of maintenance for every single module we have in the standard library.

33:44 Even if it's just sitting there, there's still the cost of updating it.

33:49 For instance, we just made async a proper keyword in 3.7.

33:53 That means any module that was using async as a variable name had to be updated.

33:57 There's potential bug reports.

34:00 There's feature requests.

34:01 Even if all those were ignored, they still take time because, for instance, when you go through and triage issues, you still have to see that it's an issue.

34:08 You still have to read it.

34:09 You still have to choose not to do it.

34:11 So there is a time sink regardless of how much maintenance you actually put into it.

34:16 So I personally have always wanted to kind of potentially scale back the amount of modules we have in the standard library because I don't think a lot of people realize we literally have hundreds.

34:24 I wrote a really quick script the other day to count the number of .RST files in the library directory that had either letters, numbers, or underscores.

34:33 And the count was 248.

34:36 So that's a rough count of how many top-level modules there are in the standard library.

34:40 I mean, that's not an insignificant number.

34:43 And you have to remember that there are only 93 core developers in total.

34:47 And over the past year, only 46 people across the globe have submitted a PR that got merged.

34:54 So there's definitely a maintenance burden here where the amount of code to be maintained is not in a good ratio to the amount of people who are able to help keep it going.

35:04 So Christian was basically saying, we got some modules we need to get rid of.

35:08 A good example from Christian's PEP is, how many modules do you think there are specifically for processing sound files in the standard library?

35:16 So there's three.

35:17 There should be one.

35:18 No, there's three.

35:19 Yeah, there's three.

35:20 There's AIFF for AIFC module.

35:24 There's the SunAU module.

35:26 And then there's the Wave module.

35:28 Wow.

35:28 Exactly.

35:29 So why do we have this?

35:31 Why do we even need this kind of thing in the standard library?

35:34 So that was Christian's point, was there are some things that literally just don't have a purpose anymore because they were put in back when basically Guido accepted anything.

35:42 Because there was no PyPI and there was no distribution model except CPython itself.

35:47 Right.

35:47 Now you can pip install the tools you need to work with MP3s or whatever it is.

35:52 And why does that need to be part of the standard library, right?

35:55 Exactly.

35:55 And we've never had a discussion as a group to really come down with a set of guidelines of what should or should not be in the standard library.

36:02 And unfortunately, as we keep alluding to, lack of governance means there's no one to have that discussion with at the moment.

36:09 But it is something we're probably going to have to decide.

36:11 Do we want to stay as heavily batteries included now that PyPI exists and is so solid?

36:17 Do we want to scale it back and be more targeted towards what a potential script writer needs to maintain their machine?

36:24 But as soon as they go past anything standard, they're just going to be expected to go and get it from PyPI.

36:30 Yeah, that's an interesting idea.

36:32 Like, I don't think, for instance, we would add argparse necessarily today.

36:35 We could have stayed with getopt or something simple and say, if your command line interface is going to get that complex, you need like get level subcommands and stuff.

36:43 There's probably going to be something developed better on PyPI that you're going to want to grab versus something we should ship in the standard library that we have to maintain.

36:51 And this is also, by the way, why we don't have requests in the standard library because it develops faster.

36:55 But also pulling requests would also mean pulling Euralib 3.

36:58 And it's just one thing that goes back to maintenance.

37:01 How do we keep this sustainable and everyone happy with the code quality while not burning out from the fact that there are nearly 200 open issues against argparsealone?

37:11 Right, and the release cycle as well being 18 months means, you know, a new HTTP thing comes out, right?

37:19 It would take forever to get that actually into Python's main HTTP library, which would be weird.

37:24 Yeah, so basically Christian's just saying we need to evaluate what's in there right now and make a decision of what really should still be there and what maybe should go out.

37:32 And he's fairly conservative because, as I said, we've not had a conversation of general guidelines of what should qualify for something being in the standard library.

37:40 Yeah, it makes a lot of sense to me.

37:41 Like, I have no expectation that I can write a proper web app without going to PyPI and using PIP.

37:46 It just doesn't make sense.

37:48 But there's obviously the compatibility thing, right?

37:51 You don't want to rip out things that someone might be using and break them.

37:54 So, definitely worth considering.

37:55 Another thing that Christian did was talk about Linux distributions and Python 2 and with the end-of-life Python 2 coming.

38:03 How are distributions that ship Python 2 going to deal with that?

38:06 Let's keep moving because I think we have some more that are more relevant.

38:08 But I think, you know, people can read up on that.

38:10 That's pretty interesting.

38:11 Lucas, maybe you should take this one because I know you do so much with static typing.

38:14 The next session was an update on Python static typing.

38:17 Static typing is still very much in progress.

38:20 There's a bunch of things that is happening with it.

38:24 One of the things that was discussed was how should we package external typing information for projects which are themselves not typed yet?

38:36 Or if they want to be typed now, how do we expose this information to the type checker like mypy?

38:44 In fact, this is one of the reasons why TypeScript and its ecosystem is so successful because the equivalent there, the definitely typed website and the related integration with NPM that the types provide has proven very successful.

39:05 So Ethan has been working on PEP about this, PEP 561.

39:11 And as far as I can tell, it's already landed and PyPI already has the required support for it and whatnot.

39:19 So in the future, we are going to be able to slowly move away from the model where TypeShed, the repository and the library shipped with mypy and other type checkers, is the sort of library of the collection of types for the entire world.

39:35 That's obviously very hard to maintain.

39:38 That's obviously very hard to do well.

39:40 So we would like to decentralize where the stops are held.

39:44 Yeah, that makes a lot of sense.

39:45 Like shipping them with, say, when you pip install, if I say pip install SQLAlchemy, it would be great if the stub file that defined the types just came as part of that just landed on my file system.

39:57 I didn't have to think about it.

39:58 Yeah.

39:58 So there were a few other updates.

40:01 I want to be like short now.

40:02 So like I don't actually steal the entire episode.

40:05 But there's a number of other peps that were typing related, like PEP 560 moved some of the typing functionality into the core interpreter so that it's now way faster.

40:17 And PEP 563, very dear to my heart, was about making the typing module a bit more usable in terms of just, I don't know, aesthetics, right?

40:27 Like enabling things like forward references for classes, which means you can actually use a type of a class within that class or before that class is even defined and whatnot.

40:40 And the most interesting probably is 544, which is protocols.

40:44 So like duck typing for static type checkers.

40:49 This was a rather long presentation and whatnot.

40:51 Like I find this top interesting.

40:53 So, you know, we can do an entire episode on that alone.

40:57 Yeah, absolutely.

40:57 And the idea with protocols being that's kind of the way Python type system works.

41:02 Like as long as I can call this function and it has this field, like we're good.

41:06 And so this is like bringing that into the type annotations world saying, well, whatever you pass here, it has to have this attribute and this method, right?

41:13 Something to that effect.

41:14 Yeah, nice.

41:15 Yes, exactly.

41:16 There was a discussion on whether virtual environments are serving us well, especially around the teaching.

41:21 Brett, this one actually features Steve Dower, your colleague at Microsoft.

41:27 So tell us about it.

41:28 Yeah.

41:28 So I believe there's actually already a pip now up on the proposal.

41:35 Basically, if you follow certain people on Twitter, one of the common issues they have is trying to explain virtual environments to beginners.

41:42 And part of that's because pip automatically installs to your global environment and your user environment by default.

41:49 There's a long running issue on pip to change that default.

41:52 That would be nice.

41:52 Yeah.

41:53 Feel free to comment on that issue.

41:54 I'm sure they love it.

41:55 Basically, virtual environments are really what everyone wants, but it's an extra level of work to try to explain how that functions.

42:04 And Steve Dower and Kuchel Dauer and various other people got into a conversation over this, where the idea of more or less having something equivalent to node modules directory came up.

42:17 The current name is proposed as Dunderpy packages.

42:21 And basically, the idea is your dependencies should be installed in the local directory.

42:25 And Python would more or less add that to your system path by default.

42:29 And you would teach tools of pip to just install in there when appropriate.

42:36 And that's it.

42:38 It's a fairly simple concept.

42:40 Right now, once again, no governance means it really hasn't been discussed beyond getting the pepper written.

42:45 But probably the discussion is going to end up revolving around whether that is enough to warrant the added complexity of having this yet another way to specify where your dependencies can live.

42:57 And does it solve enough of the problem?

42:59 Right.

42:59 Are you just changing one hard way to explain things for another and then making it worse by having pip ENV, PIP, this other way, virtual environments?

43:09 It's like, ah, can't take it anymore.

43:11 Exactly.

43:11 So that's going to be the real question, I think, is does it solve the case for enough people to warrant the cost of introducing yet another way to do this?

43:20 Because virtual environments won't go away because they do serve a purpose.

43:24 It's whether or not this will simplify it for 90, 95% of the world and that last 5% are going to be advanced users who will know how to use virtual environments.

43:32 Or do we just have to push the tooling to just get better to make it easier to work with virtual environments as it's such as pipenv or something else?

43:41 Yeah, I don't know exactly what the right answer is.

43:44 It's a tough question.

43:44 But it's, I feel like no matter which of the options you choose, they're not quite there.

43:50 They're close.

43:52 I use just virtual environments straight up with pip and it's all good, a V&V module.

43:57 But still, I hear you.

43:59 Very interesting.

43:59 Okay, so I think we have time for just two more where we're getting close to the end of our time we have together.

44:05 So the next one, I'll let whoever wants this take it.

44:08 This is PEP 572.

44:11 That was the big in-line assignment PEP that caused a lot of grief and Keto stepped down over.

44:18 And now what are we going to do in terms of decision making?

44:21 And it sounds like almost everything we've talked about hinges upon this.

44:24 Who wants it?

44:25 Well, I see everyone's leaping up to take the lead on this discussion.

44:28 This one's a hot potato, man.

44:30 No one wants this.

44:31 All right.

44:31 So, yeah.

44:32 I can do it.

44:33 It's fine.

44:33 So, basically, it was a discussion.

44:35 This was – so, the language sub button happened in May.

44:38 Guido stepped down, I believe, in July.

44:41 So, this all predated Guido taking his indefinite vacation as BDFL.

44:48 And, basically, it was a discussion of what can we do going forward to manage our PEP process better?

44:55 Because PEP 572, which is assignment expressions, was a very, as Michael alluded to, a very hot topic where a lot of things were discussed on Python ideas.

45:09 And, clearly, it got resolved.

45:11 And it was actually a reasonable discussion.

45:13 And then it went to Python dev as part of the PEP process.

45:16 And it all got rehashed yet again.

45:17 And, actually, it got a little – I don't want to say pushy is quite the right word.

45:21 But it got a bit more vehement.

45:22 And some people were making rather grandiose statements.

45:26 Like, at one point, I think someone, a core dev, said, if this gets emerged, I will refuse to review any PR that uses this feature.

45:32 Like, it got really – I personally think overblown.

45:36 And, which is partially why Guido just said, I never want to have to fight this hard to defend my decisions ever again.

45:41 I'm taking vacation.

45:42 So, this was mainly a discussion of what can we do to try to get the PEP process to not be so burdensome while keeping it, obviously, for our needs of recording history.

45:51 Making sure we don't have to have discussions about every single solitary suggestion someone has.

45:57 And try to find a good balance of load versus, basically, proper gatekeeping to make sure we don't feel overloaded.

46:04 Yeah, that's definitely important.

46:05 I guess, just to add, I think it's also something we are all looking into improving in our governance PEPs.

46:14 We want to try to look into how the PEP process has been going in the past and whether there are things we can improve on how to make decisions.

46:23 And as well as where should discussions happen.

46:28 Those are also being discussed in the governance PEPs.

46:31 Yeah.

46:31 Okay.

46:32 That's an important one.

46:33 All right, Marietta, I'm going to give you the last one because we are out of time.

46:37 But we got about three minutes to talk about a pair of presentations, one of which you gave about mentoring and diversity in Python.

46:44 Want to give us the wrap on that?

46:46 Yes.

46:46 So, basically, the whole core team has been trying to improve our diversity.

46:52 If you remember a few years ago, there was no women at all in the language summit itself.

46:59 But this year, we're trying to improve our diversity among our own contributors team.

47:17 And I shared some ideas.

47:20 I got some ideas from Sage and shared that with the core developers.

47:26 So, we've been doing some things.

47:29 I know Guido himself mentored a few women who now are core developers.

47:35 I know Victor himself is looking.

47:38 He told me he's looking for women to mentor.

47:41 And we have set up office hours over in Zulip.

47:47 And I know Zach.

47:48 Zach, where he has a calendar where people can sign up to get one-on-one mentorship with him.

47:57 All of these are all ways we are trying to do in making it to lower barrier into contributing, welcoming more underrepresented people.

48:08 I would love to see more diversity in the core developer team.

48:12 But also, it's just good for the community overall, I think.

48:14 Absolutely, yeah.

48:15 Can we make sure we put those links in to the show notes?

48:19 Like, if you help me get those links.

48:20 So, people out there who may want to contact these folks to get mentorship or consider this process.

48:26 For sure, yes.

48:27 So, we do have the core mentorship mailing list.

48:29 I will share.

48:29 And if you look at the dev guide, there is a link for core developers office hours there.

48:35 But I will share the links.

48:36 Great.

48:37 Thank you.

48:37 All right.

48:38 I think we have to leave it there because we're on a schedule.

48:41 Can't just talk forever.

48:42 So, thank you all for being here.

48:44 Marietta, Lucas, Brett, thank you for sharing this whole experience.

48:48 And I'm looking forward to hearing about the 2019 Language Summit when it happens.

48:52 Thank you so much.

48:53 Thanks, Michael.

48:53 You bet.

48:54 Bye, everyone.

48:54 Bye.

48:56 This has been another episode of Talk Python to Me.

48:59 Our guests on this episode have been Marietta Wijaya, Lucas Lenga, and Brett Cannon.

49:05 And this episode has been brought to you by Linode and Rollbar.

49:08 Linode is bulletproof hosting for whatever you're building with Python.

49:12 Get four months free at talkpython.fm/linode.

49:16 That's L-I-N-O-D-E.

49:18 Rollbar takes the pain out of errors.

49:21 They give you the context and insight you need to quickly locate and fix errors that might have gone unnoticed until your users complain, of course.

49:29 As Talk Python to me.

49:30 As Talk Python to me listeners, track a ridiculous number of errors for free at rollbar.com slash talkpythontome.

49:36 Want to level up your Python?

49:38 If you're just getting started, try my Python jumpstart by building 10 apps or our brand new 100 days of code in Python.

49:45 And if you're interested in more than one course, be sure to check out the Everything Bundle.

49:48 It's like a subscription that never expires.

49:51 Be sure to subscribe to the show.

49:53 Open your favorite podcatcher and search for Python.

49:55 We should be right at the top.

49:56 You can also find the iTunes feed at /itunes, Google Play feed at /play, and direct RSS feed at /rss on talkpython.fm.

50:06 This is your host, Michael Kennedy.

50:08 Thanks so much for listening.

50:09 I really appreciate it.

50:10 Now get out there and write some Python code.

50:12 I'll see you next time.