Python is undergoing a performance renaissance.
We already have Python 3.11 20-40% faster than even Python 3.10.
On this episode, we'll dive into a new proposal to make Python even more efficient using lazy
imports laid out in PEP 690.
We have all three folks involved on the episode, Carl Meyer, Hermann Bravo, and Barry Warsaw.
Are you ready to get into making Python faster still?
Let's dive in.
Welcome to Talk Python to Me, a weekly podcast on Python.
This is your host, Michael Kennedy.
Follow me on Twitter where I'm @mkennedy and keep up with the show and listen to past
episodes at talkpython.fm and follow the show on Twitter via at Talk Python.
We've started streaming most of our episodes live on YouTube.
Subscribe to our YouTube channel over at talkpython.fm/youtube to get notified about upcoming
shows and be part of that episode.
This episode is brought to you by Sentry and us over at Talk Python Training.
Please check out what we're both offering during our segments.
It really helps support the show.
Transcripts for this and all of our episodes are brought to you by Assembly AI.
Do you need a great automatic speech-to-text API?
Get human-level accuracy in just a few lines of code.
Visit talkpython.fm/assembly AI.
Hello, Barry, Erman, Carl.
Welcome to the show.
Hey, hello.
It's wonderful to have you all here.
I'm very excited about the work that you're all doing around Python performance.
We're going to focus on imports and this PEP that you three proposed today, but it's really
just the tip of the iceberg in terms of a bunch of cool stuff that's going on.
So I'm very excited to dive into that with the three of you.
Now, before we get to it, though, let's just do a quick round of introductions.
Barry, you've been on the show before talking about 1994 Python stuff and other things.
So maybe just a quick introduction for yourself.
Yeah.
Barry Warsaw.
So I'm still here hanging around, I guess.
You know, I should mention, you know, as we get into this PEP, though, I'm really just
the sponsor.
You know, I got to hand a lot of thanks to Jeff Hermann and Carl for doing all the work
on it.
You know, most PEPs these days, you know, require a core developer to sponsor.
I'm just super interested in the topic.
I think it's a super clever approach.
And so I think it'll help specific needs that I have at work.
And so I was really eager to sponsor it.
Yeah.
Fantastic.
Yeah.
All right.
Very cool.
Hermann, how about you?
I'm just working at Mana with Carl and I wrote the initial approach for Licey eports.
And well, here I am.
Right on.
Carl, quick introduction.
I've been around the Python community for a while.
I think the first maybe semi-notable thing that I did was write the first version of PIP
uninstalled back in 2009.
And that led to being a maintainer of pip and virtualen for a while.
And I worked on the Django core team for a while.
And so, yeah, I've been doing Python things for a long time.
And I've been working at Meta.
Those are a lot of big projects there.
Yeah.
They're projects that I was using.
And so I was interested in working on them.
And same goes for Python itself.
Yeah, absolutely.
And sorry, I kind of cut you off there.
You said working at Meta since 2016.
Is that what you said?
Yeah.
I've been working at Meta since 2016.
Working mostly on how Meta uses Python, how Instagram uses Python.
Yeah.
There have been some really cool looks inside what's going on, especially at Instagram.
there with some of the typing talks that Lucas gave, as well as sort of suspending the garbage
collector for other various things.
And a lot.
Was that you guys?
I think it was, right?
Yeah.
We had a whole.
A lot of neat stuff.
A whole saga.
We turned off the garbage collector, turned back on the garbage collector.
There's been multiple blog posts along the way explaining why we've done each of those
silly sounding things.
Sure.
I mean, they seem insane.
Well, a lot of the projects that a lot of the stuff that's coming out of here has to
do with the Cinder project.
Right.
Yeah.
And maybe, I don't know who's best to give the introduction for Cinder, but Cinder's a really
cool project about, you know, taking a whole bunch of optimizations and specializations you
all have done and sharing that back with the community a little bit, right?
Yeah.
So we started Cinder in, I think, 2017 or 2018.
There was actually two projects kind of started simultaneously.
We realized around that time that the trajectory for Instagram's kind of server footprint was
not really sustainable just in terms of how much server CPU time we were spending running
Python code.
And so we kicked off two projects.
One was called, it's now called Sky Bison.
It was like a ground up rewritten Python interpreter, you know, using all the modern dynamic language
VM ideas, like a moving garbage collector and all these different things.
We weren't the first people to try that.
And we also weren't the first people to fail.
And that project was wound down last year, just weren't able to get the performance, particularly
trying to emulate compatibility with the C API and all the C extensions, which is the same
reason many prior rewrite efforts didn't go very far.
So at the same time as we kicked off Sky Bison, we had sort of kicked off what we thought was
a short-term project of just like, what can we squeeze out of CPython?
Where can we get a little more performance out of it?
And that turned into Cinder and then ended up kind of becoming our primary approach to
Python performance.
Isn't it always the story that the C interop stuff is the big sticking point here?
I mean, Barry, you must have seen a whole bunch of examples from a core-to-of perspective on
that, right?
If we could just change the gill, but the C API defense, but if we could just, but, you
know.
Yeah.
I mean, it's both, you know, Python's, you know, advantage, you know, pros and cons,
right?
Like the approachability and the usability of the C API has led directly to the incredible
ecosystem of extension modules.
Right.
But those also are also the hindrance for moving, you know, ahead in a revolutionary way
with the interpreter, I think.
Right.
You kind of got to live within the box with the walls that are put up by those constraints.
But I do think it's super important.
You know, a lot of people would maybe just think, well, let's just get rid of it.
Like, let's just try to move beyond it.
But when you hear people say that Python is slow or it has these other problems, so often
what you'll see is, well, and what I did was I did a for loop in Python and did some math
and that was slow.
It's like, well, but if a thing is slow, so often that gets rewritten in C and then all
of a sudden it's faster than, I don't know, what Java or whatever else it is they're trying,
you know, node, but they're trying to compare it to.
And so there's this kind of crazy switch that gets flipped for like really high performance
and then, you know, maybe acceptable most of the time performance.
And it's this interop with C that is the thing that's the escape hatch.
At LinkedIn, we don't, you know, we don't have the same kind of workloads that they have at
Meta, but, you know, we've done analysis.
One of the reasons why I was particularly interested in this project is because we write a lot of
our CLIs in Python.
And so, you know, we've had proponents of other languages say, you know, complain, hey, Python's
really slow.
So I write a CLI in Python and then it takes a long time to start up.
But if you actually do the analysis, what you find is that people are not writing a lot of
the internal libraries that those CLIs import do things like they go hit the network and they
go try to do the service and create really expensive resources at module scope time.
Right.
So like those are the types of analysis that you really need to do to say, is it really Python
or is it the way we use Python?
You know, for us, it's a little of both for sure, but often it's sort of the way Python is being used
in a non-idiomatic or not in the highest performance way.
Right.
It's a little bit of rewrite of some internal code can get you a long way.
It sure can.
Also, I want to just give a quick shout out back on episode 347.
I talked with Dino VLAN like all about the Cinder project.
So if you want to check that out, they can definitely go find that.
But there's a lot of cool things that were taken out of Cinder and are being proposed.
And I guess that sort of brings us to our main topic for today is imports and this PEP 690
that you all have proposed, which I am also excited for.
I think imports are one of those things that are a little bit mysterious to people because
they have conceptions coming from other languages, especially compiled languages that are very
different than what the reality is.
So I do want to talk a whole bunch about what you're offering here, what you're proposing
here in this lazy imports PEP.
But before we do, let's maybe just set the stage with what happens when I write import requests
or I write import FastAPI or even something built in like import collections.
Who wants to take this one?
Tell people what really happens when the import statement runs or is there.
There was a time where the import system for Python was written in C.
And it was even more mysterious, I think, at that point.
And I don't remember when it was, but Brett Cannon rewrote the import system into Python.
And then about that time, I sort of went through sort of line by line and tried to understand.
I mean, because right, like imports are probably one of the oldest features of Python, right?
They've been around so long and there's so many corner cases, right?
It's a very complex system and has lots of features, lots of hooks, lots of places where
you can hook in your own behavior and really understanding how all of that fits together.
I think I probably took a couple of months to sort of walk through the entire system line
by line and document.
Unfortunately, now, if you go to the language reference guide, I think it goes into all the
gory detail about how imports work.
Everything from namespace packages to, you know, concrete packages and all the file system
hooks and meta path hooks and, you know, blah, blah, blah, blah, blah, right?
Yeah, definitely a lot going on.
I think the key thing, though, that you are hinting at there is this is a runtime type of behavior,
right?
This is a runtime experience.
Carl, you're shaking your head.
What do you want to add to this?
What Barry was saying here?
Well, yeah, I mean, I think what a lot of people don't consciously realize initially
about imports is that it's just executing some more code, right?
So it's like, you can almost think of imports in Python as like syntax sugar for a function
call where the body of the function is the module level code of some module and the return
value is the module object that you end up getting back from the import.
But what it really is, is just causing some more code to be executed where the result is
a module object that has stuff on it that you can use.
So, and one of the consequences of that is that literally anything can happen when you
import a module.
I mean, like Barry was saying, you know, it could go off and talk to the internet.
I think people have even written stuff that like automatically goes and finds modules on
the internet and downloads them just in time to import them.
But like, but yeah, that's the key point is that you don't have install.
Let's fix that for you.
Yeah.
It could be arbitrarily slow.
It could fail.
It could do anything.
Yeah.
You know, if people have a conception of an import statement being like a using statement
in C# or an include statement in C++, that's not, that's kind of effectively the same intent,
but the behavior is massively different, right?
In C++, it says, here's some symbols that should also be used for compilation.
In Python, it's literally saying, let's just execute this file top to bottom.
And usually execution means define a function, define a function, define a local variable,
define a class.
But it could mean run some random thing, search the file system, all sorts of stuff.
And it leads to things like this, the if dunder name equal dunder main, right?
Yeah.
This is a big confusion that people run into.
Why is this weird thing in Python?
Why do they recommend it, right?
And it's just because like when you do an import statement, it just runs top to bottom.
And if you put behavior in there, well, that behavior happens, right?
So here's your way to sort of skip that.
Just as a sidebar, you all have been involved for a long time.
Like, should Python have had a deaf main sort of entry point thing like so many languages have?
I'm not saying it should or shouldn't.
I just want to hear your thoughts on this.
I think this is just one of those, you know, quirky things about the,
not even really even about the language.
It's just a quirky thing about the Python, you know, the way you use Python.
And to me, it's like one of those things, you learn it and then you just use it.
But actually, I think I tend to use this much, much less now,
now that we have things like packages and entry points and things like that.
To me, you know, I almost am defining a main function.
And then in my package metadata, I say, well, there's my entry point.
And the packaging system just does it magically.
So this is kind of a convenience.
And I don't think you really actually run into it as often nowadays as you probably did,
you know, years ago.
Sure.
All my apps that are supposed to be run directly like that, they all have a def main somewhere.
And then this just calls the main with this symbol here.
Right.
Yeah.
Herman, what do you think?
Yeah.
I totally agree.
And it's probably not something that we should be using as much right now.
Yeah.
Sure.
Of course, the really interesting thing here, right?
Is that it's Dunderne.
Right.
Like that's a deeper concept about Python that you should really understand is that modules
are objects and they have this attribute called Dunder name.
And that Dunder name, that attribute is set to some string.
And when you call a, when you not import a file, but call a file using the Python executable,
that file gets done, you know, this string Dunder main assigned to that attribute.
Right.
So.
Yeah.
Kind of like understanding a little deeper about how Python works is really important,
I think, to get, to be able to use it effectively.
It is.
And you hinted on being more idiomatic to get better performance.
And I totally agree with you.
All right.
So now knowing kind of what it means to do an import, let's talk about your pep.
Who wants to introduce the PEP here?
Herman, you're number one author here.
So how about you go for it?
Yeah, sure.
Well, we all started long before the pep.
In 2020 is when I joined Meta.
It was just one month before the pandemic hit.
And so I started working in Instagram code base at the time.
But there have been a lot of problems cause a problem is there with the reload speeds in the development server and Instagram.
And also another, a ton of other common like too, as well.
So what I did is I worked in Instagram code base for a couple of years, trying to fight and touch the startup performance problems that we were in the country all the time.
And when you get into code bases, the size of Instagram is, they had a whole bunch of problems that become apparent.
One of those is...
Yeah, most people don't work at the scale in terms of lines of code or number of servers or anything like that.
You're definitely pushing the outer envelope there.
Yeah, there are thousands and thousands of modules working here.
So one of the problems that start to become a parent is startup speed.
The other problem is that refactoring modules becomes really, really hard.
So if you try to modify something to get it working better, it suddenly starts getting, you suddenly start getting an import cycles.
And all that means when you just live around imports or try to split modules.
So it gets really tricky and hard.
It's complicated.
So the start speed, when you run a module that imports other modules, and these modules start to run, then those in turn import more than modules.
And this goes on and on.
So all transitive dependencies are involved.
Right, right.
If you just import one thing, it seems minor.
But then that could import two things.
And those two things could import four things.
And then it just, if you're talking thousands of modules, it can explode.
And PEP 8 says, import goes at the top.
First thing, right?
It's like, just to get started before you can even figure out what your code is going to do.
You wouldn't do all the imports, which is the transitive closure of every import, basically, right?
Exactly.
So it just ends up loading every single module.
Nobody is going to be used immediately or it's never going to be used at all.
So this was an ever ending battle with the Instagram server.
And as soon as I realized that I was doing repetitive work, that was very complex and produced a little, very fragile changes.
And that was really hard to maintain.
And at the end, it didn't really yield the improvements that we needed.
So I thought, unless I set sail to write less imports, that's when we started implementing that.
Well, the PEP is when we tried to show to the world what we did and the results that we were having.
When we get to the U.S. Python 2020 in Salt Lake City.
I was planning to discuss this at the Language Summit.
But at the end, Carl ended up giving a really good overview there, explaining what it was.
I don't know.
Carl, can you add?
Yeah, we were, I think, Herman proposed a talk for the Language Summit at this last PyCon 2022 in Salt Lake City for the Language Summit.
And there just wasn't space in the schedule.
So the Language Summit organizers had to pick some things to leave out.
And so they didn't squeeze in lazy imports.
But then at the Language Summit, a couple of different people brought up lazy imports that they had heard that we had done this and wanted to know more about it.
And there was a lightning talks slot at the end of the Language Summit.
So in the middle of the afternoon, sitting there at the Language Summit, I just put together about 10 slides on what lazy imports is and how it works.
And I gave a quick lightning talk.
And on the way back to my seat from giving the lightning talk, Guido leaned over and said, just write the PEP already.
So a little bit of positive encouragement and a nudge.
Yeah, that's encouragement.
So then Herman was at the sprints.
And so the two of us sat down together at the sprints on the first day and said, all right, let's write a PEP.
So we put together PEP 690.
And he said, hey, oh, Barry's walking by.
Grab him.
Well, yeah, this is something that's been on my radar at LinkedIn for quite a while, like I said, because we have tons of CLIs right from Python.
And I think the thing that, like, this is not the first attempt at lazy imports.
Like, you know, there's been lots of different approaches.
But I think what really kind of struck, certainly what struck me, and I suspect what struck Guido was the really clever implementation of this particular approach to it.
I don't know who came up with that, but whoever did it was like really a stroke of genius.
Because it really gives you the transparency that I think you need to make this a success.
This is a portion of Talk Python To Me is brought to you by Sentry.
How would you like to remove a little stress from your life?
Do you worry that users may be encountering errors, slowdowns, or crashes with your app right now?
Would you even know it until they sent you that support email?
How much better would it be to have the error or performance details immediately sent to you, including the call stack and values of local variables and the active user recorded in the report?
With Sentry, this is not only possible, it's simple.
In fact, we use Sentry on all the Talk Python web properties.
We've actually fixed a bug triggered by a user and had the upgrade ready to roll out as we got the support email.
That was a great email to write back.
Hey, we already saw your error and have already rolled out the fix.
Imagine their surprise.
Surprise and delight your users.
Create your Sentry account at talkpython.fm/sentry.
And if you sign up with the code Talk Python, all one word, it's good for two free months of Sentry's business plan, which will give you up to 20 times as many monthly events as well as other features.
Create better software, delight your users, and support the podcast.
Visit talkpython.fm/sentry and use the coupon code Talk Python.
Maybe Carl or Hermann can talk about that a little bit.
Yeah, well, let's first define just like what is the PEP proposing?
So we've described traditionally what it means when you say import thing.
It executes all the Python.
And if there's some inline behaviors, that runs those behaviors.
And that all happens at the top.
Like, how is this different?
What change is this proposing to the CPython runtime?
Yeah, I mean, the basic idea is just that when you hit an import statement, they say import foo.
Instead of immediately at that point going off and finding the foo module, the source code and executing the entire module top to bottom and doing all that work.
And then, of course, all the transitive imports from that and et cetera, et cetera.
That all we do when we hit import foo is we basically remember, all right, we have this name foo.
It refers to a module somewhere.
We're going to put off the work of figuring out what that module is and actually executing it.
And we're just going to remember that whenever foo is used, that means we need to go find out what it is and actually execute it.
So then that name foo will just kind of sit there in the global namespace of whichever module imported it.
And oftentimes foo won't even be used or referenced anywhere in the module body of the module that imported it.
So we can go through the whole way importing that module and foo just continues to sit there as kind of this deferred pending import.
And then later at runtime, maybe somebody calls a function from our module.
And within that function, we have like we have a call to foo.bar or something like that, a reference to some attribute of the module.
And at that moment, the first time we actually at runtime run into a reference to the name foo, that's the moment when we'll suddenly say, OK, hold everything.
We need to go off, figure out what foo is, import it.
And now we actually have the proper foo module.
And now we can go ahead and figure out what foo.bar is and move on.
So that's kind of the essence of PEP690 is to try to do that.
Defer imports until the moment when they're first referenced, but try to do it transparently.
So that is as much as possible, apart from side effects of the import itself, you really can't tell the difference in your code that the import was delayed.
Right. The runtime behavior might be a little bit different, ideally faster and using less memory, which would be great.
But other than that, your code shouldn't, but maybe italics shouldn't know the difference.
Right. If you're doing coding styles and good patterns, basically not creating side effects during imports, you shouldn't be able to tell the difference.
Right. And so if you've got, say, a function that uses JSON parsing, but that only periodically gets called and under some pass.
Right. Like you could avoid import JSON effectively unless you need that bit of functionality.
And for larger projects that can really cascade. Right.
Right. That's the whole idea. So if you think of like a command line program that maybe has like 10 subcommands. Right.
And each of those subcommands might do something fairly different and they might have very different dependencies.
So on any given time that you run that command line program, if you're only using one subcommand, you don't actually need any of the dependencies of the other nine subcommands.
So with lazy imports, you can avoid, you can basically pay for what you use every time you run a program and only pay for what you use.
Yeah. I really like the idea. And the transparent aspect of it is, and what Barry was fond of, is that we write code the way we traditionally wrote.
We write import requests or import JSON or whatever at the top. And it's up to a special dictionary that replaces a standard dictionary that holds the globals.
It says when you access one of these things, if it's not yet materialized and really imported, go do that and then hand it off.
Otherwise, just hand off the module. Right. Something like that.
Yeah. I think you put it really well.
Thanks.
It's actually a standard dictionary for the module, but it's just a specialized lookup function, essentially.
Got it.
Right. So when that deferred object gets installed into the module's dictionary, and this is, I think it's really key because at the point at which that the dictionary lookup function finds this deferred object, that's when the object is resolved.
And what that means is that at the Python level, and even if you're an extension writer accessing the module's dictionary, you never see those deferred objects.
So they're completely hidden both from the C extension author and Python developer.
It's just completely bad. And that's where the transparency really shines.
That's where, yeah, because if you're going to force a new programming model onto people, all of a sudden they're going to not like it.
And we saw how easy that was to go from two to three. Right. So let's not try that again.
Oh, it was so easy. I got this before we get further into this. It's worth pointing out that this is in draft status, right?
It was created out a month and a week ago. So it's not super old. It's in draft. If it appears, it'll be in 312, probably. What's the status of the PEP? It's basically proposed and under discussion?
Yeah. It's currently discussed. Yeah.
I think I went and looked recently and saw that in the, in discuss.python.org in the PEPs category. I think the PEP 690 thread has more posts than any other thread there.
So it's been.
Okay. That's exciting.
It's been thoroughly discussed. But at this point, I think to some extent, the discussion is on hold while we work on getting an implementation against the Python main branch.
Cause our implementation in Cinder, Cinder is unfortunately still based on Python 3.8. We're currently working on upgrading it to 3.10, but there's some changes in the underlying dictionary and other things between 3.8 and now 3.12 alpha.
So the implementation needs some, some reworking and we need to get that available for people to look at and play with to really move to the next stage in discussion.
Yeah. Just a side note. So with projects like Cinder and stuff, it's based on 3.8, which is great. There's a lot of good features.
I mean, basically three, six and beyond you get f-strings, you get async and away, you get a lot of amazing stuff there, but the shift from 10 to 11, there's a big performance boost.
It's sort of coming along there. Are you guys looking to sort of bring all of this stuff into 11 as sort of a stable point or what's the story there?
For Cinder in general?
Yeah.
Yeah. So I mean, we're currently upgrading to 3.10 just to go two versions at a time instead of kind of leap three versions in one bound.
And also because as you said, there's a lot of changes between 3.10 and 3.11. So we want to kind of isolate those and get 3.10 stable first and get it in production for Instagram and everything.
So that's our current target. And then once that's done, we'll probably look at 3.12 next.
Sure.
We do want to bring Cinder up to date. And also we're really looking at trying to upstream a lot of the things that are in Cinder so that more people can benefit from them.
And to reduce the amount of work we have to do to be constantly rebasing all of our stuff on newer versions of Python.
So our goal, hopefully, is that lazy imports.
Sure. Everything that you get accepted is just one fewer thing you have to maintain. You can kick it to Barry. Let Barry do it.
That's right.
Well, I mean, we hope to continue to help maintain these things upstream too.
Yes, of course. Yeah, of course. But if you can make it part of the broader community thing, it's no longer your team. It's the community's benefit.
Okay. Let's talk a little bit about some of the different forms of imports and how they affect this.
Because out in the audience, Hybrid Robotics asks, when you do from library import function, does this really save memory?
I think that maybe sets the stage for talking about when there's different styles of imports we can do.
Obviously, we can just do import library. We can do from library import function or import star, all these different things.
And there's rules in the PEP about how that controls the laziness or the alternative.
I guess the current default way is called eager loading or eager imports, right?
I think it's really good that Hybrid Robotics brought up this question of from library import function, because that's actually a key way that PEP 690 is different from the existing ways of doing lazy imports.
So there are existing things out there. There's a lazy import loader in the standard library.
And there's something called demand import on the package index that came out of Mercurial.
And all of these things take the approach of having a custom module object.
Right.
That when you do a get attribute on it, it has a dunder get adder or dunder get attribute implementation that waits for an attribute access and then goes out and does the import.
And that works for import foo, but it doesn't make from foo import bar lazy because from foo import bar gets the foo module and then you immediately access the bar attribute on it.
And so if you're using this lazy module object style, then effectively it's just eager because you get the attribute off it right away and that makes the import happen.
And the difference with PEP 690 is that with from foo import bar, we just stick a lazy object into the namespace of the importing module under the name bar instead of under the name foo.
But it's still a lazy object and the import still won't happen until something later actually uses the name bar.
So even in the case of from foo import bar, we're still able to make it lazy and it still will save memory at least until or unless you actually use the imported thing.
Yeah, that's a great summary.
I would like to point out maybe just not entirely sure where they're coming from here.
So today, if you write from library import function versus just import library or from library import star, you're not really saving any memory.
It basically is doing the same thing, right?
The module object is created.
It's imported.
All the stuff is done.
It's just what symbols are defined for you, right?
It's more of a syntax thing than it is a memory thing right now.
Yeah, that's correct.
Yeah.
But in the thing you're proposing, this will still be the lazy version, which is great.
Yes.
Yeah.
So there's different ways in which we do import.
Sometimes people even do imports inside of try blocks.
So like try to import this thing.
And if it's not there, maybe try to shim it in or report that this module is a dependency or other things.
And it's literally the import statement that is supposed to succeed or fail that communicates back whether or not that was okay.
Right.
And with this lazy version, you're going to change.
It's not going to fail within that try block because within the try block, it's going to create a deferred lazy thing.
And that'll always work, right?
It actually could also fail in certain circumstances.
But for the most part, it should just work.
If you have an import inside a try block or inside a class or inside a function, all those imports are eager imports.
Right.
Exactly.
So what I was getting at is that you were actually specifying that if there's an import within a try except or within a width block, you're actually not letting that be lazy.
You're making them be eager, right?
That's right.
Yeah, exactly.
Although imports inside this important thing are also lazy.
So if any of those fail and you are expecting those to fail, it could not throw an exception there in that reason, in that case.
Okay.
And then also from thing import star has to be eager.
Why does that one have to be eager?
Yeah, because we don't have the clean names that are being imported.
So there's no way to add these lazy objects to these names.
So we need to just import everything and see what are the names that are being imported.
That's the main reason for it.
Yeah.
Right.
Because when you say star, you don't know what to put into the module's symbol table, right?
So you've got to actually do the import to figure out the star.
That's pretty interesting.
All right.
So there's some interesting examples you have in the PEP that people can check out about.
Here's sort of a fake slow module that just does time.sleep, but it effectively shows that it could be slow.
And you can run it like do an import and it'll run instantly basically because it's not actually, if you're not accessing that module really at that point, it's pretty much instant.
I guess one thing to point out here, and this is, I don't know how I feel about this, but maybe we could talk about it a bit.
The proposal is that this is not the default behavior for Python indefinitely into the future, that you have to pass a dash L flag to the interpreter or set an environment variable or something along those lines.
You want to talk about why make it opt in instead of maybe opt out?
I wish it was a default.
That's what I wish to.
But the thing, the reality is that there are a lot of applications and modules that are using or relying on import side effects.
So we can just expect those to be lazy or compatible anytime soon.
So you're saying there's a lot of bad code out there.
Is that what you're saying?
No.
No, I'm just teasing.
But if it has side effects, right, if it's like, oh, because you imported this, we've initialized the database connection.
There's actually some really good discussion on the discourse thread.
I feel like this is actually an important aspect of the PEP because one of the things that was sort of pointed out in this discussion is that I both author libraries and I author applications, right?
As a library author, I can't really reason about whether I can sort of say, yeah, maybe my library is unsafe for lazy, but I can never really say, you know, assert that my module is safe for laziness and be certain that in all the way that all my downstream consumers are going to use my library, right?
That it's always going to be safe for lazy imports.
So by using it, I'm a little uncertain about the environment variable, but I feel pretty strongly like the flag, the dash, capital flag.
Yeah.
It's the responsibility of the person running it, sort of.
It's the leap.
Yeah.
It's the top of the tree.
Yeah.
To give an example of what Barry's talking about is like, I mean, for better or worse, some level of import side effects are just kind of built into Python.
Like if you have a module that subclasses a class from another module, you may think that you have a module with no import side effects, but actually importing your module adds something to the Dunder subclasses of the module you inherited from.
And so even code that apparently looks very clean and clear of import side effects does technically have some side effects on import.
And the real question is, is anybody actually looking at Dunder subclasses on the parent module?
And, you know, there's a lot of common library patterns like decorators that when you decorate a function, they register that function to that class and some registry and some other module.
So then suddenly again, you have an import side effect.
Like Flask or something.
Yeah.
Right.
Yep.
Exactly.
So you really have to, it really has to be the person writing the application and testing the application.
You says, I'm going to try lazy imports.
I'm going to see if things work.
I know how my dependencies are used.
If it works well and all my tests pass, then I can consider just turning this on at that level.
Sure.
Okay.
Interesting.
Taking Barry's idea that it's the application owner who should sort of make this decision.
What about having some code artifact that allows you to signal that as well?
Right.
So as I run it, I can control whether or not this happens.
But what if I just the very first line of my app.py or whatever it is, I just say import lazy or something like that.
And then every subsequent thing from there on behaves that way.
That way, if I distribute my app to someone else, I don't have to convince them or teach them about dash L.
I can just say, no, just double click this thing or run this.
And it'll just, it'll go in a consistent way.
As long as you don't have like PTH files that are getting in the way, you know, before your main gets called.
Go ahead, Carl.
Sorry.
Yeah, no, I was just going to say there was a lot of discussion about that in the thread.
And there actually is a draft of a heavily revised version of the PEP.
For Barry's review, actually.
So Barry, you can.
Barry, let me throw that up for you.
We can go through it.
Yeah.
One of the changes that is made in that draft is both removing the environment variable because people were concerned about the risk that if somebody just got the idea that, oh, lazy imports makes everything faster, they might try to just set the environment variable in their shell.
And then all of the Python programs they run will try to run with lazy imports and might break.
And then they're reporting bugs.
And that's not a great experience.
So we want it to be really limited to an application that's tested with lazy imports.
So the latest draft of the PEP removes the environment variable and actually adds exactly what you're talking about, a programmatic way to enable lazy imports for your application in code.
What is the code way?
Well, I mean, it's just a proposal in my draft PR to the PEP text right now.
So we'll see what Barry says about it.
Sure.
It could well change.
I just rubber stamped it.
So the reason I ask is there's sometimes certain things you've got to do to, like, tweak the path or something to get imports to work right if you're running from some weird location or something.
Think of, like, alembic migrations and stuff like that.
But in a lot of the editors and the linter's, they'll whine at you and say, no, no, no, these imports that go below that modification to the path statement, they go above it.
Like, no, the whole point of the thing above it is so the things below it don't fail.
Right.
Is there, like, whatever it is, I just want to put in a little hint.
Like, if you could make the editors not complain that the stuff below it should go above it because the stuff below it is controlled by the stuff above it.
You know what I mean?
Yeah, the current proposal, I don't think, I mean, I suppose if it becomes merged, the linters and such could special case it.
I think hopefully that's not too huge of an issue because the idea is, I mean, you potentially have a large application and this would occur, like, in one main module, right?
You would just put like that.
Call this function.
Right.
You put the hash, ignore, and then you're good to go.
Right.
Got it.
Okay, cool.
Let's go talk about some benefits here.
So, in terms of performance, so it says it's already been demonstrated for startup time improvements of 70% and memory reduction of real-world CLIs by 40%.
Those are not joking around.
This is not, like, playing at the edges of performance changes.
This is significant.
So, are those numbers from what you're doing with Instagram or other tools that you tested it on or something like that?
These are numbers from Instagram, Instagram server, and also I love command line tools that we use inside Meta.
So, yeah, these are real numbers.
It's about 70%.
Sometimes we've seen 50% improvement in startup speed.
Memory is also up to 40%.
We've seen 20%.
Right.
These are upper bounds, of course.
Sure.
But what I think is good about that is it's not, you know, sometimes people say, well, let's see how fast Python is.
I'm going to do a wild true loop and see how many times I can increment a number and compare that to doing that and see, like, okay, nobody does this and they don't really care about how fast.
Like, how do real apps behave?
How do real?
And so, this is you taking the stuff that you all work with day to day and trying to make it faster and getting significant benefits from it.
Not weird little edge case benchmarks.
Yeah, I think an important thing to note there that Hermann mentioned is that this isn't just one code base.
So, you know, we've seen this as repeatable scale of improvement across a variety of different tools.
In fact, recently, we've started to see we have a lot of data science people and researchers who use a lot of Jupyter notebooks.
And they've started to really quickly pick up Jupyter notebooks kernels based on Cinder with lazy imports enabled because they're seeing similar startup time and memory use numbers for their Jupyter notebooks.
So, like, across a fairly wide range of use cases and types of programs, we are seeing these kinds of numbers as consistently repeatable.
Yeah, that's interesting to the data science side.
A lot of those libraries are pretty large.
And so, I suspect that that's probably pretty valuable.
So, one of the things you all talk about is that this proposal also will eliminate, what would you call it, like, false import cycles?
I don't know.
Do you want to talk about this cycle benefit that we might get here?
Yeah.
Throw it out to all of you.
Whoever wants to grab it.
I didn't.
Can try to say something about the import cycles.
When you have imports at the top of the module, and you're not using these imports, and some of the imports that you are declaring there are importing something on the end to us.
At some point, it's importing the first module.
Then you have a cycle, right?
So, if one of these imports is not actually being used, and it's being deferred to a use case where it's only resolved inside a function,
further down, which is not being immediately called, then we won't have these cycles.
And then a lot of times when we have this kind of imports that are declared on the top of the modules, because
But it hates this that is recommended to have all imports in the top of the module.
And so people just start putting imports there because it's cleaner and looks better, but
it needs to stop provoking these, causing these import cycles.
Nice.
So hopefully this will solve some of those issues.
I know when I first got into Python, I didn't understand why there wasn't a better way out
of like sharing code in a bi-directional way.
It was really, it took a lot of thinking like, okay, how can I structure my code into different
files that don't feel like it's all just went in one file and yet allow me to reference it?
Like, for example, if I've got one class and it works with another and it has a function
that returns one type of it, but the other one might have a field, which is one of those.
And you want to do a type declaration of saying which one is which, or you need to create the
call initializer of one or something.
Those kinds of bi-directional relationships are hard to model.
And it sounds like this doesn't really address or fix or change those in any significant
way.
But the other ones just about the timing can go away, right?
The only import cycles that lazy imports don't fix are the ones where basically it would be
a cycle, even if you had it in the same module, like where you're literally using names in a
bi-directional way at module level.
And so there's no way you could order them even within one module and have it work.
And so essentially forward references, those kinds of cycles, the lazy imports doesn't fix.
But really any other kind of cycle or anywhere in the cycle, one of the uses is inside a function.
All of those are taken care of.
And so even for the type annotation ones, if you use from future import annotations, PEP 563,
or another proposal, PEP 649, anything that makes the type annotations also lazily evaluated,
so they aren't eagerly evaluated at import time, then all of a sudden your cycles went with that
plus lazy imports.
Then all the cycle problems with type annotations also just disappear.
I was just going to say, I think one of the interesting things that I think it was Carl
pointed out when we were talking about this is that if you've got the converse is right,
if you've got an application or something that's using these lazy imports, and then you turn
them off, you might be hit with a bunch of cycles that were sort of, you know, hidden from
you because of the lazy feature, right?
So you have to be a little bit careful about engaging with lazy imports and then turning them
off.
Right.
Getting everything eager.
So it's just one of those things to watch out for.
Oh, interesting.
Yeah.
If you're doing it by running it with the run flag, you could be effectively hiding a runtime
error that someone else would hit if they ran it without.
Yeah.
Right.
Kind of back to my like, could the code define this instead of a runtime flag?
So it's like absolutely consistent.
Yeah.
Let's talk about debuggability a little bit.
So normally when you have an exception, if I import thing and there's a problem importing that
thing, I'll get an exception on the line that the import did.
But with this lazy thing, it'll be the import error will occur where the first attempt to
touch it happens.
Right.
But you all do some work to figure out and sort of report back.
the error came from where the original import statement was, right?
Yes.
The error should be reported where the import is being declared so that it's easier to do
But this can also produce import error and send anywhere.
So the other thing that we are thinking about adding to this implementation is to have an error wrapper around the real error that is being thrown.
So we can easily know that it's just an import error coming from a lazy import.
Yeah.
It could theoretically, I guess, get caught in a situation where people would have not expected it.
Right.
It could have a overly aggressive try accept block.
There's like, well, there's only two types of exceptions that come out of here possibly ever.
And either of them, I'm going to handle it in this case.
Well, now here's a third.
Surprise.
Yeah.
Okay.
But at least from sort of figuring out where the error came from, I guess you guys, what do you call it?
Deferred exceptions is what you actually called that, right?
Right.
So there's a couple of code examples in here.
And I'm going to kind of flip through here and check it out.
You also have some code APIs.
I talked about my wish for some code thing to declare laziness, I guess, is the way to put it.
But different things happen.
Like if you do the import within a try block, that forces them to be eager.
But you also have defined this eager imports context manager, sort of.
That will force the imports to be eager, right?
And I say sort of because it's not actually the eager imports thing.
It's the width block.
Whatever you put in there, if you do an import in it, is what's going to trigger it, right?
Yeah.
It doesn't matter.
Yeah.
And then the third way is you can import set eager imports and actually pass like namespace names in there.
Because one of the problems is I'm going to import a thing that's a package whose code I don't control.
And I need it to, for one particular thing, eagerly import that, right?
So you kind of have to force it down the line where it's out of the chain of control there.
So you can sort of set like, I want to do FastAPI dot whatever, like exceptions or whatever's in there.
I want to make sure that that eager loads, even though I don't control FastAPI, right?
So you want to talk about some of these code tricks that you have here?
Yeah.
Or tools, I guess is a better word.
Yeah.
Let's say the eager imports is just for opting out certain modules.
Application owners should be the ones adding the incompatible libraries here if they find or figure out that they aren't playing well with lazy imports.
Yeah.
So if I write a library and I see that my sub module cannot deal with being lazy, I could put this in my code to avoid, no matter what people set for the runtime flag, I could avoid that problem for my library, right?
Yeah, exactly.
So you can always try first and if you're the application owner and you know exactly what you need to do this application for, to work, you can try it in modules.
And a lot of the libraries just work out of the box.
So you don't need to do anything.
There are some, some libraries that don't work.
And if you can make those work in some other way, you can then add these as eager imports.
What's the, I don't know if it's a lazy imports in those libraries.
Yeah.
These are really interesting to see here.
Like you can even pass a callback that will be handed a module name and it can, it can decide.
You could say, I don't want to just say all the names that are of the sub module, but just anything that matches this pattern.
Let's just tell it.
We want to eagerly import that.
Yeah.
It looks nice.
I like it.
There's a, another question from audience back to a code question and explicit opt-in is a hybrid robotics.
Also asked.
I really liked the idea of import lazy or lazy import library or something like that to be explicit about it.
And you all actually specifically addressed whether or not there should be some syntax code that makes this happen instead of changing the default one.
Right.
What's the thoughts there?
I think generally we're not opposed to the idea of syntax for lazy imports.
The kinds of like memory and startup time wins that we've seen really depend on very broad application of lazy imports with just very narrowly targeted opt-outs.
That's what you need in order to really get to a situation where you're actually paying for what you use in a robust way where you're not in a situation where you just accidentally add one import in one place.
And all of a sudden all your gains just disappear like that.
If it's really across the code base, then it's very robust startup time and memory wins where you add one import.
And while you might pay for a little more, but you're not going to suddenly start paying 100% of the cost again.
It feels like adding new syntax is a much bigger hurdle in terms of the steering council and having to change the Python grammar and all of that.
And when the syntax, the per import syntax isn't really even useful in gaining the wins that are the primary motivation for the PEP, then it doesn't feel like a good trade-off to add that to the PEP.
Sure.
Yeah.
I think I agree with that.
That seems pretty reasonable to me.
What about the reverse?
What about if you, instead of doing something like this, where you say, I'm going to force these to be, I'm going to run some code to make these modules eager.
What if as a library author, I could write eager import my thing or something along those lines.
Now, I really don't want to see more syntax in Python.
There was a huge battle over whether a colon goes by an equal sign.
So I'm not necessarily suggesting that we should do this.
I would say there's nothing necessarily wrong with that.
It's more just that adding new syntax is a higher bar and maybe we should get some experience.
When the same thing is easily possible with existing syntax, like a context manager, we may as well get some experience with the future in a less invasive way before we go adding syntax.
Right.
If you see so many people using it and benefiting from it, then you could consider it.
I mean, one example of that is like 3.4 to 3.5 when there was asyncio introduced and then there was async and await introduced on top of that.
Right.
Even things like property, right?
Like the property decorator was, you know, a feature that was available for the decorator syntax was added.
So it's a tried and true strategy for Python.
Let's get experience with the feature and then we can make adjustments and syntax to make it prettier, you know, down the road.
Yeah, that's a good path.
Awesome.
All right.
Well, really, really good work on this, you guys.
This is exciting.
I would like to see the status change from draft to something further.
I guess I have to go do something after this chat.
Perhaps so.
Perhaps.
Awesome.
But now people know about the conversation much more broadly and maybe we'll get some more, even more comments on the discourse thread there.
Let's maybe wrap that up because I think we're about out of time.
But before I let you all out of here, you've got the two questions to quickly answer since there's three of you.
Barry, if you're going to write some code, what editor are you using these days?
It's always going to be Emacs for me.
I know.
You know, it's funny because Brett works at Microsoft.
And so, you know, LinkedIn and Microsoft, we share.
He has something to do with some editor.
I think people, a few people may have heard of him or something.
Yeah.
And I keep telling Brett, I want to, I want to, you know, if I was starting today, I would probably, yes, Emacs is just kind of deeply in my bones.
Got it.
German, how about you?
I used to use Sublime Text, even though it didn't have AutoComplete.
So I added some AutoComplete from porting, Komodo IDE Editor, AutoComplete to Sublime Text.
But now that it's in meta, I'm using VS Code.
And also being when I'm in the terminal.
Right on.
Carl?
Yeah, I used to be an Emacs user.
I mentioned that's how Barry and I very first connected back at PyCon 2008.
But I have switched to VS Code.
There's a lot of integrations with VS Code in terms of meta infrastructure and maintaining all that stuff myself for Emacs just became more of a hassle than I wanted to pay.
So I've switched to two jobs.
One was enough.
Right.
All right.
And then notable PyPI package, something that you think is pretty cool you just want to give a shout out to, popular or not.
Go in reverse order.
Carl, how about you?
Well, I guess I'll give a shout out to something that I use in all of my projects, which is, I guess, depending on the project, either Pyre or mypy.
I'm a big fan of type checking my Python codes.
Yeah, fantastic.
German?
Yeah, well, Pyre is pretty good.
Oh, I don't know.
And microsort, which sorts your imports at the top of the file.
So it's really useful.
If there's an import syntax that makes all the other ones lazy, all these tools are going to have to learn about, well, this one doesn't go before the other.
Or you could give it something like A, A, A, A, A, A, A, lazy.
I guess.
All right, Barry, how about you?
Notable package?
For me, it's a PDM, which is a package manager and sort of a build backend.
I went through an exercise a couple of months ago where I just was finally wanting to get rid of all my setup.pies and setup.cfgs and just fully embrace pyproject.toml and see how far it could go.
And I was actually pretty happy with being able to get rid of both of those legacy packaging files.
So I tried a bunch of the different package managers and I really liked PDM.
So I kind of, you know, settled on that one for my personal stuff.
Yeah, fantastic.
All right.
Well, final call to action.
People want to maybe have their thoughts heard on this.
What do they do?
I guess the discourse thread.
Yeah.
Discussed up.
Python.org.
Look for the PEPS category and look for PEPS 690.
Yeah.
Cool.
You can love it or not love it or stuff like that.
There's sort of ways to give just a heart feedback as well.
So cool.
The other thing you can do is pressure the sponsor of the PEPS to stop being so lazy.
And that's something.
I would find it incorrect if you were not lazy on this particular one.
I mean, you got to just swim in that waterfall and see how it feels before you can really make a decision.
Right, right, right, right, right, right, right.
Awesome.
All right.
Well, gentlemen, thank you for being here.
It's been great having you all on the show.
Thanks.
You're welcome.
Thanks so much, Michael.
Thanks for inviting us.
Appreciate it.
Yeah, you bet.
Bye.
This has been another episode of Talk Python to Me.
Thank you to our sponsors.
Be sure to check out what they're offering.
It really helps support the show.
Take some stress out of your life.
Get notified immediately about errors and performance issues in your web or mobile applications with Sentry.
Just visit talkpython.fm/sentry and get started for free.
And be sure to use the promo code TALKPYTHON, all one word.
Want to level up your Python?
We have one of the largest catalogs of Python video courses over at Talk Python.
Our content ranges from true beginners to deeply advanced topics like memory and async.
And best of all, there's not a subscription in sight.
Check it out for yourself at training.talkpython.fm.
Be sure to subscribe to the show.
Open your favorite podcast app and search for Python.
We should be right at the top.
You can also find the iTunes feed at /itunes, the Google Play feed at /play, and the direct RSS feed at /rss on talkpython.fm.
We're live streaming most of our recordings these days.
If you want to be part of the show and have your comments featured on the air, be sure to subscribe to our YouTube channel at talkpython.fm/youtube.
This is your host, Michael Kennedy.
Thanks so much for listening.
I really appreciate it.
Now get out there and write some Python code.
I really appreciate it.