Do you love Python's async and await, but feel that you could use more flexibility or
higher order constructs, like running a group of tasks and child tasks as a single operation,
or streaming data between tasks, combining tasks with multi-processing or threads,
or even async file support? You should check out AnyIO. On this episode, we have Alex Granholm,
the creator of AnyIO, here to give us the whole story. This is Talk Python to Me,
episode 385, recorded September 29th, 2022.
Welcome to Talk Python to Me, a weekly podcast on Python. This is your host, Michael Kennedy.
Follow me on Twitter where I'm @mkennedy and keep up with the show and listen to past episodes
at talkpython.fm and follow the show on Twitter via at talkpython. We've started streaming most of our
episodes live on YouTube. Subscribe to our YouTube channel over at talkpython.fm/youtube to
get notified about upcoming shows and be part of that episode. This episode of Talk Python to Me is
brought to you by Compiler from Red Hat, an original podcast. Listen to an episode of their show as they
demystify the tech industry over at talkpython.fm/compiler. It's also brought to you by us over
at Talk Python training where we have over 240 hours of Python courses. Please visit talkpython.fm
and click on courses in the nav bar. Transcripts for this and all of our episodes are brought to you by
assembly. Do you need a great automatic speech to text API? Get human level accuracy in just a few
lines of code. Visit talkpython.fm/assembly AI. Alex, welcome to Talk Python to Me.
Thank you.
Yeah, it's fantastic to have you here. You have so many cool open source projects out there. We're here to
talk about any IO, but actually several of them I've covered on Python bytes on the other podcasts that
I run. And we've talked about SQL code gen and typeguard. And I didn't associate that with you
specifically and back over to any IO. So yeah, a lot of cool projects you got going on there.
Yeah, too many actually. I managed to hand over a couple of them to other people where
Seaboard 2 and Sphinx Autodoc type hints, because I'm really the stretched thing at the moment. I barely
have time for all of the projects that I'm maintaining.
I can imagine. That's, you know, how do you juggle all that? You know, it's, I'm sure you have a full
time job and you have all these different projects, right? How do you prioritize?
Yes. Yes. Basically, I get the equivalent of a writer's block from time to time. So when that
happens, I just either don't try to code up at all, or I just switch to another project.
Yeah, true. If you're talking about type hints versus async programming, like if you're stuck on one,
you probably are not stuck on the other, right?
Yeah.
Yeah. Interesting. Well, we're going to have a lot of fun
talking about all of them. I doubt there's going to be any writer's block or speaker's block here,
podcaster's block. It'll be good. We'll have a good time chatting about it and sharing with everyone.
Before we get to that, though, let's hear your story. How'd you get into programming in Python?
I got into programming at the age of eight. It was on an MSX compatible machine. I started with BASIC,
as so many others did. I did some simple text-based games at first, just playing around. At some point,
I got Commodore's Commodore's 128 and I did some simple graphical demos with it. Then I got an Amiga 500.
Oh yeah, the Amiga's were cool. They were special.
Mm-hmm. Yeah. I dabbled in almost BASIC, then other kinds of tools also. I don't really remember that much of it. Then at some point, I did something with Mac. That is macOS Classic. There was this tool called HyperCard. It's precursor for Flash, basically. So that's something I did some things with. Simple games and whatnot.
Yeah. Okay. Skipping forward a bit. I got into PC programming like C++, mostly C. Then I think it was in the latter half of 2005. No, actually it was much earlier in 1999. I started with Perl. Hated it. Then I think the next step was in 2005 when I got to learn PHP. Hated that too.
Kept searching.
Then finally, finally in 2007, I got to know Python. Then that was Lava's first site, really. At that point, it was, I think, Python 2.5. And of course, I stuck with it.
I did some Java professionally for a while, but I never really, really got to love it. It had this corporate industrial feeling to it.
Sorry, I'm not dressed up enough to program my Java today. Let me go get my tie. I'll be right back.
Yeah. Python was really cool. When I started learning it, my first practical application I made in what, 30 minutes after starting to learn it. It's really staggeringly easy to learn.
It is.
That's one thing I love about it.
It really is. It's one of the few languages you can be really successful with, with a partial understanding of what's going on. Right. You don't even have to know what a class is or what modules are. You can just write a few functions in a file and you're good to go.
It's almost like it's English.
Yeah. Very cool. Raul out in the audience says, third time's the charm. The third language, you found the one you like there. Excellent. And how about now? What are you doing these days?
I've been working for several years on a project, a very complicated project where, okay, this is always a hard part to describe it. It's a sort of working well as application. I'm part of a bigger team. I'm the lead backend developer.
It collects IoT data and visualizes it and it provides all sorts of peripheral services to it. This is the first time I really had to spread my wings with databases.
Oh yeah. Okay. What technologies are you using there?
On the backend, we use TimescaleDB, which is a PostgreSQL extension.
Okay.
This is for storing that time series data. Then on the backend, we use my framework called Asphalt.
I don't know if you've encountered that one. I think it's really cool, but it's not in widespread use. It's not a web framework per se. It's more like a generic framework where you can compose applications from a mix of free-made components and custom-made components.
What's it called?
Sorry?
What was it called?
Asphalt.
Asphalt. Like the road?
Yes.
I do not spell it.
There we go.
Yeah.
No. I did a search and I just found a snake and a python snake.
Yeah, that's it. That's a good thing.
That python snake on some asphalt road. Yeah. You got to be careful here. Okay. Is it a little bit like Flask or what makes it special?
Well, Flask is a web framework. This is a generic framework. You can build any kind of applications with it. It doesn't have to be involved with web.
I see any network thing, not necessarily HTTP, so it could be UDP or it could be just...
It doesn't even have to do any networking at all. You can build command line tools with it. You can just have this mix of components and the YAML configuration to give settings to them all. I think this really would require a whole different session.
It does sound like it would be a whole different session, but this is news to me and very interesting.
Yeah, I haven't really advertised much. I'm working on version 5 at the moment, which does incorporate any I/O support and it brings the tech up to date with the current standards.
Okay. Yeah, this looks very asynchronous based. It's an async I/O based micro framework for network oriented applications, it says.
Yeah.
And built upon UV loop, which is how all the good Python async things seem to be back these days.
Mm-hmm.
So it has a lot of sort of modern Python features. It's got async I/O, it's got UV loop, it's got type hints, those sorts of things. When you started in Python in 2007, none of those existed.
Yeah.
How do you see the recent changes to Python in the last five years or so?
I would say that Python has been developing at an incredible speed. I really love it. So many useful stuff coming out with every release.
I agree.
Yeah, basically from 3.5 to 3.8 or something. There were just so many amazing features that came out then. And now we're seeing these libraries built upon it, right?
Right.
All right. Well, let's transition over to our main topic that we're going to talk about, which is what I reached out to you for, not realizing the other two interesting projects that I already gave a shout out to are also yours. We'll get to those if we got time.
So with Python in 3.4, we had async I/O introduced the actual frameworks that supported that. And then when it really came into its own was Python 3.5 when the async and await keywords were added to the language.
Python 3.5. And Python came out of the box with some support for great async programming. But then there are these other libraries that developed on top of that to make certain use cases easier or add new capabilities. And any I/O falls into that realm, right?
Yeah. So before we talk about any I/O, we should talk about Trio.
Yep.
Have you heard about Trio?
Yes, I have heard about Trio. I even had Nathaniel on the show, but it's been a little while.
That was back in 2018, I talked to Nathaniel. So Nathaniel Smith. So there's probably quite a few changes since then, actually.
Yeah, let's talk about Trio.
Yeah, actually, the last version of Trio was released just yesterday. The thing about any I/O is that it's an effort to basically bring the Trio features to async I/O land.
So Trio is fundamentally incompatible with async I/O. There is a compatibility layer called Trio async I/O,
async I/O, but it's far from perfect. So what any I/O does really is allow developers to add these features from Trio to their async I/O applications and libraries, one by one, without making a commitment.
For example, at my work, I use any I/O for just a handful of tasks. I think we should talk about the features.
Mm-hmm.
Async I/O for just a few months, but the other thing about Trio is that they're not going to be able to add them to the same thing.
Async I/O does not. So it's kind of a meta-async framework.
I see. So it builds on top of these different frameworks, right?
Yeah.
Yeah. So it builds on top of these frameworks and their underlying primitives.
Yeah. So for people who are not familiar, Trio adds things like this concept of grouped tasks.
So normally in async I/O, you start one task, you start another. They're kind of unrelated, but
even if they're conceptually solving parts of the same problem. And then with Trio, you can do things
like create what's called a nursery and then you can have them all run or you could potentially cancel
unstarted tasks. And there's other coordination type of operations as well, right? That's the kind of stuff
that Trio adds.
Yeah. So the point of any I/O is, as I said, to bring these Trio features to async I/O.
Right. And because when you do Trio, it's an end-to-end async stack.
Yeah. Which means things have to be built for Trio, right? It's like if I have, let's say,
HTTPX. I don't know how easy it is to integrate those kinds of things that are expecting an async I/O
event loop over into Trio. Well, I already have my own event loop running. It's hard to coordinate the
tasks, right?
If you're talking about HTTPX, it had a Trio and async I/O backends. Now it defaults to the
any I/O backends. So it runs by default on both.
Okay.
About any I/O features, it provides Trio-like task groups on top of async I/O.
Uh-huh.
In here, we mentioned that Python 3.11 has its own concept of a task group, but the mechanics
are quite a bit different. That requires a bit of explaining.
Yeah. How does it work here?
The thing is that async I/O task group, so not this one, but the standard library task groups,
which are in Python 3.11, they basically just start normal async I/O tasks and you can cancel
use the task objects with using the task objects that come out of the create task method.
So what sets any I/O task groups apart from async I/O task groups is the way cancellation is done.
And since any I/O was designed based on Trio, when you do start soon, it doesn't return any task
objects that you can cancel. Instead, cancellation is done via so-called cancel scopes. So each task group
has its own cancel scope. If you cancel that, you basically cancel all the underlying tasks, but it goes even deeper.
Because this is a bit complicated, so bear with me. Cancellation is not done on a per-task basis,
but on a per-cancel scope basis. You can have cancel scopes nested so that if you start a task and it starts
a cancel scope, you can just cancel that scope and it cancels everything up to that point.
Okay. So like if I call a task, if I create a task and then somewhere inside for it to do its job,
it also creates a task. Those can be grouped into the same basic scope, right? So there's not these
like children tasks running around. Yeah. You don't even have to start another task. If you cancel a
cancel scope, then anything you basically wait on gets automatically canceled. Bam. This is called
level cancellation in contrast to the edge cancellation mechanism employed by any SEKIO.
In edge cancellation, you just cancel the task once and it gets canceled error raised in the task. So you can
ignore it, which is by the way, bad thing to do, but then the task won't be canceled again. Usually there
are exceptions to this, which are a topic of debate in the community, but the cancel scope basically,
they define boundaries for cancellation. So if you say you cancel a task group's canceled scope, only the tasks
started from that task group are canceled. So when those tasks return back, so all the tasks are done,
then the code just goes forward from this async context manager. Basically when all the tasks are done,
then however they end, unless some raised exceptions, that's a different situation. If they were either
canceled or successful, then the code just goes forward to the all tasks finished part.
This is really neat. The other thing that's standing out here as I think about these task groups. So for
those of you listening, you can just create an async with block to create the task group. And then
in there you can just say task group dot start soon and give it a bunch of async methods to start
running. One of the things that's cool about this is it automatically waits for them all to be finished
at the end of that context manager, the with block, right?
The standard library task groups work the same way actually.
Okay. And those are in 3.11?
3.11? Yes. Yep. We'll see how the mechanism will work. There's a new mechanism for cancellation
on cancellation of tasks. It's not really battle tested. It's something that was added fairly late
in the game to 3.11. So it's not yet clear if there are edge cases where it fails totally. This is also
a debated topic in the community. Sure. The other thing here that I wanted to ask you about is you don't
say like create task and you don't say start, you say start soon. Why do you say what's this like
uncertainty about? Tell us about that. Okay. So start soon. It's actually does the same thing as create
the task because creating task doesn't start running it right away. It starts only running it on the
perhaps over the next iteration of the event. Right. Or maybe not. Maybe the event loops all backed up.
Maybe it's the, you know, it takes a while, right? Yeah. So it's basically the same as a loop that
calls soon. So you schedule a callback. That's all that tasks are. They are callbacks with bells and
whistles. Start soon is modeled based on trial, but it's, I should mention that there's also a method called
start, which works a bit differently. This showcase is the start method. So this is very, very useful.
This feature is not present in the standard library task groups. So basically it's very useful starting
a background service that you need to know that the task has actually started before you move on.
So in the example that you have on the docs here is you create a task group. And the first thing is to
start a service that's listening on a port. The next thing is to talk to that service on the port.
Right. And if you just say, kick them both off, who knows if that thing is actually
going to be ready by the time you try to talk to it. Exactly. This is something I use in practice all
the time. And this is different than what you would get just with the asyncio create task or whatever, right?
Yeah. And even the new task group feature doesn't have this.
This portion of talk Python is sponsored by the compiler podcast from Red Hat. Just like you,
I'm a big fan of podcasts, and I'm happy to share a new one from a highly respected open source company,
compiler and original podcast from Red Hat. Do you want to stay on top of tech without dedicating tons
of time to it? Compiler presents perspectives, topics and insights from the tech industry,
free from jargon and judgment. They want to discover where technology is headed beyond the headlines and
create a place for new IT professionals to learn, grow and thrive. Compiler helps people break through
the barriers and challenges turning code into community at all levels of the enterprise.
One recent and interesting episode is there, the great stack debate. I love, love, love talking to
people about how they architect their code, the tradeoffs and conventions they chose, and the costs,
challenges and smiles that result. This great stack debate episode is like that. Check it out and see if
software is more like an onion or more like lasagna or maybe even more complicated than that. It's the first episode
in compiler series on software stacks. Learn more about compiler at talkpython.fm/compiler. The link is in your
podcast player show notes. And yes, you could just go search for compiler and subscribe to it, but follow that link and
click on your players icon to add it. That way they know you came from us. Our thanks to the compiler podcast for keeping
this podcast going strong.
So I guess you could in the standard library, you could start it and then you would have to just
wait for it to finish and then you would carry on. But it's like two steps, right?
The workaround would be to create a future, then pass that to the task and then wait on that future. So it's a
bit cumbersome. And then you have to remember to use a try, accept in case that that task happens to fail. Otherwise, you end up waiting on the future forever.
Mm hmm. Another, I really like this idea. Now, the other thing that I don't see in your examples here,
where I'm creating a task group and starting these tasks and waiting for them to finish is
management of the event loop. If I was doing that with my code, I'd probably have to create a group
or a loop and then like, you know, call some functions on it. And, and here you just use any IO. What, where's the event loop being managed?
I'm not sure. What do you mean? Well, like a lot of times when you're doing async stuff,
you have to go and actually create an async event loop and then use the loop directly.
And, you know, you're working with a loop for various things.
And well, there is that a run command at the bottom. Yeah. Yeah. Okay. So basically the,
you just say any IO dot run or asyncio dot run. Okay. And I can, if, even though I'm using the
any IO task groups, I can still just, I can mix and match this with like more standard asyncio event
loops. That's the premise. So you can just, like, ease into it. Nice.
So for example, if I have a FastAPI web app and you know, FastAPI is in charge of manning the event,
managing the event loop. And if I've got like an async API endpoint, I could still go and use an
any IO task group and get all the benefits in there. Yep. Okay. That's beautiful.
I should mention that FastAPI also depends on any IO. Oh really? Okay. Yes.
How interesting. Yeah. I've seen Sebastian Ramirez talking about some, some little functions that he
wrote and he's like, I would love to see these just get back into any IO. I didn't realize that
FastAPI itself was using them. Yeah. Okay. So very useful. We've got these task groups. We've got the
concept of cancellation. Another one that's not exactly cancellation, but as, as sort of cancellation is
timeouts. Do you want to talk about how you do timeouts? Yeah. I meant to talk about that. As I recall,
in Python 3.11, there is a, it's a similar construct. I think it was a with timeout or some asyncio.timeout or
something similar. I don't remember really, but what this move on after does is it creates a cancel scope
with a timeout. Basically, this is a very, very practical use of cancel scopes. What it does is it
start a timer and after one second, it cancels this scope. So anything under that gets canceled.
So in this case, just at the sleep command gets canceled and then the task just keeps going.
So the way you described it before, it sounds like if there was a bunch of tasks running,
if any of them try to await something, they're also going to get canceled. Is that right?
In this case, you mean? Yeah. No, only the part that is within the with block.
Right. Well, that's what I mean. But if you had done multiple tasks within like the move on after,
right? Like I say, I try to talk to the database to insert a record and I try to call an API and
the database times out. Within a single move on after block, you can only have one thing going on,
which is the await here. So even if you start multiple tasks from that task group, they are not
enclosed within that cancel scope. I realized that castle scopes are complex and difficult concept.
And I don't think I can adequately explain them, but I hope that this will at least
give some shit some light into that. Yeah. It's a really cool idea because your code could,
it's async. So it's not as bad as if you were to like lock up waiting for an API call or something
that's going to time out on the network eventually after a really long time. But it's still a
cancellation. Still, you don't want it to clog up your code, right? You want to just say,
sorry, this isn't working. Yeah. One place where I often use a construct like this is finalization.
So when you are closing up things, then you can use this, this move on after to the timeout for closing
resources. Yeah, that makes sense. Because you want to be a good citizen and in terms of your app and
release the resources as soon as possible, like a database connection or a file handle. But if it's,
if it's not working, you know, like, oh, I made it. I made a try at it after a second. We're done.
Exactly. And also, I should mention that this is where any of your biggest caveat lies. It is in
in finalization. I often run into problems with the cancel scopes because the thing with cancel scopes is that
that when you run code within a cancel scope and that scope gets canceled, then anything awaiting on
anything within that cancel scope is always canceled time after time. So you cannot wait on anything as
long as you are within the cancel scope. And asyncIO code is not expecting that. So it might have a final
clause where it does await, say, connection.close. But that also gets canceled if you are within an
NEIO cancel scope. And it's one of the biggest practical issues with NEIO right now. And we are
trying to figure out the solution for that. Just something to keep in mind when you are writing
NEIO stuff. Yeah, that is tricky, right? But help us on the way.
You say, well, I'm going to try to call this web service and I'm going to wait it. And if it fails,
you know, probably internally, what you want to do is close the network connection as well.
Right. But if you try to wait closing the network connection. Yeah. So what happens there? Does it
eventually just get cleaned up by the garbage collector or dereferenced?
Well, garbage collector doesn't work that well with async stuff because the destructors could be called
in any thread. So you can't rely on that. You can't do any async callbacks in the destructor. So it's a
better, it's a good idea not to try any of that and instead just raise a resource warning. If you're
you're writing any I/O where code, you would have either this shielded cancel scope or better yet,
a move on after which shield true. What does that do? At least temporarily protects the
the kind of test from cancellation. So let's say you have move on after say five and with shield true.
It means that even if the outer cancel scope is canceled, your actual task will start running until
it exists as cancel scope or if the timeout expires. So you have a five second window to close any
resources that need closing. Got it. And so you just create a, you could do that say within your exception
handler or something, right? Yeah. Okay. Or actually, I think finally,
a finally block might be the best place to do that. Sure.
But depending on the audio use case, of course. Yeah, of course. Okay. Very interesting. So all
this stuff about task groups and scheduling tasks and canceling them, that's very trio-esque, but it's
also just a small part of any I/O. There's a bunch of other features and capabilities here that are
probably worth going into. Some cool stuff about taking async I/O code and converting it to threads or
converting threads to async I/O and similarly for sub processes. But let's maybe just
talk real quick about the synchronization primitives. These are things like events, semaphores. Maybe
not everyone knows what events and semaphores are in this context. Give us a quick rundown of that.
Yeah. Well, these are pretty much the same as they are on async I/O. Many of them use just the async I/O
counterfeits straight up. So events are a mechanism for telling another task that something happened,
something significant, a significant happened, and they need to react to it. It's often used to
coordinate the tasks. So one thing doesn't happen before something else has happened in another task.
Yeah, there might be two tasks running and one says, I'm going to wait until this file appears.
And the other one's going to eventually create the file, right? But you don't know the order.
Yeah. So one option is to just do polling. Like, well, I'm going to async I/O.sleep for
a little while and then see if the file's there. Try to access it, you know, and do that over and over. A much more
responsive way and deterministic way would be to say, I'm going to wait on an event to be set. And the thing
that creates the file will create the file and then set the event, which will kind of release that other task to carry on. Right?
Right. Moving on, semaphores are a mechanism for saying that you have this limited, you have a number of limit,
let's say a connection pool or something. And you want to specify that whenever some part of the code needs
to access to this resource, it needs to acquire the semaphore. So you set a limit, and then each time
a task enter, that's a semaphore, that decrements the counter. And when you hit the limit, then it
starts blocking until something else releases it. Right. So people might be familiar with thread locks
or async I/O's equivalent, where you say only one thing can access this at a time. So you don't end up
with deadlocks or race conditions and so on. But semaphores are kind of like that, but they allow
multiple things to happen. Say maybe your database only allows 10 connections, or you don't want to have
more than 10 connections. So you could have a semaphore that says it has a limit of 10 and you have to
acquire it to talk to the database. That doesn't mean it stops multiple things from happening at once. It
just doesn't let it become a thousand at once. Right? Exactly. I really like this idea. And I was
showing some people some web scraping work with async I/O where it's like, oh, let's go create a whole bunch of
ACPX requests or whatever type of requests, you know, something asynchronous talking to some
servers to download some code. And if it's a limited set, you know, no big deal. But if you have
thousands of URLs to go hit, well, then how do you manage not killing your network or overloading that?
And the semaphore actually would be perfect. So for people listening, the way that you do it is you
create a task group and then just you pass the semaphore to start soon. That's really clean. And then
any I/O takes care of just making sure it gets access and then runs and then gives it back. How's that work?
I'm not sure what sort of answer you are expecting, but as I recall, this current implementation is
actually using the underlying async libraries events. Okay. So there are actually methods to
acquire and release the semaphores. It just implements an async context manager that acquires it at the
beginning and releases it at the end. There's an event involved for notifying the any awaiting task that
it has a semaphore slot available. It's super clean. And the fact that you don't have to write that code,
you just say the semaphore is associated with this task through your task group. I really like it.
Actually, semaphores are not associated with a particular task. That's what capacity limiters are for.
Okay. So you can release a setup from another task while capacity limiters are bound to the specific
task that you acquired them in. Okay. Yeah. And the semaphore example, was it being passed? Oh, yeah.
It's just being passed as an argument, isn't it, to the task. And it's up to the task to use it. I see. Okay.
So this other concept, the capacity limiter. Yeah. It sort of does that. Yeah. So this is from Trio. It's very similar to the semaphore. So you can set, you can actually set the borrower. But in most circumstances, you want the current task to be the borrower. And limiters are actually used in other parts of any IO as they are in Trio. For example, to limit the number of threads that you allocate,
or limit the number of sub processes that you spawn. Sure. You don't have too many sub processes, right? You've got a thousand jobs and you just for each thing a job, start it in a sub process. You're going to have a bad time. Right. So as documentation says, they are quite like semaphores, but they have additional safeguards. Such as? Well, they check that the borrower is the same. Okay. Yeah, that makes sense. So by default, they check that the task they are used for both acquiring or releasing on the
Yeah, nice. Okay. Well, this I didn't know about capacitor, capacity limiters. That's fantastic. I love the idea. Okay. Let's jump over to you talked about the threads, and the sub processes. Let's talk about this thread capability that you have here. This is very nice. Any IO dot two thread. What is this?
Yeah, so this is also modeled based on trio. It's basically any IO's way of doing worker threads. So in asyncio, you have these thread pool executors that do the same as run sync. Async IO's API is somewhat problematic because you have basically two methods.
So you have, I forget the older one, the newer one is called two thread. The first one was what it run in executor, whatever, whatever it was. They both have their own issues. Two thread doesn't allow you to specify any thread pool. So it always uses the default thread pool. And there is no way to add that to the API because it was done in such a manner.
Then the older function does have this parameter at the front. But the problem is that it doesn't propagate context variables, unlike the newer two thread function. So context variables, if you don't know about them, they are a fairly recent addition to Python.
Yeah, what are those?
They are basically thread locals. Are you familiar with thread locals?
Make a comment for people who don't know, like thread local variables, which also exist in Python, allow you to say, I'm going to have a...
variable, maybe it's even a global variable, and it's thread local, which means every thread that sees it gets its own copy of the variable and where it points to and what value it is.
And so that way you can initialize something at the start of a thread. If you have multiple threads, they can all kind of have their own copy so they don't have to share it.
But that falls down because asyncio event loops, when you await those, all that stuff is running on one thread, just the one that's running the loop. That's what you're talking about is that equivalent, but for asyncio, right?
Yeah, so context variables are much more advanced concept. They basically, yeah, as you said, thread locals for async tasks.
That sounds very tricky. I've thought about that. I have no idea how to implement that. So that's pretty cool. I guess Python would know.
Yeah, the thing is, when it starts running a task, when it switches between tasks, it runs that callback within that context that the task is tied to.
Right. So this has been somewhat of a problem because that older method in asyncio for running worker threads, it doesn't propagate these variables, but the newer function to thread does.
But then you can specify which thread pool you want to use.
Right. Okay, so it's similar to the built-in one, but it gives you more capability to determine where you might run it.
Yeah. When you call run sync, it allows you to specify a limiter, and it uses the default limiter, which has a capacity of 40 threads.
40? That seems like a pretty good default. Much more than that, and you end up with memory and context switching issues.
Yeah. It was arbitrarily set to 40, but then Nathaniel and Trio, so I just followed suit.
Sure. Yeah, so basically, if you've got some function that is not async, but you want to be able to await it so that basically run it on a background thread, here you just say any IO dot to thread, and then dot run sync, and you give it the function to call.
And now you can await it, and it runs on a background thread in this thread pool, which is really nice.
Yeah, that's how it works.
So one thing that I'm noticing a lot in the API here for any IO is often, now for the synchronous functions, it's completely obvious why you wouldn't do it, but even in the sort of creating tasks ones, what I'm noticing is that even the async functions, you pass the function name and then the arguments to like start soon, as opposed to saying, call the function, passing the arguments, and getting a coroutine back.
Why does it work that way?
It seems like it would make it a little less easy to use, like, say, type hands and autocomplete and various niceties of calling functions and editors.
Yeah, there was a good reason for that.
I can't remember that offhand, but at the very least, it's consistent with the synchronous counterforce, like run sync.
Yeah, cool.
All right, so we have this stuff about threads, and you have the to thread, also from thread, which is nice.
What's from thread to?
Yeah, so when you are in a worker thread, and you occasionally need to call something in the event loop thread, then you need to use this from thread to run stuff on event loop thread.
Right, because the worker method is not async.
Otherwise, you would just await it, right?
It's a regular function.
But if in that regular function, you want to be able to await a thing, you can kind of reverse back.
I see.
That's an interesting bi-directional aspect.
All right, subprocesses.
We all know about the GIL, how Python doesn't necessarily love doing computational work across processes.
Tell us about the subprocess equivalent.
This is a relatively easy way to both run tasks in a subprocess and then opening arbitrary executables for running asynchronously.
AsyncIO has similar facilities for running async processes, but these async subprocess facilities are not really up to par with, say, multiprocessing, which has some additional nice things like async shared queues and other synchronization primitives.
But they are still pretty useful as they are.
Yeah.
So basically, you can just say anyIO.toProcess run sync and you give it a function and now you can await that subprocess multiprocessing.
Yeah.
There are the usual caveats like because they don't share memory, then you have to serialize the arguments and that could be a problem in some cases.
Sure.
So basically, it pickles the arguments and the return values.
It sends them over.
Yeah.
It could be even so that the arguments are pickable, but the return value is not, which obviously causes some confusion.
Yeah.
Absolutely.
People maybe hear that pickling is bad and you can have all sorts of challenges like code injection and whatnot from pickling through security.
So this, I would think, is not really subject to that because it's you calling the function directly.
Right.
It's like completely just inside.
There's no way to inject anything bad.
Yeah, exactly.
It's all the multiprocessing that's handling it anyway.
So it should be okay.
Yeah.
Cool.
Another one that is pretty exciting has to do with file support.
So we have open in Python.
And you would say with open something as F, right?
But that's, there's no async equivalent, right?
Yeah.
These file facilities are really just a convenience that wraps pictures around these file objects.
Yeah.
So if somebody wants to know, there is no actual async file IO happening because that's not really a thing on Linux.
And even on Windows, it has terrible problems.
Okay.
So what's happening with this any IO dot open file?
It opens a file in a thread and then this opens, it starts an async context manager that on exit closes the file.
And it also closes the thread, I guess, right?
Well, it uses throwaway threads.
Okay.
Basically from the thread pool.
All right.
I see.
So it'll use a thread to open the file and get the file handle and then throw away the thread.
And then does it do something similar?
Well, return the thread to the pool.
Yeah.
Return to the pool, which is much better than creating it and throwing it away completely.
Also that read call is done in the shed.
Okay.
So it just is sort of a fancy layer over top of thread pool.
But it's really, really nice.
You write basically exactly the same code that you would write with regular open and a context manager, but the async version.
Although I got to say that the opening part of creating a context manager here, you say async with, which people are probably used to.
And then async with await open file, which is, it's a bit of a mouthful.
The reason for that is because you can just do await open file and then go about your business and then manually close the file.
Got it.
I'm not sure if there's a more convenient way to do this.
I might be open to adding that to any IO, but for the moment, this is how you do it.
Yeah, you could probably wrap it in some kind of class or something that's synchronous, but then has an A enter.
But yeah, I'm not sure if it's totally worth it.
But yeah, that's quite the statement there.
And the other area that's interesting about this is if you want to loop over it line for line, instead of doing a regular for loop, you can do an async for line in file and then read it asynchronously line by line, right?
Yeah.
So the A next method just gets the next line in the worker side.
Yeah, this is fantastic.
So people are working with files.
They definitely can check this out.
And also related to that is you have an asynchronous pathlib path.
Yeah, that's a fairly recent addition.
Yeah, it looks great.
So you have like an async iterator.
You have an async, is it a file, async read text, all built into the path, which, you know, is like what's built into a regular path, but not asynchronous.
Yeah, quite nice.
We're getting kind of short on time here.
What else would you like to highlight here?
That's really important.
I would like to highlight the streaming framework here because it's one of the unique things in any IO.
Okay.
Yeah, let's talk about it.
Trio has its channels and then it has Socas and whatnot.
But any IO has something that Trio doesn't have.
And really, asyncio has some kind of streaming abstraction, but not quite on this level.
In any IO, we have a streaming abstract base class hierarchy.
We have object streams and byte streams.
The difference between these are that object streams can have anything like the integers, strings, any arbitrary objects.
And byte streams have only bytes and they are modeled according to TCP.
With byte streams, you can send a number of bytes or receive bytes, but they might be chugged differently.
These are abstract streams.
So you can, say, build a library that wraps another stream.
Say you build an SSH client that creates a tunnel and it exposes that as a stream.
So long as you are able to consume a stream, you don't have to care how the stream works internally or what other streams it wraps.
A good example of a stream wrapper is the TLS support.
That's right there.
So TLS support in any IO can wrap any existing stream that gives you bytes, even if it's an object, whether it's an object on or byte stream.
So it does the handshake using the standard library.
Actually, standard library contains this sense IO protocol for TLS.
Okay.
Sense IO protocol, if you are not aware of it, it's basically a state machine without any actual IO calls.
It's a very neat protocol implementation that lets you add whatever kind of IO layer you want on top of that.
So this is what any IO uses to implement TLS.
So you have both a listener and a connect and a TLS wrapper.
So any kind of TLS, you can even do TLS on top of TLS if you like.
I'm not sure that's useful, but you can do it.
It's super encrypted.
Yeah, this is very flexible.
We have all sorts of streams that even these unreliable streams, which are modeled based on UDP.
Oh, really?
Okay.
So UDP is implemented by using these unreliable streams.
I don't think there are any more unreliable streams implementations than UDP.
But yeah, I'm really part of this streaming class hierarchy.
And it's too bad that there are no cool projects to show off this system.
But maybe in the future we will have.
Another thing that I would like to highlight is a system of type attributes.
That has been really useful in practice.
Yeah, before we move on to the type attributes, let me just ask you real quickly.
Can I use these streams and these bidirectional streams where you create like a send stream and receive stream?
Can we use those across like multiprocessing or to coordinate threads?
I'm sure we could use it for threads, right?
Sadly, no.
This memory object stream is, as they call it in TRIO channel.
This is one of the most useful pieces of NEI, really.
I use this every day at work.
So these are basically like cues on steroids.
Yeah, exactly.
That's what I was thinking.
Unlike ASYNCIO cues, you can actually close these.
You can clone them.
So you can have multiple tasks waiting to receive something like workers.
And we have multiple senders.
I see.
Like a producer consumer where some things are put in, but there's like five workers who might grab a job and work on it.
You can have five consumers and five producers all talking to each other.
Yeah.
And then you can just iterate over them.
Also not possible with cues.
You can close the cues, sorry, streams, so that when you iterate on them, if all the other other ints are closed, then the iterator adjusts the ints.
Oh, wow.
Okay.
So if the send stream goes away, then the receive stream is done.
It's at the end.
Yeah.
That's fantastic, actually.
I really like that.
Yeah.
I think this is also coming to...
Go ahead.
Sorry.
...asyncIO.
Okay.
I think this is also coming to the standard library.
Nice.
All right.
At some point.
Last thing we probably have time for here that you wanted to highlight is typed attributes.
What's the story of typed attributes?
If you knew about the asyncIO extras in...
I think they have both in protocols and streams.
For example, with TLS, you want to get the certificate from the stream.
Like if you have negotiated a TLS stream, you want to get the client certificate, right?
So this is a type of way to do it.
You can add any arbitrary extra attributes to a stream.
Just declare it in the extra attributes method.
But the niftiest part here is that it can work across wrapped streams.
A very good example is that say you have a stream that is based on HTTP.
You have an HTTP server and you have access to a stream, let's say, WebSockets.
Then you want to get the client IP address.
Well, usually you may have a front-end web server like Nginx at the front.
Normally, what you would get when you ask for an IP address, you actually get the IP address of the server.
What you need to do is look at the headers.
This is something you can do transparently with these type attributes.
So basically, wrap a stream that understands HTTP, you can have that handle the request for the IP address, the remote IP address,
and have it look at the headers and look for a forwarded header and return that instead.
Nice.
Let me see if I got this right here.
So people are probably familiar with Pydantic.
And Pydantic allows you to create a class and it says what types are in the class and the names and so on.
And those serialize out a JSON message.
It sounds to me like what this is built for is when I'm talking binary messages over a stream like a TCP stream, I can create a similar class that says, well, I expect a message that is a string and then a float.
Read that out of the stream.
Is that right?
Yeah.
A good example here is if you go back to the streams part.
Text streams.
Okay.
So this is something that translates between bytes and streams on the fly.
So this is a perfect trivial example of a stream prepper.
Okay.
Yeah.
So you have a text received stream that will do the byte decoding.
Yeah.
Very nice.
And you don't have to care like what's downstream of that.
And even if you have three layers on top, you can just still ask for, say, client remote IP address.
If there's a network stream somewhere downstream, that's the stream that will give you your answer.
Cool.
Yeah.
The stream work here is really nice.
There's a lot of things that are nice.
The coordinating, the task groups, the coordinating limitations, like with a limiter, capacity limiter.
A lot of cool building blocks on top of it.
And also the fact that this runs against or integrates with regular asyncio means you don't have to completely change your whole system in order to use it.
Right?
Yeah.
Very cool.
All right.
Well, I think we're about out of time to talk about any IO.
Do you want to take 30 seconds and just give the elevator pitch for SQL A code gen?
This is a really exciting project that you created here.
Yeah.
This is one of those side projects that are on the verge of a major release.
So what this does is it takes an existing database.
It connects to an existing database, reflects the schema from that, and then writes modal code for you.
The next major version even supports data classes and other kinds of formats.
The SQL model one is the one that's most exciting for me because that'll give you Pydantic models that use a SQL model, which is very exciting.
Yeah.
So nice.
What else?
Think if you're a consultant or you talked about Java earlier, right?
Imagine you've got like a Java code base and you want to do a proof of concept in Python and SQL model or SQLAlchemy.
And somebody says, well, why don't you try building a simple version that talks to our database?
And if that thing has like a hundred tables and complicated relationships, it's no fun to sit down.
Like a big portion of that project might be just modeling the database.
And here I can just say SQL A code gen, connect to Postgres.
Boom.
Out comes SQLAlchemy classes.
That's a huge jumpstart for getting started.
Or if you're a consultant jumping into a new project.
Yeah, exactly.
If you have a really large database in this, we will save little hours, you'll feel time.
At least hours, yes.
And a lot of frustration, right?
Because with like SQLAlchemy, you've got to have the model match the database just right.
And this will do that for you.
Yeah.
Okay.
Super cool project.
Typeguard is another one you have.
Not super complicated, but an interesting capability to grab on.
Yeah.
Also one of those that are on major of a version of a major release.
I sadly have not had enough time to finish the next major version.
And then there's of course, Python 3.11, which brings a whole bunch of new features that I have not been able to yet incorporate into Typeguard.
And sadly, I also have not started using it myself.
It's a sad story really.
Yeah.
Okay.
But the premise is that you have this pytest plugin.
You activate it during the test run.
And then in addition to static type checking your application, which you usually do with mypy, PyRite or what have you.
You can also do a runtime type checking because the static tools don't always see the correct types.
You might not be in control of that, right?
You might write a library.
Your library might have types declared on what it's supposed to take.
The person consuming your library has no requirement to run mypy.
And they have no requirement to make sure what they typed matches what you said you expect.
And because they're hints, they're not compiled options in Python.
It might at runtime, you might get something you don't expect, even though you put a type there.
Yeah, exactly.
So this is how you get the runtime assurance that you have the right type there.
Nice.
So all you do with this type guard library is you put an at type checked decorator on a function?
The best way would be to use the import hook.
Okay.
So there's an import hook that will automatically add these decorators while doing the import.
So you don't have to alter your code at all.
There are some open issues with that import hook.
Like somebody reported that this import hook is installed too late.
The modules in question were already imported.
So that's something I have yet to fix or find a workaround for.
Sure.
That's the idea.
Right.
So you can either use the decorator and be somewhat guarded about how you're doing it and only apply
to certain parts, like say your public API.
Or you could just say install import hook and then everything that gets imported gets wrapped
in the type checked decorator.
And what that does is it looks at the type hints and the declared return value and will raise
an exception if say you say that your function takes an integer and it's passed a string.
That becomes a runtime error.
Right.
Or you can just issue a warning.
Sure.
The warning may be nice, but it's still, I think it's pretty cool.
You can opt into having Python type hints become enforced basically.
Yeah.
What I use this for is in the asphalt.
When I accept the configuration for a component, I use this decorator or rather an assert to
check that the types are correct.
So I don't, I don't raise any mysterious warning type errors or value errors further down the
line.
Or even worse at runtime.
Sure.
Okay.
That's, that's cool.
There is one of the features of asphalt or highlights is runtime type checking for development
and testing to fail early when functions are called with incompatible arguments and can
be disabled for zero overhead.
So it sounds like, you know, you're maybe doing an import hook in development mode.
That's handling all this for you.
Is that right?
Well, actually in this current version, I'm using the assert.
So in case you didn't know when you have asserts, they are normally run without any switches
to Python.
But if you run Python without the debug mode, then asserts are not compiled in the bytecode.
So just by using this switch, you can disable these potentially expensive asserts.
Yeah.
Okay.
I didn't know that.
I'm familiar with that from C and C# and other compiled languages with their pragmas
and that type of thing.
But I didn't realize that about Python asserts.
Yeah.
There's this one thing that actually, if you have this code, if under debug, and there's a
bus of code under that block, that whole block gets omitted from the compiled code.
If you run Python with the debug mode disabled.
Okay.
Yeah.
Very cool.
All right, Alex.
Well, those are some cool additional projects.
You know, I feel like the SQL A code gen, we almost could spend a whole bunch of other
time on it.
Another one is the AP scheduler.
Again, could almost be its own show, but we're out of time for this one.
So thanks so much for being here.
Now, before you get out of here, I've got the two final questions to ask you.
If you're going to write some Python code, what editor do you use?
I've been looking at the different editors available.
And so far, PyCharm wins, hands down.
Right on.
I'm with you there.
So it has so many of these intelligent features and what have you that, for example, I use its
database features to browse through my database.
I use its refactoring features to change my code relatively safely.
And its Docker support gives me auto-completion.
The list goes on and on and on.
And most of these IDs are not nearly as sophisticated.
I agree.
Excellent one.
Now, notable PyPI package.
I mean, we talked about a bunch.
You can recommend any of these we talked about.
You can say something else you found interesting.
Well, I think I already mentioned Trio, but this is a difficult question, really.
Maybe poetry.
Okay.
Yeah.
Poetry.
Yeah.
Poetry is something that I use for my application at work.
It's the closest thing in Fison to, say, Yarn.
So I manage the dependencies and lock down the dependencies using poetry.
It's quite handy for that.
There are some issues with poetry, like when I just need to update one dependency,
update them all, and small issues like that.
But other than that, it's great.
Yeah.
It looks really great.
I know a lot of people are loving poetry.
It's a good recommendation there.
All right.
Final call to action.
People are interested in any I.O.
How do they get started?
Well, there's somewhat of a tutorial there.
I really don't have a really long tutorial on it.
I like Trio, so I'm heavily leaning on Trio's documentation here.
Because any I.O. has such a similar design to Trio, then a lot of Trio's manual can be
used to draw parallels to any I.O.
You can almost use Trio's documentation, the tutorial, to learn how any I.O. works.
Yeah, it's highly inspired, right?
Anything else, then you should just come to Gitter.
I think there's a link getting help at the bottom.
Okay.
Yeah.
So there's a Gitter link, and I'm usually available there.
Great.
Okay.
Yeah.
Very, very nice.
And I'm guessing you accept contributions?
Sure.
Yeah.
So, yeah.
Let's see.
Over here, we've got, what is that?
33 contributors?
So, yeah.
Excellent.
If people want to contribute to the project, and maybe that's code, or maybe even they
could put together a tutorial or something like that if they're interested.
Maybe.
Yeah.
Perhaps.
Okay.
Excellent.
Well, thank you for all the cool libraries, and take the time to come share them with us.
Thanks for having me.
Yeah.
You bet.
Bye.
Thanks, everyone, for listening.
Bye.
This has been another episode of Talk Python to Me.
Thank you to our sponsors.
Be sure to check out what they're offering.
It really helps support the show.
Listen to an episode of Compiler, an original podcast from Red Hat.
Compiler unravels industry topics, trends, and things you've always wanted to know about
tech through interviews with the people who know it best.
Subscribe today by following talkpython.fm/compiler.
Want to level up your Python?
We have one of the largest catalogs of Python video courses over at Talk Python.
Our content ranges from true beginners to deeply advanced topics like memory and async.
And best of all, there's not a subscription in sight.
Check it out for yourself.
Be sure to subscribe to the show.
Open your favorite podcast app and search for Python.
We should be right at the top.
You can also find the iTunes feed at /itunes, the Google Play feed at /play,
and the direct RSS feed at /rss on talkpython.fm.
We're live streaming most of our recordings these days.
If you want to be part of the show and have your comments featured on the air,
be sure to subscribe to our YouTube channel at talkpython.fm/youtube.
This is your host, Michael Kennedy.
Thanks so much for listening.
I really appreciate it.
Now get out there and write some Python code.
Thank you.