Ever since Python 3.5 was released, we've had a really powerful way to write I/O bound async code
using the async and await keywords. On this episode, you'll meet Nathaniel Smith, who wrote
the Trio async framework that significantly simplifies complex coordinating operations
using async and await. This is Talk Python to Me, episode 167, recorded June 21st, 2018.
Welcome to Talk Python to Me, a weekly podcast on Python, the language, the libraries, the
ecosystem and the personalities. This is your host, Michael Kennedy. Follow me on Twitter
where I'm @mkennedy. Keep up with the show and listen to past episodes at talkpython.fm
and follow the show on Twitter via @talkpython. This episode is sponsored by Linode and Rollbar.
Please check out what they're offering during their segments. It really helps support the
show. Nathaniel, welcome to Talk Python.
Hey, thanks for having me.
It's great to have you here. I've wanted to have you on for a long time. I, you know,
listeners of the show probably know that I'm a fan of async programming and parallel programming.
It's definitely gotten some feedback and some attention, but I think it's super important
for Python. And I think what you're doing is incredible. Some of the stuff you showed at
PyCon this year was incredible. So I'm just, I'm really excited to talk about it. Yeah, it's
going to be fun. Before we get to that though, let's get your story into programming and Python.
So let's see. So for getting into programming, I mean, I guess I was fortunate in my choice of
parents. I was sort of, you know, born into sort of a relatively affluent family. And my mother was
also a professional programmer. So she mostly stopped, you know, switched to sort of part-time consulting
consulting when the kids came along. So she could be a stay at home mom. So actually I first learned a
program through in my elementary school, they did have this program, this teaching logo to us that my
mom created this project at my elementary school. Wow. That's pretty awesome. Many moms volunteer at
schools, but not any of them. Right. Learning programs. Yeah, no, you know, my mom is pretty
incredible. She also loves podcasts. So I mean, I guess, hey, hi, mom. Maybe she'll listen to this.
Yeah. I mean, she later went on to get a master's in education, and now teaches art and science and
programming. Yeah. But so yeah, so I got started early and was fortunate enough to have my own computer
starting around like age 10 and so on. So this was like in the days when I was like,
it was DOS. Windows 3.1 was pretty exciting. Like 386, 46 type thing. Oh yeah. Yeah. Turbo button to
like make it go faster. Yeah. Yeah. We don't have turbo buttons anymore. No, it just always goes fast.
Isn't that weird, right? Yeah. Yeah. Isn't there a little button to make your laptop half the speed?
Why would everyone want that? So, and then I, from there I installed Linux when I was like 13 on my
computer messing around. I had no idea what I was doing. It was Slackware 96 because like, was the name of it?
It was, it's like, you know, Windows 95 was the big thing then. So like, this was one better, right?
96.
Exactly. Yeah. Right. So yeah. So I've kind of been, you know, bumming around that kind of,
you know, programming open source world for something years now. It took me a little longer to get into
Python. I don't remember exactly when it probably, you know, five years later. So I know the first time
people were talking about, you know, Python was this cool new thing. It's, you know, kind of like
Perl, but so forth. first time I read the tutorial, I got to the point where it said that
Lambda was syntactically limited to a single expression and just like rage quit. I was like,
whatever, stupid language.
Where are the curly braces? Come on.
Yeah. Well, it wasn't, I mean, I guess also like, so like, but I had like sort of like logo,
I learned scheme and things like that. So like, this was like obviously terrible. I'd later came
back to it, got over that. And I, and under, you know, understand now why that makes sense in terms
of the block structure and everything, but the statement expression distinction that Python has,
but, it took me a little while, but yeah. So I, yeah, I guess I got into Python around
like 2.2 or so. And it sort of gradually become, you know, just sort of my main language.
In the meantime, I was also like, you know, I was getting a PhD, stuff like that. So using Python
for both like hobby hacking stuff. And also like for my, in my work, you know, for data analysis,
I wrote some various libraries there. What's your PhD in?
Yeah. Cognitive science.
Oh, right. Okay. Very awesome.
Yeah. Yeah. So it was like studying how people understand language and brains understand language.
Yeah. So yeah. So along the way, and then, yeah, getting embedded and all these different sort of,
you know, entangled in all these different sort of open source projects, like NumPy and so on.
Yeah. Yeah. That's really great. So what do you do these days?
So I'm actually kind of in a bit of transition right now. So the last few years, I've been
working at UC Berkeley at the Berkeley Institute for Data Science, which is a sort of new
data science center. That's what started up on campus. And so in my position has been sort of
this unique, one really where it's, been, I've been grant funded to sort of figure out how to,
you know, make sure the Python, you know, continues to work well, works better for scientists. There's
a, actually there's a blog post we can like put a link to there. I wrote just sort of talking about
all this sort of stuff I've done there, but some highlights being like, I got, some grants for
a NumPy development, $1.3 million over a few years, which is basically the first time NumPy
has been funded. We'll have full-time people working on it. Did I made a color map people like,
did some work on packaging? So like, many Linux sort of led that effort so we can have wheels on
Linux now. That sounds like a really fun job. Like you get to actually make a difference in
the open source space. Exactly. Yeah. I mean, it partly it's, it's sort of funny. Like this is what
I talk about in the blog post, but like, you know, open source is just sort of like desperately underfunded.
And like, it's amazing how well and effectively we can use the little bits of volunteer time and
things that people manage or, you know, people have spending, you know, a few hours a week.
But there's a lot of things that like larger projects that have huge impact that can't be
done in that mode where you actually need to like sit down for a little while and think about it and,
you know, understand the landscape, put some focused effort into something.
Yeah. It's a difference on like what you take as your goal. If you have a month of uninterrupted full
time, some project or a goal part of project versus I'm going to squeeze this in
Saturday morning before the kids get up, right? These are the same types of things you attack.
Exactly. Yeah. I mean, it's like a month of time is not that much in the grand scheme of things. One
person, one month is like compared to, you know, the amount of labor that like, you know, Google has
available. Yeah. But it's still, you know, enables all kinds of things that right now just
aren't happening. And so there's actually lots of low hanging fruit because no, there is nobody
almost who has that kind of time to worry about these sort of broad open source ecosystem kind of projects.
There's just, there's all kinds of... We saw how much of a difference was made with the Mozilla
grant, the Python package.
Or the PyPI. Yes.
Yeah. The PyPI, like that was like, hey, look, $170,000. Oh, actually this can happen. It's
been dragging on for years. Now it's, you know, it's real.
Yeah. Yeah. I mean, getting the team together to make that happen. Yeah. I had a little bit of
involvement on that. I'm on the PSF's packaging working group. I didn't, I wasn't heavily involved
in that. You know, I like made some introductions, gave some feedback on the grant and things.
So it was, yeah, it's just super exciting to see that up close. Because yeah, it was very successful.
And it wasn't...
Like it really got a lot out of how much investment was put into it.
Yeah. I mean, it was like, that had been dragging on for literally six years. The old PyPI was just
totally unworkable. But, you know.
It was like Kryptonon for people who wanted to contribute to open source. They looked at it and
like, oh, no, no, I'm not touching that. That's...
Yeah. I made one patch to the old PyPI. And it was just a trivial thing. It was just that we wanted it
to start to say that, oh yeah, many Linux is now a legal thing for a wheel to have in its name.
So it was just like, there's like a list of strings that allowed, I'm just adding one new string,
right? That was the most terrifying patch I've ever made. PR I've ever made. Because it's like,
if I have a typo, then PyPI dies, right? And there are no tests. There's nothing, right?
The consequences of failure is so high.
Yeah. Exactly.
Yeah. The new PyPI is way, way better. You should all go and help make it contribute. Because now
it's all set up. You can develop locally and there's a real test suite.
Yeah. It's really nice. I had the people involved on the show a couple episodes back too. So it
definitely got a chance to dig into that. Oh yeah. Right. That's right. Yeah. Yeah.
So let's talk a little bit about async. Yeah.
Yeah.
Because I think that that's one of the sort of revitalized areas of Python last,
since 3.4, 3.5, it really started coming along, right? So what, in 3.4, we got asyncio as a thing.
And in 3.5, it really, I feel like it really got more accessible with async and await.
Right. With the new syntax features to make it easier to use. Exactly. Yeah.
Yeah. Yeah, exactly. Like the keep it, the foundation was built previously, but it was,
it was still this sort of callback hell type of programming, right? Right.
Right. We should maybe take a little bit of a step back and just, you know, let people know about
the presentation you gave at PyCon, which introduces your project TRIA, which is what we're going to
focus on. But in your talk, you sort of set the stage and said, look, there's these two
philosophies or types of async programming that you might consider. So maybe, maybe you could touch on
that a little. Well, so, I mean, I think the first thing to say is for those who aren't kind of already,
you know, up on the jargon, async is sort of a subset of concurrent programming. And so concurrency,
meaning writing programs that do two things at the same time, which is very handy. You know,
it's something in our real life we do all the time, you know, you know, I'm working on one thing and my
friends working on another thing at the same time. It's very natural. But writing a program that does
that is a little bit trickier, especially, you know, in Python is kind of generally a sequential
language, right? It makes it easy to do, I want to do this and then that and then the other thing.
It doesn't directly have built into like the syntax, whatever ways to say, I want to do these
two things at the same time. And then when they're both done, do these other things.
So, so generally, there's this general question of like, how do you write concurrent programs in Python?
And then I think what you were thinking of, there's kind of two philosophies of concurrency,
which is one is the kind of preemptive concurrency that threads give you, where just everything
just kind of runs all the time interleaved kind of in arbitrary ways.
To be clear, this is in general threads, not Python threads, because we have this whole
thing called the gil.
From the programmer, from the user's point of view, the gil doesn't make, gil makes things
slower, but it doesn't make it really change what it feels like to use it compared to threads in
general. True.
So in general in threads, you might have like two threads actually running at the same time,
like on two different views. Because of Python, we have the global interpreter off the gil,
multiple, then it means that mostly only one thread can actually run at a time.
But because in Python interpreter controls that, it can decide at any moment to switch which thread
is running. From your point of view, as writing code in Python, it might as well be running multiple
things at the same time, basically, right? Because since it could switch at any moment,
you kind of have to act like it's just constantly switching, right?
And the reason this is kind of a challenge is because if you have a data structure, and you have two
different threads or whatever, you know, you have two different concurrent pieces of code acting on
the data structure at the same time, if they're not like, if you're not careful, you can make a big
mess. So like, you know, one, you have the classic examples are things like, you know, your program is
like managing like a bank. And so I'm going to withdraw money from this account, and then add it,
put it in this account. But if you don't do that, like atomically, so like ice, you know, first one,
one thread says, okay, does Alice have enough money? Yeah, great. Okay, I'll take that money out and put it
into Bob's account. Another thread says, oh, does Alice have enough money for this transfer to Carl?
And then if they're in a leave right, first both threads check and say, oh, yeah, Alice has plenty of money.
And then both threads take money out of the account. And now Alice has spent the same money twice, actually.
Yeah, great for Alice, but not so nice for the bank.
So yeah, someone's gonna be trouble running writing that software.
Yeah, right. So yeah, if you're writing that software, you need to sort of manage this.
Yeah. So one of the things that kind of helped me this like click in my mind was
thinking about how programs temporarily enter these invalid states.
Right.
What you're describing is basically that like at the beginning of that function,
the bank is in a valid state. At the end of the function, everything is valid. But somewhere in
between in order to accomplish that operations as a series of steps, it has to become invalid. And
long as nobody observes it in this invalid state, it's all good. But when they do like, like you're
describing halfway through, it's not okay anymore. It's kind of like transactions and databases,
but in time. And yeah, they're pretty similar in some ways, I guess.
Yeah. So yeah, exactly. So with threads, which the solution to this is you have to sort of explicitly,
like, make sure you notice all those places where you're passing through that invalid state and do
some kind of in like market somehow in your source code, like say, okay, I'm going to take a lock here.
That's going to make sure that anyone else who tries to use this has to wait for me to be done
and things get back to the valid state before they can look at it. Right?
Yeah, exactly. Okay. But that's really error pro.
And this is preemptive concurrency, right?
Yeah, exactly. It's still talking about kind of how threads work, right?
Yep. So you would have to like find all these places where you go through a temporarily
invalid state and mark them in your source code. And if you forget one, then you have this nasty bug
where Alice gets to suspend the money twice or all kinds of weird things can happen.
And it usually has to do with timing. And so it's very,
super hard to debug.
Yeah. And it's like, yeah, it's like super subtle. Like, yeah,
like it only happens one in a thousand times randomly. And it happens,
it depends how much memory you have. And it's only happens in production and not on your tests.
And just all kinds of, it's really, really, yeah, it's bad. Yeah.
Yeah. I've heard them described as Heisen bugs. And I just love that term.
Yes. Yes. Heisen bugs. Right. And it's just like,
and so it means that when you're working with threads,
you just have to be like constantly vigilant, right? Like every line of code,
which we think, okay, is this the one that's going to introduce this terrible bug?
So that's, that sucks, right? You don't want to live like that.
So, yeah, right. You just have to be like constantly paranoid. So yeah. So the idea for
async concurrency, because we kind of flip it around instead of saying that like, okay,
we have to go through and find all the places in the source code where something dangerous is
happening and mark those. You say, you know what, let's be conservative. Let's assume something
dangerous could be happening anywhere. That's the default. So by default, only one thing is allowed to
happen at a time. And then we'll mark in the source code, you know, okay, here's a place where
I want to let other things happen. It's okay for, I'm not in the middle of, you know, just doing some
delicate operation, you know, adjusting someone's bank account. This is a fine place for that to happen.
This portion of talk Python to me is brought to you by Linode. Are you looking for bulletproof
hosting that's fast, simple, and incredibly affordable? Look past that bookstore and check
out Linode at talkpython.fm/linode. That's L I N O D E. Plans start at just $5 a month for a
dedicated server with a gig of RAM. They have 10 data centers across the globe. So no matter where you are,
there's a data center near you. Whether you want to run your Python web app, host a private get server
or file server, you'll get native SSDs on all the machines, a newly upgraded 200 gigabit network,
24 seven friendly support, even on holidays and a seven day money back guarantee. Do you need a little
help with your infrastructure? They even offer professional services to help you get started
with architecture, migrations, and more. Get a dedicated server for free for the next four months. Just visit
talkpython.fm/linode.
And you can still have bugs. You can still make mistakes and put that in the wrong place.
Because now there's only like a finite number of places to check when something goes wrong, right?
It's just much easier to reason about that kind of program.
Right. And that could be done with just straight up asyncio. But with three, five and above,
like basically the await keyword is the marker in your code for that.
Exactly. Yeah.
Yeah. Yeah. That's pretty beautiful. That's pretty beautiful. So I'm guessing,
I'm guessing most people probably under understand like how to use async and await the keywords,
but maybe just describe like what's, how to define like a super, super simple async method with the
keyword, just to the people understand this.
Yeah. Like you said, I'm going to be focusing on trio here and trio does kind of use only kind of a
simple subset. There's some extra complexity needed for like backwards compatibility with async
IOS sort of called that place thing. We can get, talk about that more later. But sort of for if,
especially if you're just, you know, if you're using trio, async await is very simple actually.
And so here's what you need to know is that there are two kinds of functions. There's the async ones,
which are these, the special functions that might let other code run during them. And so you kind of
need to be aware of what you call one of them that like, you know, the ground might shift under
feet, data structures might change while it's happening. And there's the regular functions,
like the synchronous functions where those always happen atomically. And so, and we're going to do is
we're going to make all the regular Python functions you already know, those are going to be the synchronous
ones. Because, you know, no one, you know, they're all written on the assumption that like, you know,
Python's a sequential language, it does things sequentially. And so they, no one like thought about
how to handle this async stuff when they were writing all these libraries that already exist.
So those are going to be all atomic. And then we need a way to mark these special async functions.
And we want to mark it at two places. So what we want to mark it when we define the function
so that we, you know, as part of the API, is this an async function or not? And we want to mark it at
the call sites. When you call one of these, when you're like reading through your code, you see, ah,
this is a point where I'm calling an async function. This is a point where other code might run in the
background, I need to make sure I'm not in the middle of some, you know, dangerous operation.
Right?
Right. Okay. So you've got like, maybe a call to a web service or something.
Yeah, that might return like a future, which then you hook in a call.
So there are no futures in Trio. That was one of the things I got rid of.
Yes. Yeah, that's beautiful. Yeah. But you got to, you have to wait on the result, basically.
Right?
So yeah. I mean, so yeah, so the way, yeah, the way you think about it for Trio is just that,
it's just like, there are these special functions and the way you call it is you type
a weight and then the function call. It's like a new kind of syntax for calling a function.
Right.
And that's sort of all you need to know. There is, yeah, like you said, like there's the
complexities around Python also supports a weighting on objects and having future objects,
like the function to return and so on and so forth. But it's kind of unnecessary clutter is
kind of how Trio's perspective on it. So we just don't do any of that.
Yeah. I feel like the AsyncIO layer is like, let's get something kind of working,
but it's not that delightful. And I feel like Trio really cleans it up. So there's a really great
example that you gave in your talk about happy eyeballs, which is like a way to try to use
basically DNS resolution to connect a socket to a server and some concurrency stuff around that,
which is, I don't want to talk about that yet. Maybe we'll have some time to talk about it later. But
basically there's a version in Twisted, which is how long was the one in Twisted? Hundreds?
And so, yeah, well, yeah, there's two different versions of Twisted I talked about in the talk.
One is the sort of the classic one that's in in master, which is like 120 lines long, I think,
roughly. I mean, not that it's not super meaningful to talk about lines of code like this, but like
just kind of give you a sense. Yeah. Yeah, exactly.
A better sense of complicated is to reason about that is it has inside this method. It has another
internal function. Inside that function, there's another function defined. And then inside that,
there's a fourth level of internal functions. So like, yeah, that's bad.
Because it's all these. Yeah. Yeah.
Almost like it's full of all these go to's and in some weird sense. And then, and you basically say,
now, if we apply the building blocks or the primitives of trio to it, oh, look, it's like 20 lines,
and it's totally straightforward, which is really great. So I think, you know, let's talk about trio.
Why don't you start by telling people what it is. Trio is a library for a sink concurrency in Python.
It's like an alternative to libraries like a sink or twisted or so on, which NATO or threads,
sort of all of them. And I think there's kind of, I think it was, there's kind of two pieces. So one
is there's sort of the idea trio, the ideas are like a research project or something where I have,
I sort of did some analysis of like, what is it about twisted and I think I own someone that makes
it so difficult to use sometimes, or why are things like happy eyeballs so complicated?
Where are some common errors sort of come from? It came up like, oh, that's actually I sort of,
as I dug into it, I realized actually, there's, you know, a small number of things that kind of
seem to cause a lot of the problems, and even ended up digging into some old literature from like the
60s, about early programming language design. On Univac or what was that one on? That was like
a really old computer. Oh, oh, Flowmatic. The, that language, which is, yeah, the Grace Hopper's
language, the precursor to COBOL, which is a really interesting language. That's going way back. Yeah.
Yeah. Yeah. And sort of talking about this transition, there was a lot of debates then about like,
how do you structure your language, not even getting into the concurrency part, just like,
how do you do a language at all that's like usable? And one of the big things that happened was
this switch from using go to as the main control structure to having things like if statements and
functions and for loops and so on. And I realized there's actually sort of an analogy there that's
surprisingly like precise between a lot of that, a lot of these async libraries are actually still
kind of in the go to stage, where sort of the basic primitives are in sort of a technical way,
kind of analogous to a go to, and they cause similar kinds of problems. And then if you look at,
okay, how did, you know, Dijkstra solve these problems back in the late sixties,
we can take those and apply them to concurrency. And that leads to something called,
I call it a nursery concept. Yeah. It's so interesting. And I had, yeah. And I had never
thought about the relationship between go to and many of these programming, these threaded programming
models, but you really hit it. Well, I think it's a super good analogy and it's really relevant.
Because so what Dijkstra said was, look, you should be able to treat these building blocks as black boxes.
Stuff goes in, stuff goes out. That's all. If you know kind of what it does, you don't need the details,
right? I mean, this is like a huge part of abstraction and programming, like functions,
classes, it's modules, et cetera. Right. Like, like even just think of like,
like in Python, think of the print function, right? Like in actually it does all kinds of complicated
stuff, right? It's got talking to the operating system interfaces to do IO and it's like buffer
a thing and character second version and blah, blah, blah. But like, you don't have to care about
that. You just type print hello world. And it, you know, it prints all the world.
You can just treat that as like this little atomic unit. And that's great.
That kind of. Yeah. It's such an important part of a building block of programming,
but these threads mean that stuff can still be going. You're just like all over the place. Right.
And it's very similar to the go-to's, which I thought that was a great analogy.
Yeah. I mean, specifically the issue is, so like, let's say the, the analog of print,
like in say twisted, you have this transport object. That's like your connection,
like a network connection and you call it's right method. And this is like,
I want to send this data to the remote site now. And that's like a function. You call it
it returns. That's all fine. But what's sort of confusing is that when you call it,
it returns, it hasn't actually finished yet. What it's actually done is sort of scheduled
that right to happen in the background. And that makes it hard to reason about the flow.
Cause it's like, Oh, the functions returned, but actually in a sense, it's like kind of still running.
Right. And now if I want to write and then do something else that happens after that,
that's hard to manage because I don't necessarily know when that's actually finished. I have to use some
other API to like check and ask, okay, has it actually written yet?
Yeah. And so yeah.
Did it succeed? Did it not succeed? Then how do you deal with that? And then how do you get that
back into your, yeah, your whole, it's yeah. It feels like almost like JavaScript.
Well, I mean, JavaScript, you know, is also in this general family of like, yeah,
it is as this async currency model that's sort of endemic to it. We use all over the place and it's
all callback based and it's all, yeah, it has that same kind of go-to problem.
Yeah, exactly. So let's talk about the architecture and then maybe like we can see about how that kind
of brings us back into something that Dykstra might be proud of.
Okay. Sure. Yeah. I guess I should also mention, so I don't know how,
this is a discussion that really benefits from like diagrams, which is not something podcasts or
there is also, I have a blog post called notes on structured concurrency or go statement considered
harmful, which goes into this and sort of this set of ideas and much more detailed pictures and all
that. But so yeah, so the basically the idea is that, I mean, so it's what I said, right? That like,
when you call a function, it should be something you can treat as a black box. It does some stuff,
it returns. That's kind of what Dykstra's point. I mean, that's the problem with go to is also that
like, you know, the old go to world, you could call a function and maybe it returns, maybe it jumps
somewhere else, maybe some other function and some other point of your code suddenly returns instead
because the control go to just lets, you know, functions have a certain control flow they're supposed
to have, right? Where you have your function and then you call something. So control jumps into that
function. It executes that code and then it comes back again to your function. That's kind of this
nice little structure. That's why it's, you could treat it as a black box because you know, oh, it'll
do some stuff, but then it'll come back. Right. Go to isn't like that. It's just, it's just a one way
you leap off somewhere else and then maybe you come back, you could jump back, it's something you do manually.
Choose your own adventure book. You don't know when you're done, you don't know where to begin or
you're just like, it's always a mess. It's always a surprise. Yeah. Yeah. I mean, it's choose your own
adventure accepted. Then you're like, you know, actually I want to switch. I'm going to, it says
I can go to page five or 10, but I'm going to 17. Let's see what happens. So yeah. So in that,
and that breaks this function abstraction, right? So it means like you call a function and you know,
hopefully like if it's like a nice, well-written library and people aren't like messing with you,
it will come back normally. But there's no kind of guarantee of that in the language. Like if
someone decided to be clever, they could, it could do something else. It could jump somewhere else.
And then it becomes very hard to sort of reason about the overall structure of your program,
if that's true. And it also breaks sort of higher level structures that people are using the program.
But like, so, so think about exceptions, right? So it's very, a handy thing about exceptions is like,
oh, something goes wrong. Then, you know, and I didn't think about how to deal with it locally,
then the exception will propagate out in Python. And so until either someone catches it and knows how to
deal with it, or the program crashes, you get a trace back and like, that's not great. But at least,
you know, you didn't just like go blindly on doing the wrong thing. At least, you know,
you get a trace back, you can try and figure out what's going, what happened, right?
But not with, not with threading. Not if you care.
Well, okay, well, we'll get there in a moment, right? But if you have go to,
then it's where do you, when you raise an exception, it goes up the stack, right? It's like,
okay, does this function want to handle it? Okay. How about it's caller? How about it's caller's
caller? Right? Because you have this nice stack kind of, you know, you know who the caller is,
you know who the caller's caller is. If you have go to, then control just bounces around sort of at
random. So who wants to know about this exception? Like I don't have a well-defined sense of a caller
even, right? It's just like these basic things we take for granted growing up and you know, this,
you know, our ancestors struggled with and sort of fixed for us. And now we take for granted,
right? But like these basic assumptions just aren't there in the language of go to.
Yeah. With blocks have similar issues, right? So like a with block is supposed to like,
you say with open file and then you know, okay, inside the block, the file's open,
outside the block, the file's closed. That's nice. It's great. It makes it easy to manage resources.
But if you have go to, you like, you jump into the middle of a block, you jump out of the middle of
the block. Like what happens is the, where does the file open and close? Like how does that even,
I don't even, it doesn't work, right? Exactly. Yeah. So yeah. And then,
then if you look at like problems we're dealing with threads or things that are a struggle and like
asyncio or twisted, but these are just kind of the things that are problematic, right? So you call
a function, is it, you know, it's still running maybe when it returns, it makes it hard to like
control sequencing, hard to treat things to black boxes. Like you don't know, like you need to like,
go read the source code when you use some random, asyncio like based library to find out like,
okay, I called this function, but did it actually do the thing then? Or is it scheduled something in
the background? Like you have to go, does the error go to the caller or does it go to the callback?
Right. Yeah. So if you spawn off this background thread or background task, as I think it calls it,
and it crashes, there's an error. There's an unhandled exception with that. Well, you've lost,
you don't have a nice sort of call stack that you've, you've spun up this new independent entity that's
executing and it's now split apart from your original code. And so there's sort of like, where does that
if there's an unhandled error, an exception, then what actually happens is that in threads and
they sync IO and twist or whatever, is it, you know, maybe print something on standard, like,
hey, I hope someone's looking, something went wrong. Then it throws away the exception and it carries on.
And you know, hopefully... It does make your software more reliable,
because there's way fewer crashes when you don't actually... Well, for some value of reliable,
right? Yeah. Exactly. Nice. Okay. So what are the building blocks of Trio that like...
Yeah, exactly. Yeah. Yeah. So the main thing that is, in Trio, we have this thing we call a nursery.
And the idea, it's just a sort of a silly joke, right? About if you want to spawn a child task,
then it's not okay for it just to be go off independently. Like it has, you have to like,
put it somewhere and it'll be like taken care of. It's like a nursery is where children live.
Okay. So concretely, what it is, is if you want to spawn a child task in Trio, you have to first
have a nursery object. The way you get a nursery object is you write async with Trio.openNursery as
nursery. So it's like this open, you have like a with block that opens a nursery and then you get this
nursery object. And the object is tied to that with block. And so, and then once you have that,
you can say nursery, there's a method on that nursery to spawn a task into it. And all the tasks you
spawn to the nursery will run concurrently. But then the trick is that that with block,
its lifetime in the parent is tied to the lifetime of the child tasks. So that you can't exit the
with block while there's still children running. If you hit the end of the with block, then the
parent just stops and waits for them. The stuff within the with block could be interwoven
and like in certain ways, async and concurrent. But taken as a whole, the with block is like a black box.
You start it, it runs, and then out comes the answer at the end, and everything's done,
or it's gotten into its canceled state or whatever happens to it, right?
Exactly. Exactly. Yeah. And so right. And that so that lets us do things like so right. So now,
if you call a function in Trio, you know, well, okay, it might internally open a nursery and have
some currency. But by the time it finishes, it's done, it hasn't left anything running. Or if you have,
if something crashes, we have some child task, you know, has an exception that is no, it doesn't catch,
then we say, okay, what do we do with this? Oh, well, wait, the parent is just sitting there waiting
for the child. So that's what we do is the exception hops into the parent and continues executing. So
in there, sorry, continues like propagating out. So Trio sort of follows the normal Python rule of,
you know, you can catch an exception, but if you don't, then it will keep propagating until someone
does. Right. So that nursery is a really important block block here. And I think, you know, it's,
it's really cool to be able to start all these child tasks and they can even be sort of children
of the children, right? Like you could pass the nursery and the child task could spawn more child
tasks and whatever. And so it's all going to sort of be done at the end.
This portion of talk Python to me has been brought to you by roll bar. One of the frustrating things
about being a developer is dealing with errors, relying on users report errors, digging through
log files, trying to debug issues, or getting millions of alerts, just flooding your inbox and ruining your
day. With roll bars, full stack error monitoring, you get the context insight and control you need to
find and fix bugs faster. Adding roll bar to your Python app is as easy as pip install roll bar. You can
start tracking production errors and deployments in eight minutes or less. Are you considering
self hosting tools for security or compliance reasons? Then you should really check out roll bars,
compliance SAS option, get advanced security features and meet compliance without the hassle of self
hosting, including HIPAA ISO 27001 privacy shield and more. They'd love to give you a demo. Give roll bar a
try today. Go to talkpython.fm/roll bar and check them out.
The other thing that's really important in this type of threading is to not block forever, to not wait
on something forever, right?
Oh yeah, so you're talking about timeouts and cancel scopes.
Timeouts and cancel scopes and all that stuff.
Yeah, yeah, yeah.
Yeah, so this is a really basic problem. If you're doing anything, kind of network programming,
for example, or any kind of thing, your programs talking to other programs, other resources,
because sometimes those will just stop responding. So it's like, if you make an HTTP request,
then maybe it'll succeed. Maybe it'll fail, you'll get a 404 or something like that.
Or, but there's also a third possibility, which is that it just never finishes, right? Like the
network disappeared and it just like sits there forever. And your program, yeah. And like, you kind
of, and I mean, if it's just writing a little script or whatever, it's like that you're running
at the command line, it's fine. You know, you have some point you'll get bored and hit control C,
it's fine. But for sort of more complicated systems, you need to be robust against this.
Yeah.
Or unattended systems.
Yeah.
Like, well, we have 10 workers, but we're only getting this super low throughput. Why? Well,
because eight of them are like just blocked forever.
Yeah. You need to somehow, you know, detect this, be able to get out and say, okay, actually,
let's stop doing that. Try something else, raise an error or something, but like, you know,
not just sit there forever. So that means, yeah, there's just like, and this is just a problem
that's just endemic, right? It's just every time you do any kind of networking or other kinds of IPC,
you need to have this kind of timeout support. So that's one thing that makes it tricky.
The other thing that makes timeouts tricky is that usually the place where you need to set
the timeout is like buried way, way down deep inside the code. It's like, at some point your
HTTP client, you know, is using some socket library and the socket library is like trying to send some
bytes or read some bytes from the network, like a very, very low level operation. That's probably buried
under like five levels of abstraction. But then where you want to set the timeout is like way out at the
top where you say like, I just want to say, if this HTTP request isn't done in 10 seconds,
then give up. So timeouts are this problem that's like, you need them everywhere and you need
coordination across different layers of your system. So it's this really complicated problem that like
covers, you really kind of need your timeout framework that's like used universally by everyone
and kind of like coordinates, can coordinate across the system.
And it can be super tricky. Like imagine you want to call like two web services and do a database
transaction. And if one times out, you want them all to like roll. Yeah. Right. Which is like super,
like how do you coordinate that? Like there's just, you know, like, yeah, almost impossible. Right. Unless,
yeah. Unless you put it in trio. So yeah. So right. Yeah. So the traditional way to do this is like,
you have a timeout framework sort of inside each library, like inside, you know, your HTTP client has a
timeout argument that you pass or whatever, but that doesn't help with people just don't do that
reliably. And yeah. And it doesn't solve these coordination problems. So in trio, you say like,
no, this is something that we're just going to bake into the library, the IO library itself.
So all libraries that work with trio, if you have an issue that in trio, it uses trio's timeout system.
And so in this, this thing called cancel scope. So again, there's a lot more details on this in a
blog post that I wrote called timeouts and cancellation for humans. We'll put the link up there.
Podcast description, I guess. But in basically, yeah, it's, it's fairly simple to use. Basically,
the way it works is you can just anywhere in your code, say, you know, with timeout,
10 seconds, and then you put code inside that with block. And if it takes more than 10 seconds,
then trio raises a special exception called canceled that just sort of, you know, unwinds out of there.
And then that with block catches the exception. So, you know, it's a way of saying, okay,
whatever's happening with block, stop doing that, and then carry on at that point.
The other nice thing that's important about this, as compared to a lot of other systems we're doing,
working with timeouts and this cancellation, is that we have the with block to limits, like you can say,
this is the operation that I want to cancel, which is really important. So like asyncio doesn't work like this.
So when I think IO, you can say, I want to cancel something, and that injects the special exception,
but then there's nothing that keeps track of, you can't look at the exception, figure out, okay,
did I want to cancel this specific network operation, or this HTTP request, or this like entire program,
like, kind of, you don't keep track of that. When trio, because we said it's with block, we can say,
okay, I know this is the actual set, this is the operation that I'm trying to cancel right now.
And these can be nested.
Yeah, it's awesome. And you, if you want to do like a timeout, you can create a with block that says,
basically, do this to success or fail after 10 seconds. What is the syntax for that? It's like,
async with, I forget.
This one is just a regular with.
Okay.
Because, so async with is just like a regular with except that a regular with does, calls a method at
the beginning and the end of the block. And async with does an await call of a method at the beginning
of the block. And we talk about a weight being the special, you know, thing we call these special
functions. Yeah. And so for, it happens that for tree for timeouts, for nurseries, you have to use
an async with because at the end of the block, it might, the parent might have to wait for the
children. And you want to let them run while it's waiting. Right. So it has to be an await there.
So let's use async with for a timeout. You're just, you know, setting something at the beginning
of the end, but there's, it doesn't, it doesn't have to actually like stop and wait for, let other
things run there. It can be synchronous. That's just a little detail, but yeah. So it's, it's basically,
so with block, what we, the main one, sort of this basic one is called move on after to kind
of remind you that it's going to run the code. And then if the timeout happened, it doesn't
raise an exception. Like it, you can stop and see, okay, did it, was it canceled or not? But like,
it keeps executing after the block. So you can like, look at it, which is, this again has to do with,
like, like the simplest case for a timeout is just like, okay, if the timeout expires,
blow up everything, give up, raise an exception, right? But a lot of times you want to say, oh,
well, did this thing get canceled? If so, do some fallback, do something else. So the core thing in
Trio is, is not to raise an exception after that. It's like to provide some object you can look at to
see what happened and then figure out how you want to deal with it.
Yeah. The other thing that I thought you did well with these cancellation scopes is if there's an
exception on one of the tasks and there's a bunch running in one of these width blocks, you can
catch that and use that to cancel the other ones and just say, no, no, things are going wrong. We're
out of here. Every, all threads are done. We're out of here. Yeah. So yeah. So the nurseries actually
kind of need the cancel scopes because what, one of the things the nurseries do is this exception
family thing, right? So if an exception happens in a child, it isn't caught, then it has to hop
into the parent. So we have this kind of, I guess the way I think about it is normally you think of
like your call stack. It's like this little chain of like, you know, A calls B calls C calls D,
right? What nurseries do is they kind of make it that into like a tree. So like A calls B, B calls C and D
and E sort of simultaneously. And then maybe D calls several things at the same time, right? So you have
this kind of like this tree structure of your, for your call stack. And now some exception happens down
inside one of those branches. Well, what do you do with exceptions? You unwind the stack,
right? You go, you run any handlers inside that function. And then you go up to the parent,
you kill that functions call and you move in there and run any handlers there and so on.
But then we have this problem. If you're going up, unwinding the stack, and you hit one of these
places where branch, the stack now branches, how do you sort of unwind past that point? Right?
You don't want to like orphan the other children, like you just unwind the parent without
stopping them because that's our whole thing in Trio, right? We say we're not going to let
orphans run around unsupervised. That's a bad idea. Dijkstra doesn't like that. So what we do is,
but we have to unwind the stack, right? So what we do is we use the cancellation at that point to go and
we cancel all the other tasks that are inside that same nursery. So we sort of prune the tree,
unwind those tasks back up to this point. And then we can keep unwinding from there with the original
exception. Yeah, that's, that's really clever. I really like the way you put it all together.
So there's also some other pieces, like some file system bits. I love the, the whole trick of saying
like a wait sleep for a little bit of time, just go, I'm basically yielding this thread to let it do
other work, but then just carry on as opposed to say like a time.sleep and things like that.
So maybe tell us about the higher order of building blocks.
Yeah. So I guess, yeah, so far we've sort of been talking about kind of the core ideas in Trio, which
are, you know, not specific to Trio the project. In fact, there's so, I've had a, talking to a number
of other language designers or other languages, like, oh, this is interesting. We're also struggling with
these problems. But then, yeah, but Trio is also a specific project you can download. It has
documentation and API and all that. And one of the things I try to do with that project is to make
it sort of really sort of usable and accessible, kind of have this philosophy of like, the, you know,
kind of the buck stops here for developer experience. Like if there's something that you're trying to,
you're trying to write a program using Trio and there's something awkward or difficult or a problem
you have, then it's up to us to solve that. And even maybe it's, it's so we, it's, we have the project
itself. We also have things like that, like, you know, has the core, like networking and concurrency
stuff. We also have testing helpers and we have a documentation, like a plugins for Sphinx to make
it easier to document these things. And so there's something called pytest-Trio. How's that work with
that? So the main thing that pytest-Trio gives you is that you can have, when you write a test, you can say,
okay, I guess we need to give a little more information here. So Trio, there's sort of a
stereotype pattern that all Trio programs for, where you have like an async def main or whatever you want
to call it. That's like your top level async function. That's like, that's where you go into
that Trio's magic async mode. If you use all the Trio stuff.
Right. So maybe you start by saying trio.run main.
Yeah. Yeah. So the thing about async functions, you can only call them from other async functions.
So you have this problem. How do you call your first async function? And that's what trio.run.
So you have this little bit of ceremony at the very top of every Trio program. And this is also the case
for like asyncio and so on. But like, you say, I have to like switch into trio mode. And that's kind of
annoying in a test suite to do that on every single test. So that's the most basic thing that pytest-Trio
does for you is it makes it so you can write async def test, whatever, and it will take care of like
setting up trio and turning it on there. But it also has some other handy stuff. So one thing
is that it allows you to have async fixtures. And it does some sort of magic to like goes,
it switches into trio mode, and then sets up your fixtures and then calls your test,
and then tears down the fixtures. So it's sort of all within this async context.
And it's also integrated with some of the testing helpers that are built into trio itself.
So in particular, the one that's sort of the most, you know, gee whiz, awesome,
is that trio has this ability to use a fake clock. So this is an issue when you're writing like
networking programs. Like often you want to like have a test about, okay, what does happen if this
HE request just hangs forever? Like do my timeouts work correctly? Stuff like that, right? But it's
really annoying to run these tests because it's like, okay, and now I have to sit and wait for a
minute for the timeout to fire and make sure something happens. And that happens every time you run your
test. It just spends a minute sitting there doing nothing. And you're like, I really like, okay, but
like, this is really boring. I want my test suite to finish already. And I need like 100 of these
tests. And yeah.
So killing my build time.
Exactly. Yeah. Right. It's like really ruins your flow. So one thing I've done, actually,
what I did is when writing trio itself, I said, okay, I really want trio's own test suite to run
really quickly. I figured that'll force me to like come up with the tools that my users will need to
make their test suite run quickly. Yeah.
So if you just type pytest trio, you know, if you want to contribute to trio, we'd love to have
you. We're very friendly and all that. If you type pytest trio, it runs in like five seconds.
It has like 99 point something percent test coverage, which is like completely, it's very
difficult to get there because trio is this really complicated sort of networking library. It's all
this stuff that's usually hard to test. Part of that is that for all the timeout tests,
we have this magic clock. And so the way it works is you say, okay, trio, I don't want you to use,
I know it says like sleep 30 seconds or whatever. I don't want you to actually use sleep 30 real
seconds. I only want you, I want you to sleep 30 virtual seconds. And so it's a special thing you
sort of pass the tree at our run to say, every time you have timeouts, sleeping, anything inside this call,
I want you to sort of use this virtual clock instead. And the way the virtual clock works,
it starts out at time zero and it just stays there. And you can like advance it manually if you
want or things like that. But normally what you do is you just use the automatic behavior,
which is, it's just, it's, it's at time zero. And then it sort of watches what your program is
doing. Anytime that your program sort of like finishes running everything and just stops and
is waiting for stuff to happen, then it looks to see, okay, looks like the next thing that would
wake up, there's like a sleep 10 and sleep 20. Okay. So in 10 seconds, that's the next one that'll wake
up. I'm just going to jump the clock forward 10 seconds and then start running again. Right.
So anytime it knows it's going to be waiting for a certain amount, it's like, all right,
we'll just, we'll let the wait start and then we'll just go right past that.
So it's basically, yeah, you just write your test the way you normally would with like for use timeouts
regularly, test your real code, but sleeps, whatever is easiest. And then what's annoying about that
normally is then your, your test takes like 30 seconds, a minute, whatever it is to run. Most of
which the time is just sitting there doing nothing, waiting for time to pass. So if you flip the switch to
use the special clock, then it does this exactly the same things, but it just skips over all those
times when it's sitting doing nothing. And so suddenly your test runs in like a few milliseconds.
Oh, that's awesome. Yeah. It's pretty awesome. And then it tests trio is hooked up to that.
So you can just turn this on with just like flip a switch on any test.
Oh, that's great. Yeah. So one of the things that makes me a little bit sad about Python's async
loops and stuff is like the asyncio based apps and the trio based apps. Those are not exactly the same
and they're not exactly compatible. It's not like the core you're using the same core. And so it just
keeps running like the asyncio loop and the trio loop. These are not the same. They got to be like
brought together with different APIs. Right. Yeah.
But you seem to have, you do have some interoperability. So like trio can work with
libraries that maybe assume asyncio is there or something, right?
Trio itself is just like a totally different library than asyncio. I've looked at, you know,
could I build it on top of asyncio? And there's sort of a number of reasons why that didn't sort of
make sense. And yes, and there is this big problem because it's just because of technical things about
how these async concurrency systems work. There has to be like one library ultimately that controls
all the actual networking like asyncio or trio or whatever, or twisted or tornado or something.
And that means to like, if you have a, like a say an HTTP client that's written to work on async
IO, it will necessarily work on trio because it's using a different set of networking primitives underneath
or vice versa. And this is sort of a, a larger sort of ecosystem problem, right? So there used to be,
there's twisted and tornado and G event and they all, none of them could interoperate. You'd have to
like pick which one they're using. And asyncio was sort of one of the reasons it exists is to try
and solve that problem and become the standard one, but then twisted and tornado and everyone can use.
And now they can all work on top of asyncio. And now all those libraries written for twisted and tornado,
you can mix and match however you like. And then here comes trio and kind of ruins that
by being here's this new thing you should use. So to try and kind of mitigate that,
there is this library called a trio asyncio, which lets you use asyncio libraries on top of trio.
The way it does this is kind of, it creates like a virtual asyncio loop that internally uses trios primitives
under the cover. And it kind of lets you, you know, kind of cordon them off and kind of a little container,
sort of all the weird stuff they sink IO can do. You can do that stuff, but, the kind of in a little box
that won't like leak out to pollute the rest of your program, your trio program.
I think this is really encouraging because that means if you maybe have already invested in AC IO and
you've already got some code written on it, like you could still
Yeah.
Trio without going, I'm rewriting trio. And is that worth it? Is that a good idea?
Yeah. Or again, it gives you sort of an incremental path. You can say like, well, okay, I can at least
get it running on trio first of all, and then I can start porting one piece at a time and eventually
end up all in trio. Hopefully.
Exactly.
Now, the reason it's not, you can't just magically make this all work because
trio and Cinco really have fundamentally different ideas about things. Now, obviously, I think trio's
ideas are better. They're kind of the new thing that I try to fix all these problems, but it's not that
it's the differences aren't just like in terms of the internal implementation. The differences are in
terms of just like the fundamental concepts that are exposed.
Right. Like the philosophy of it.
Yeah. Right. It totally changes how you write the library on top.
Right. So it's not something you can just sort of magically switch.
But there's a little bit of an incremental aspect to it. So we're almost out of time.
Right.
Just really quickly, what's the future of trio? Like where is it going? What you got planned?
And is it production ready?
So yeah. So I should be clear. Yeah. Right now, the trio library itself is very solid,
but there is not much of an ecosystem around it. So like there is not currently an HTTP client or
an HTTP server that you can just use out of the box and it's like mature and all that
for trio. There are some solutions for these kinds of issues. And I don't want to say too
much because, you know, this will change quickly. We have a chat channel. If you go to our
documentation or whatever, you can like find out what the latest news is about what you should use.
But it's not something that, you know, is ready today to run big websites or something like that.
Okay.
Just because the libraries aren't there yet. If you'd like to help, you know, write those libraries and
make it happen. I'd love to have you. We have a really solid contributing policy and things like
that. You can check it out. The other thing that's happening is asyncio. So I also, I spent a lot of
time. I am a core Python developer. I talked to Yuri Selvanov as the main asyncio developer and Guido
about all this stuff. And so there is this, Yuri is quite keen on saying, oh, well, right. You know,
trios ideas are better. We should add them all into I sync IO. This is quite complicated. There's a lot
of, I mean, we could probably do a whole other podcast about all the trade offs there. And maybe
we should, I don't know. It's pretty interesting. It is interesting.
Maybe relevant. So that's something that's also happening is that Yuri is going to be trying to
add nurseries and cancel scopes and things to I sync IO. So I think there's a lot going to be a lot of
limitations. It's a lot of the value in trios of things people can't do. And I sync IO has already
got like six layers of abstraction built in there or I don't know. It's not actually six. It's like
four. And they're all totally doing all the things that trio says, these are, these are things that
should never be done. It shouldn't be possible. So you can't, that's also, you could fix just adding
a new layer on top, but you know, it's still better than nothing, right? Like, you know,
I think it would continue to exist. So we do want to make it as good as possible.
By these ideas. Yeah, absolutely.
And ultimately we don't, I mean, maybe like no one knows for sure whether like the make a new thing
plus a compatibility layer, like trio, the trio, the single thing I mentioned, is that going to be
the best thing or is making asyncio better going to be the best thing? We'd, none of us know for
sure. So we are trying both versions. I will sort of see.
I'm super excited just to hear that that collaboration is happening. I think that's great. All right.
I think that we're out of time for trio. It's super interesting project. I really love what you've
done there. I think it's, I think it's brilliant. So, people should definitely check it out.
Thanks a lot.
Yeah, you're welcome. So quick two final questions. If you're going to write some Python code,
what editor do you use?
I use Emacs. I've been using it for 20 years. I'm stuck.
It's great. It's not any, I don't know that it works for other people or not just because,
yeah.
Yeah, sure. I definitely, I started on Emacs as well. And notable pipe effect.
Yeah. Well, trio, obviously.
And, pytest.strio?
Yeah. pytest trios, Sphinx controlled trio. There's, if you go to the, you know,
github.com/python-trio to see all the different projects under the trio organization. And they're
sort of trying to build up that ecosystem. Like I said, so.
Yeah. Sounds cool. Yeah. So final call to action. People are excited. They want to try trio. Maybe
they want to contribute to it. What do they do?
Yeah. So check out, start with the documentation trio.readthedocs.io. That also will give you
links to our chat is, sort of a place to hang out. it has our contributing docs. If you want
to get involved like that, we give out commitments on your first pull request acceptance. So there's
lots of people. Yeah, we, we want, you know, this is a project for everyone. I don't want to just be my,
you know, personal little thing.
Yeah, that sounds great. Awesome.
Yeah.
All right. Nathaniel, thank you for sharing your project and creating it. It's quite great. And,
we may have to come back and dig into this a little bit more. This is fun.
Yeah. Thanks for having me. Yeah. Yeah. Talk to you later.
You too. Bye bye.
This has been another episode of talk Python to me. Our guest on this episode was Nathaniel Smith,
and it was brought to you by Linode and Rollbar. Linode is bulletproof hosting for whatever you're
building with Python. Get four months free at talkpython.fm/linode. That's L I N O D E.
Rollbar takes the pain out of errors. They give you the context insight you need to quickly locate and
fix errors that might have gone unnoticed until your users complain. Of course, as talk Python to me,
listeners track a ridiculous number of errors for free at rollbar.com/talkpython to me.
Want to level up your Python? If you're just getting started, try my Python jumpstart by building 10
apps or our brand new 100 days of code in Python. And if you're interested in more than one course,
be sure to check out the everything bundle. It's like a subscription that never expires.
Be sure to subscribe to the show. Open your favorite podcatcher and search for Python. We should be right at the top.
You can also find the iTunes feed at /itunes, Google Play feed at /play, and direct RSS feed at /rss on talkpython.fm.
This is your host, Michael Kennedy. Thanks so much for listening. I really appreciate it. Now get out there and write some Python code.
I'll see you soon.