The relatively recent introduction of async and await as keywords in Python have spawned a whole
area of high-performance, highly scalable frameworks and supporting libraries.
One such library that has great async building blocks is OmniLib. On this episode, you'll meet
John Reese. John is the creator of OmniLib, which includes packages such as AIO iter tools,
AIO multiprocess, and AIO SQLite. Join us as we async all the things. This is Talk Python to Me,
episode 304, recorded February 16th, 2021.
Welcome to Talk Python to Me, a weekly podcast on Python, the language, the libraries, the ecosystem,
and the personalities. This is your host, Michael Kennedy. Follow me on Twitter where I'm at
mkennedy and keep up with the show and listen to past episodes at talkpython.fm and follow the
show on Twitter via at Talk Python. This episode is brought to you by Linode and Talk Python training.
Please check out the offers during their segments. It really helps support the show.
John, welcome to Talk Python to Me.
Howdy. It's good to be here.
Yeah, it's great to have you here as well. It's going to be a lot of fun to talk to you about
async stuff. I think we both share a lot of admiration and love for asyncioing all the
things.
I definitely do. It's one of those cases where the things that it enables is so different,
and you have to think about everything so differently when you're using asyncio,
that it's a nice challenge, but also has potentially really high payoff if it's done well.
Yeah, it has a huge payoff. And I think that it's been a little bit of a mixed bag in the terms of
the reception that people have had. I know there have been a couple of folks who've written articles
like, well, I tried it. It wasn't that great. But there's also, you know, I've had examples where I'm
doing something like web scraping or actually got a message from somebody who listened. Maybe they were
listening to Python Bytes, my other podcast. But anyway, I got a message from a listener after we
covered some cool asyncio things and web scraping. They had to download a bunch of stuff.
Like it takes like a day to literally it takes all day or something. It was really crazy. And then
they said, well, now I'm using async and now my computer runs out of memory and crashes. It's
getting it so fast. Like that's a large difference right there. Right. So there's certainly a category
of things where it's amazing.
Yeah, I think the case we've seen it most useful for is definitely doing those sorts of like concurrent
web requests. Internally, it's also extraordinarily useful in monitoring situations where it's like you
want to be able to talk to a whole bunch of servers as fast as possible. And maybe the amount of stuff
that comes back from it is not as important as being able to just talk to them repeatedly.
Yeah.
But you're right. There's definitely a lot of cases where people are not necessarily using
it correctly or they're hoping to like add a little bit of async into an existing thing.
And that doesn't always work as well as just building something that's async from the start.
Yeah. And there's more frameworks these days that are welcoming of async from the start,
I guess. Yeah. We're going to talk. Yeah. We're going to talk about that. But before we get too
far down the main topic, let's just start with a little bit of background on you. How'd you get
into programming in Python?
Sure. So my first interaction with the computer was when I was, you know, maybe like five or six years
old. My parents had a TI-99-4A, which is like the knockoff Commodore attached to the television.
Yeah. And I think back to that, like, how could you have like legible text on the CRT TV?
It was pretty bad.
It's bad, right?
It's like my biggest memory of it is really just every time we would try to play a game and the
cartridge or tape or whatever wouldn't work correctly, it would just dump you at a basic
prompt where it's just expecting you to start typing some programming in. And like nobody in
my family had a manual or knew anything about programming at the time. There was like, I think
maybe we figured out that you could like print something to the screen, but nothing beyond that.
And it wasn't until we ended up getting a DOS computer, you know, a few years later that really
started to actually do some quote unquote real programming where we were writing like batch
scripts to do menus or like, you know, deciding what program to run or things like auto exec on a
floppy disk in order to boot into a game.
I was just thinking of all the auto exec bad stuff that we had to do like, oh, you want
to play Doom, but you've got you don't have enough high memory, whatever that was. And so
you've got to rearrange where the drivers are. I mean, what's what a weird way to just I want
to play a game. So I've got to rework where my drivers are.
Make sure you don't load your mouse driver when you're booting into this one game that doesn't
need the mouse because otherwise you run out of memory. Yeah, it was kind of crazy.
And my biggest memory of programming there was there was QBasic on it and it came with
this gorilla game where you just like throw bananas at another gorilla from like some sort
of like city skyline.
Like a King Kong knockoff, Donkey Kong knockoff type thing.
Yeah, exactly. And I would struggle to figure out how that was actually doing anything.
And he's like, I'd try to poke at it and figure it out as I went. Didn't really do that much,
but it was actually my first opportunity for quote unquote open source projects because
there's a video game that I really, really liked called NASCAR racing.
And one of the things that I learned was you is on the burgeoning part of the internet for
me, at least was people would host these mods for the game on like geo cities or whatever.
And so these, these would change like the models for the cars or the wheels or add tracks or
textures or whatever. And I actually wrote a batch script that would let you like at the
time that you wanted to play the game, pick which of the ones you had enabled because
you couldn't have them all enabled. So it would like, right.
It's basically just a batch script that would go and like copy a bunch of files around from
one place to another. And then when you're done with the menus or whatever, then it would
launch the game. And I remember posting that on geo cities and, you know, having the silly
little like JavaScript counter or whatever it was, take up to like a couple hundred page
views of people downloading just this script to switch mods in and out. And so that was
like the, the first real taste of like open source programming or open source projects that
I had, but that actually like led into the way that I really learned programming, which was
I wanted to have my own website that was more dynamic than what geo cities had.
And so I ended up basically picking up Pearl and eventually PHP to write your web pages that I hosted
on my own machine at home from like IIS and active.
And how did you get a, what did you do? Do you use like dying DNS or something like that?
Yes, exactly. Dying DNS. It was the jankiest setup, but it at least worked and I could impress
my friends and it wasn't until I got to college and I was working on my first internship where
the main project I was working on was essentially improving an open source bug tracker written
in PHP in order to make it do the things that my company wanted to be able to do in it. So
like adding a plugin system and things like that. And in the process of that, they, I eventually
became a maintainer of the project and they had a bunch of Python scripts for managing releases,
like doing things like creating the release tar balls, running other sort of like linter type
things over the code base. And that was my very first taste of Python. And I hated it because it
was just like, I couldn't get past the concept of like, you're forcing me to do white space.
Like how barbaric is this? But it actually didn't take long before I realized that that actually makes
the code more readable. It's like, you can literally pick up anybody else's Python script and it looks
almost exactly like how you would have done it yourself.
Yeah. And you've got a lot of the PEP 8 rules and tools that automatically re, you know,
format stuff into that. So it's very likely, you know, you've got black and PyCharm's reformat
and whatnot, right?
This was all before that. So I think this was when like Python 2.6 was the latest.
This was quite a while ago.
Right before the big diversion.
Yeah, yeah, exactly. Like I had no idea what Python 3 was until like 3.2 or 3.3 came out because
it was just sequestered in this world of writing scripts for whatever version of Python was on
my Linux box at the time.
Right. You know, I suspect in the early days, probably the editors were not as friendly or
accommodating, right? Like now, if you work with PyCharm or VS Code or something, you just write
code and it automatically does the formatting and the juggling and whatnot. And once you get used to it,
you don't really think that much about it. It just magically happens as you work on code.
I'm wanting to say at the time I was just doing something stupid like Notepad++ or,
you know, one of the other like really generic free text editors.
Like Notepad, but Consolas fonts.
Or it was Eclipse. It might have been Eclipse.
Yeah. Was it maybe PyDev?
I don't think I ever use a Python specific editor.
Like, yeah, I think I've tried PyCharm exactly once and I do just enough stuff that's not Python
that I don't want to deal with an IDE or editor that's not generalized.
Right. Sure. Makes sense.
Speaking of stuff you work on, what do you do day to day? What kind of stuff do you do?
I'm a production engineer at Facebook on our internal Python foundation team. And so most of
what I do there is, you know, building infrastructure or developer tools, primarily enabling engineers,
data scientists, and AI or ML researchers to do what they do in Python every day. So some of that is like
building out the system that allows us to integrate open source third party packages into the Facebook
repository. Some of that is literally developing new open source tools for developers to use.
A while back, I built a tool called called Bowler that is basically a refactoring tool for Python. It's based off of lib two to three that's in, you know, open source Python essentially gives you a way to make safe code modifications rather than using regular expressions, which are terrible.
Yeah, for sure. And based on like the AST or something like that. Yeah, exactly. Yeah. Okay.
And it's like the benefit of lib CST is that it takes in the concrete syntax tree. So it keeps track of all the white space comments and everything else. So that if you modify the tree in lib two to three, it will then allow you to write that back out exactly the way the file came in. Whereas the AST module would have thrown all of that, you know, metadata away.
Right, right. Formatting spaces, whatever. It doesn't care.
Yeah. And one of the newer projects I've worked on is called USort and it's microsort. Essentially, it's a replacement that we're using internally for ISort because ISort has some potentially destructive behaviors in its default configuration. And our goal was essentially to get import sorting done in a way that does not require adding comment directives all over the place.
Right, right, right, right.
And you can't just put a skip directive on that function call, because that just means ISort won't try to sort that one, but it'll sort everything else around it as well. And so what we ended up seeing was a lot of developers doing things like ISort skip file, and just turn off import sorting altogether.
One of the things of ISort is like, first do no harm. It's trying its best to make sure that these common use cases are just treated normally and correctly from the start. In most cases, it's a much safer version of ISort. It's not complete. It's not a 100% replacement, but it's the thing we've been using internally. And it's one of the cases where I'm proud of the way that we are helping to build better tools for the ecosystem.
Yeah, this is really, I never really thought about that problem. One thing that does drive me crazy is sometimes I'll need to change the Python path, so that future imports, regardless of your working directory behave the same if you don't have a package or something like that, right? Something simple.
That's super common in the AI and ML type of workflows.
Yeah, and I get all these warnings, like, you should not have code that is before an import. Like, well, but this one is about making the import work. If I don't put this, it's going to crash for some people if they run it weirdly and stuff like that, right?
Yeah.
Interesting. Yeah, very, very cool project. Nice. All right. So let's dive into async, huh?
Sure.
Yeah. So maybe a little bit of history. You know, Python, it's hard to talk about asynchronous programming in Python without touching on the gill, global interpreter lock, normally spoken as a bad word, but it's not necessarily bad. It has a purpose. It just its purpose is somewhat counter to making asynchronous code run really quick and in parallel.
I mean, it's one of those things where if you imagined what Python would be without the global interpreter lock, you end up having to do a lot more work to make sure that, let's say, if you had multi-threaded stuff going on, you'd have to do a lot more work to make sure that they're not clobbering some shared data. Like, you look at the way that you have to have synchronizations and everything else in Java or C++.
We don't generally need that in Python because the GIL prevents a lot of that bad behavior. And the current efforts to kind of remove the GIL that have been ongoing for the past eight to 10 years, in every single case, once you remove that GIL and add a whole bunch of other locks, the whole system is actually slower.
So this is one of those things where it's like, it does cause problems, but it also enables Python to be a lot faster than it would be otherwise.
And probably simpler.
Yeah.
Yeah. So the global interpreter lock, when I first heard about it, I thought of it as a threading thing and it sort of is, but you know, it's primarily says, let's create a system so that we don't have to do locks as we increment and decrement the ref count on variables. So basically all the memory management can happen without the overhead of taking a lock, releasing a lock, all that kind of weirdness.
Yeah.
So we've got like a bunch of early attempts. I mean, we've got threading and multiprocessing have been around for a while. There's Jeevent, Tornado, but then around, I guess, was it?
Python 3.4. We got asyncio, which is a little bit of a different flavor than, you know, like the computational threading or the computational multiprocessing side of async.
It's actually an interesting kind of throwback to the way that computing happened in like the 80s and early 90s, where like Windows 3.1 or classic macOS, essentially you can, you know, run your program or your process and you actually have to cooperatively give up control of the CPU in order for another program to work.
So there'd be a lot of cases where, like, if you had a bad behaving program, you'd end up not being able to do multitasking in, you know, these old operating systems because it was all cooperative.
In the case of asyncio, it's essentially taking that mechanism where you don't need to do a lot of context switching in threads or in processes.
And you're essentially letting a bunch of functions cooperatively coexist and essentially say when your function gets to a point where it's doing a network request and it's waiting on that network request,
your function then will nicely hand over control back to the asyncio framework, at which point the framework and event loop can go find the next task to work on that's not blocked on something.
Yeah. And it's very often doesn't involve threads at all or, you know, the one main thread, right?
Like, so, yeah, you know, it's not a way to create threading.
It's a way to allow stuff to happen while you're otherwise waiting.
Yeah. In the best case, you only ever have the one thread.
And now in reality, it doesn't work like that because a lot of our, you know, modern computing infrastructure is not built in an async way.
So like if you look at file access, there's basically no real way to do that asynchronously without threads.
But in the best case, like network requests and so forth, if you have the appropriate hooks from the operating system, then that can all be completely in one thread.
And that means you have a lot less overhead from the actual runtime and process from the operating system because you're not having to constantly throw a whole bunch of memory onto a stack and then pull off memory from another stack and try to figure out where you were when something interrupted you in the middle of 50 different operations.
This portion of Talk Python To Me is sponsored by Linode.
Simplify your infrastructure and cut your cloud bills in half with Linode's Linux virtual machines.
Develop, deploy, and scale your modern applications faster and easier.
Whether you're developing a personal project or managing large workloads, you deserve simple, affordable, and accessible cloud computing solutions.
As listeners of Talk Python To Me, you'll get a $100 free credit.
You can find all the details at talkpython.fm/Linode.
Linode has data centers around the world with the same simple and consistent pricing regardless of location.
Just choose the data center that's nearest to your users.
You'll also receive 24-7, 365 human support with no tiers or handoffs regardless of your plan size.
You can choose shared and dedicated compute instances, or you can use your $100 in credit on S3-compatible object storage, managed Kubernetes clusters, and more.
If it runs on Linux, it runs on Linode.
Visit talkpython.fm/Linode or click the link in your show notes, then click that create free account button to get started.
Right, if it starts swapping out the memory it's touching, it might swap out what's in the L1, L2, L3 caches.
Yeah, exactly.
It can have huge performance impacts, and it's just constantly cycling back and forth out of control a lot of times, right?
Yeah.
In a lot of our testing internally when I was working on things that would talk to lots and lots of servers,
it's like we would hit a point where somewhere between 64 and 128 threads would actually start to see less performance overall
because it just spends all of its time trying to context switch between all of these threads.
Right, right, right.
You're interrupting these threads at an arbitrary point in time because the runtime is trying to make sure that all of the threads are serviced equally.
But in reality, like half of these threads don't need to be given the context right now.
So by doing those sort of interrupts in context, which is when the runtime wants to rather than when the functions or requests are wanting to,
you end up with a lot of suboptimal behavior.
Yeah, interesting.
Yeah, and also things like locks, mutexes, and stuff don't work in this world because it's about what thread has access.
Well, all the codes on one thread.
So to me, the real zen of AsyncIO, at least for many really solid use cases, kind of like we touched on,
is it's all about scaling when you're waiting.
Yes.
If I'm waiting on something else, it's like completely free to just go to it.
If I'm calling microservices, external APIs, if I'm downloading something or uploading a file or talking to a database,
or even maybe accessing a file with something like AIO files.
Yeah.
Yeah, there's a cool place called AI Awesome AsyncIO by TML Fear.
It's pretty cool.
Have you seen this place?
I have looked at it in the past.
I end up spending so much time looking at and building things.
It's like I haven't actually gotten a lot of opportunity to use a bunch of these.
Most of my time, I'm actually not working that high enough on the stack to make use of them.
Right, right, right.
These are a lot of more frameworks.
We do have some other neat things in there as well, like AsyncSSH.
I hadn't heard of that one.
But anyway, I'll put that in the show notes.
That's got, I don't know, 50, 60 libraries and packages for solving different problems with AsyncIO, which is pretty cool.
Yeah.
Whenever I talk about AsyncIO, one of the things I love to give a shout out to is this thing called Unsync.
Have you heard of Unsync?
I had not heard about it until I looked at the show notes.
But it sounds a lot like some of the things that I've seen people implement a lot of different cases.
It's a very filling a common sort of use case where you have, like I was saying earlier, where people want to mix AsyncIO into an existing synchronous application.
You do have to be very careful about how you do that, especially vice versa.
Or a lot of the stumbling blocks we've seen tend to be cases where you have synchronous code that calls some Async code that then wants to call some synchronous code, but on like another thread so that it's not blocked by it.
And you actually end up getting this like in-out sort of thing where you have like nested layers of AsyncIO.
I'm not sure how much this may or may not solve that.
I think this actually helps some with that as well.
Basically, the idea is there's two main things that it solves that I think is really neat.
One, it's like a unifying layer across multiprocessing, multithreading, and straight AsyncIO.
Right?
So you put a decorator onto a function.
If the function is an Async function, it runs it on AsyncIO.
If it's a regular function, it runs it on a thread.
And if you say it's a regular function, but it's computational, it'll run it on multiprocessing.
But it gives you basically an AsyncIO, Async and await API for it.
And it figures out how to run the loop and all.
Anyway, it's pretty cool.
Not what we're here to talk about, but it's definitely worth checking out while we're on the subject.
Ultimately, it gives you just a future that you can then either await or ask for the result from, right?
Yeah, exactly.
Exactly.
And the result, instead of saying, you've got to wait until it's finished before you can get the result,
you just go, give me the result.
And if it needs to, it'll just block.
So it's a nice way to sort of cap the AsyncIO.
You know, like one of the challenges of AsyncIO is, well, five levels down the call stack,
this thing wants to be Async.
So the next thing's Async.
So the next thing up is Async.
And like all of a sudden, everything's Async, right?
And so it was something like this.
I mean, you could do it yourself as well.
You can like just go create an event loop, run it.
And at this level, we're not going to be Async above it.
But we're coordinating stuff below using AsyncIO.
And here's where it stops.
Yeah.
It sounds like a nicer version of what I see dozens of when you have lots and lots of engineers
that aren't actually working on the same code base together, but they're all in the same
repository.
And we end up seeing these cases where everybody has solved the same use case.
So I do think this would be useful.
And I'm actually planning on sharing it with more people.
Yeah.
Check it out.
It's like a subtotal, I think, 126 lines of Python in one file.
But it's really cool, this unifying API.
All right.
I guess that probably brings us to OmniLib.
You want to talk about that for a little bit?
So this is what I thought would be fun to have you on the show to really focus on is like
AsyncIO, but then also you've created this thing called OmniLib, the OmniLib project that
has solves four different problems with AsyncIO.
And obviously you can combine them together, I would expect.
The origins of this really is like I had built the like AIO SQLite was the first thing that
I wrote that was an Async framework.
And then I'd built a couple more.
And at one point I realized these projects are actually getting really popular and people
are using it, but they're just like one of the hundred things that are on my GitHub profile
and graveyard.
So I really felt like they needed to have their own separate place for like, these are the
projects that I'm actually proud of.
I thought that was actually a good opportunity to be able to make a dedicated like project
or organization for it.
And essentially say that everything under this I'm guaranteeing is going to be developed under
a very inclusive code of conduct that I personally believe in and want to try and also at the same
time make it more welcoming and supportive.
You know, other contributors, especially newcomers or other otherwise marginalized developers in
the ecosystem and try to be as friendly as possible with it.
And it's like, this is something that I tried to do beforehand and I just never really formalized
it on any of my projects other than like, here's a code of conduct file in the repository.
Yeah.
But this is like really one of the first times where I wanted to put all these together and
make sure that these are really like, this is going to be whether or not enough people
make it a community.
I want it to be welcoming from the outset.
Right.
That's really cool.
And you created your own special GitHub organization that you're putting it all under and stuff like
that.
So it's kind of the things that are graduated from your personal project.
Is that a story?
Yeah.
And kind of the threshold I tried to follow is like, if this is worth making a Sphinx documentation
site for, then it's worth putting on, you know, OmniLib project.
So they're not all asyncio.
That just happens to be where a lot of my interests and utility stands at.
So that's what most of them are, or at least the most popular ones.
But there are other projects that I have also in the back burner that will probably end up
there that maybe not as useful libraries or whatever, but either way, like I said, these
are the ones that I'm at least proud of.
Nice.
That's cool.
So you talked about the being there to support people who are getting into open source and
whatnot and having that code of conduct.
What other than that, is there like a mission behind this?
Like I want to make this category of tools or solve these types of problems, or is it just
these are the things that you've graduated?
It's something I've tried to think about.
I'm not 100% certain.
I would like it to have maybe more of a mission, but at the same time, it's like, especially
from things I've had to deal with at work, it's like, I don't want this to be a dumping
ground of stuff either.
Like I want this specifically, it's like, like in the opening statement, I want it to be a group
of high quality projects that are, you know, following this code of conduct.
So from that perspective, it's like, at the moment, it's like my personal interests are always
in building things where I find gaps in, you know, availability from other libraries.
So that's probably the closest to a mission of what belongs here is just things that haven't
been made yet.
Yeah.
But either way, I just want to have that, you know, dedication to the statement of like,
I want these to be high quality.
I want them to be tested.
I want them to be, you know, have continuous integration and testing and well-documented
and so forth.
Yeah.
Super cool.
All right.
So there's four main projects here on the homepage.
I mean, do you have the attribution one, but that's...
Like helper tool.
Exactly.
Let's talk about the things that maybe they're the AIO extension of.
So in Python, we have iter tools, right?
Which is like tools for easily creating generators and such out of collections and whatnot.
So you have AIO iter tools, which is awesome.
And then we have multiprocessing, which is a way around the GIL.
It's like, here's a function and some data, go run that in a sub process and then give me
the answer.
And because it's a sub process, it has its own sub GIL or its own separate GIL.
So it's all good.
So you have AIO multiprocess, which is cool.
And then one of the most widely used databases is SQLite, already built into Python, which
is super cool.
And so you have AIO SQLite.
And then sort of extending that, that's like a raw SQLite, you know, raw SQL library that's
asynch.io.
Then you have AQL, which is more ORM-like.
I'm not sure it's 100% ORM.
You could categorize it for us, but it's like an ORM.
Yeah, I've definitely used like in quotes, in scare quotes, ORM-like.
Because I want it to be able to essentially be a combination of like well-typed table definitions
that you can then use to generate queries against the database.
As of right now, it's more like writing a, like a DSL that lets you write a backend agnostic
SQL statement.
Right.
Okay.
Yeah.
DSL domain specific language for people who aren't entirely sure.
Yeah.
So really it's essentially just stringing together a whole bunch of method calls on
a table object in order to get a SQL query out of it.
The end goal is to be able to have that actually be a full end-to-end thing where you've defined
your tables and you get objects back from it.
And then you can like call something on the objects to get them to update themselves back
into a database.
But I've been very hesitant to pick an API on it for how to actually get all that done
because trying to do that in an async fashion is actually really difficult to do it right.
And separately, like trying to do asyncio and have everything well-typed, you know, it's
like two competing problems that have to be solved.
Yeah.
I just recently started playing with SQLAlchemy's 2.0, 1.4 beta API where they're doing the
async stuff and it's quite different than the traditional SQLAlchemy.
So yeah, you can see the challenges there.
And it's also a case where it's like having something to generate the queries to me is more
important than having the thing that will actually go run the query, especially for a lot of internal
use cases.
We really just want something that will generate the query or we already have a system that
will talk to the database once you give it a query and parameters.
It's the piece of actually saying, defining what your table hierarchy or structure is,
and then being able to run stuff to get the actual SQL query out of it, but have that work
for both SQLite and MySQL or Postgres or whatever other backend you're using.
Having it be able to use the same code and generate the correct query based off of which
database you're talking to is the important part.
Yeah, cool.
Well, there's probably a right order to dive into these, but since we're already talking
about the AQL one a lot, maybe give us an example of what you can do with it.
Maybe talk us through, it's hard to talk about code on air, but just give us a sense of what
kind of code you write and what kind of things it does for us.
This is heavily built around the idea of using data classes.
In this case, it specifically uses adders simply because that's what I was more familiar
with at the time that I started building this.
But essentially, you create a class with, you know, essentially, all of your columns specified
on that class with the name and the type.
Not like SQL.
But not like super heavy.
Or Django style.
Yeah, exactly.
Like native types like id colon int, name colon str, not sa.column dot, you know, sa.string
and so on, right?
Yeah, exactly.
Like I want this to look as close to a normal data class definition as possible and essentially
be able to decorate that.
And you get a special object back that when you use methods on it, like in this case, the
example is you're creating a contact.
So you list the integer id, the name of it, and the email.
And whatever the primary key doesn't really matter in this case.
Whether the id ends up getting auto incremented, again, doesn't really matter.
What we're really worried about is generating the actual queries.
And you're assuming like somebody's created the table.
It's already got a primary key for id.
It's auto incrementing or something like that.
Yeah.
And you just want to talk to the thing.
Yeah.
Talk Python to me is partially supported by our training courses.
Do you want to learn Python, but you can't bear to subscribe to yet another service?
At Talk Python Training, we hate subscriptions too.
That's why our course bundle gives you full access to the entire library of courses for
one fair price.
That's right.
With the course bundle, you save 70% off the full price of our courses and you own them
all forever.
That includes courses published at the time of the purchase, as well as courses released within
about a year of the bundle.
So stop subscribing and start learning at talkpython.fm/everything.
And so essentially you take this contact class that you've created and you can call a select
method on it that will then, you know, you can add an aware method to decide which contacts
you want to select.
There's other methods for changing the order or limits or furthermore, if you wanted to do
joins or other sorts of things.
It kind of expects that you know what general SQL syntax looks like because you string together
a bunch of stuff kind of in the same order that you would with a SQL query.
But the difference is that in this case, like when you're doing the where clause, rather than
having to do an arbitrary string that says, you know, column name like, and then some string
literal, in this case, you're saying like where contact dot email dot like, and then passing
the thing that you want to check against.
And the other alternative is you could if you wanted to look for a specific one, you could
say like where contact dot email equals equals, and then the value you're looking for.
And so you're kind of using or abusing Python's expression syntaxes to essentially build up
your query, definitely using a domain specific language in this case.
But essentially having the fluent API, once you string all this together, you have this query
object, which you can then, you know, pass to the appropriate engine to get an actual finalized
SQL query and the parameters that would get passed if you were doing a prepared query.
But you could also potentially like in the future, the goal was you would also be able
to make manage your connection with a QL and basically be able to tell it to run this query
on that connection.
And regardless, you'd be able to do this the same with SQLite or MySQL or whatever.
And the library is the part that handles deciding what specific part of the incompatible SQL languages
that they all use will actually be available.
Right.
Yeah.
Like, for example, MySQL uses question mark for the parameters.
Yeah.
SQL server uses, I think, at parameter name.
There's like, they all have their own little style.
It's not the same, right?
Yeah.
And some of that is kind of moot because of the fact that the most of the engine libraries
that we use commonly in Python, like AIO, MySQL or SQLite or whatever, they're already kind
unified around the, there's a specific PEP that defines what the database interface is going to look like.
Right.
The DB API 2 or whatever.
Yes, that.
So some of that work has already been done by the PEPs and by the actual database engines.
But there's a lot of cases where it's a little bit more subtle, like the semantics, especially around using a like expression.
MySQL does a case insensitive matching by default, but SQLite doesn't.
AQL tries to kind of like unify those where possible.
But also there's cases, especially when you're getting into joins or group buys, things like that, where the actual specific syntax being used will start to vary between the different backends.
And that's where we've had more issues, like especially the whole point of SQLite for a lot of people is as a drop in replacement to MySQL when you're running your unit tests.
And so you want your code to be able to do the same thing regardless of what database engine it's connected to.
And this is one way to do that.
Okay, that's cool.
Yeah, with SQLite, you can say the database lives in, you know, colon memory.
Yeah, exactly.
And then you can just tear it up for your unit tests and then it just goes away.
Nice.
So maybe that brings us to the next one, the AIO SQLite.
Sure.
This one is all about async.io.
You can see from the example here.
You want to tell us about that?
Yeah, this was again, born out of, you know, a need for using SQLite, especially in testing frameworks and so forth to replace my SQL.
Cool.
And essentially what I was doing was taking the normal SQLite API from Python and essentially saying like, how would this look in an asyncio world?
Like if we were re-implementing SQLite from the ground up in an asyncio world, how can we do that?
And essentially, so in this case, we're heavily using async context managers and awaitables in order to actually run the database connection to SQLite on a separate thread and provide as much of an async interface to that as possible.
So when you connect to a IO SQLite, it spawns a background thread that actually uses the standard SQLite library to connect to your database.
And then it has methods on that thread object that allow you to actually make calls into that database.
And those are essentially proxied through futures.
So if you want to execute a query, when you await that query execution, it will basically queue the function call on the other thread and basically tell it, here's the future to set when the result is ready.
So once the SQLite execution or cursor or whatever has actually completed doing what it's supposed to do on that background thread, it then goes back to the original threads event loop and says, you know, set this future to finished.
And so that allows the thing originally awaiting it to actually come back and do something with the result.
Yeah, it sounds a little tricky, but also super helpful.
And people might be thinking, didn't we just talk about the GIL and how threading doesn't really add much?
But when you're talking over the network or you're talking to other things, a lot of times the GIL can be released while you're waiting on the internal SQLite or something like that, right?
Yeah.
So the internal SQLite library on its own will release the GIL when it's calling into the underlying SQLite C library.
And that's where it's waiting.
So that's good.
Yeah.
The other side of this is that it's one thread.
I'm not really aware of anybody who's opening, you know, hundreds of simultaneous connections to a SQLite database.
The way that people expect to do with, say, like AIo HTTP or things like that.
So while it is, you know, potentially less efficient, if you wanted to do a whole bunch of parallel SQLite connections, the problem really is that SQLite itself is not thread safe.
So it has to have a dedicated thread for each connection.
Otherwise, you risk corruption of the backing database.
Which sounds not good.
Right.
Yeah.
It's like, basically, you end up either where two threads clobber each other or more specifically, what SQLite says is, if you absolutely try to talk to a connection from a different thread, the Python module will complain unless you've specifically told it, no, please don't complain.
I know it's unsafe, at which point SQLite will be really upset if you try to do a write or modification to that database.
So there are layers of protections against that, but it is one of the underlying limitations that we have to deal with in this case.
So if you wanted to have simultaneous connections to the same database, you really have to spin up multiple threads in order to make that happen safely.
You could always do some kind of thread pool type thing, like we're only going to allow eight connections at a time and you're just going to block until one of those becomes free and finished or whatever, right?
It's definitely a tricky thing.
So like the expected use case with AIO SQLite is that you'll share the database connection between multiple workers.
So you'll like in the piece of your application that starts up, it would make the connection to the database and store that somewhere and then essentially pass that around.
And so AIO SQLite is basically expecting to use a queue system to say whoever gets the query first is the one that, you know, gets to run it first.
And whoever asked for the query second, you know, is the second one to get it.
So you're still doing it all on one thread and it's slightly less performant that way, but it's at least safe and still asynchronous at least.
Yeah, that's good.
Very nice.
And one of the things that looking at your example here, which I'll link in the show notes, of course, is Python has a lot of interesting constructs around async and await.
You know, a lot of languages, you know, you think C# or JavaScript or whatever.
It's kind of async function await function calls.
We're good.
But, you know, we've got async with, async for, a lot of interesting extensions to working with async and other constructs.
Yeah, it actually makes it really nice in some ways.
And essentially these are just syntactic wrappers around a whole bunch of magic methods on objects.
Right.
Like await thing, enter, do your thing, right?
Then await exit.
Right.
The nice part is that for some amount of extra work in the library, setting up all those magic methods everywhere and deciding, you know, the right way to use them.
The benefit at the end is that you have this very simple syntax for asynchronously iterating over the results of a cursor.
In that case, you don't have to care that after, you know, 64 elements of iteration, you've exhausted the local cache and now SQLite has to go back and fetch the next batch of 64 items.
In that case, it's like that's transparent to your application.
And that's where the coroutine that's iterating over that cursor would then hand back its control of the event loop.
And the next coroutine in waiting essentially is able to then, you know, wake up and go do its own thing, too.
Oh, how cool.
I didn't even really think of it that way.
That's neat.
Maybe next one to touch on would be AIO multiprocess.
Sure.
It just now crossed a thousand stars today or recently.
Oh, yeah.
Yeah, it did.
Yeah, very recently.
That's awesome.
That's my real pride and joy here is getting all those stars.
There's this interesting dichotomy set up between threading and multiprocessing in Python.
So with multi-threading, you're able to interleave execution.
So with the gil, it means that only one thread can actually be modifying Python objects or running Python code at any given time.
So you're essentially limited to one core of your CPU.
And these days, that's a big limitation, right?
Right, right.
Exactly.
Like I see servers on a regular basis that are like 64 to 100 cores.
So only using one of them is basically a non-starter.
You get a lot of people with pitchfork saying, why aren't we using Rust?
And so essentially what the alternative of this multiprocessing, where you're spinning up an individual process and each has its own gil,
this does allow you for CPU intensive things to basically use all of the available cores on your system.
So if you're crunching a whole bunch of numbers with NumPy or something like that,
you could use multiprocessing and saturate all of your cores with no problem.
In this case, essentially what happens is it spawns a child process or forks the child process on Linux.
And then it uses a pickle module in order to send data back and forth between the two.
And this is great.
And it's really transparent.
So it's super easy to write code for multiprocessing and make use of that.
But the issue becomes if you have a whole bunch of really small things, you start to have a big overhead with pickling of the data back and forth.
Right.
And the coordination back and forth is like really challenging, right?
Yeah.
So like if you're pickling a whole bunch of smaller objects, you actually end up with a whole bunch of overhead from the pickle module where you're serializing and deserializing and creating a bunch of objects and, you know, synchronizing them across those processes.
But the real problem is when you start to want to do things like network requests that are IO bound.
In an individual process, like with multithreading, you could probably do 60 to 100 simultaneous network requests.
Right.
But you guys maybe have more than 60 servers or something.
Sure.
Right.
But like if you're trying to do this with multiprocessing instead, where you have like a process pool and you give it a whole bunch of stuff to work on, each process is only going to work on one request at a time.
So you might spin up a process and it waits for a couple seconds while it's doing that network request and then it sends it back and you haven't really gained anything.
So if you actually really want to saturate all your cores, now you need a whole bunch more processes.
And that then has the problem of a lot of memory overhead.
Because even if you're using copy on write semantics with forking, the problem is that like Python goes and touches all the ref counts on everything and immediately removes any benefit of copy on write forked processes.
Right.
Which might do like the shared memory, right?
So if I create 10 of these things, like 95% of the memory just might be one copy.
But if you start touching ref counts and all sorts of stuff, you know, Instagram went so far as to disable the garbage collector.
Right.
To prevent that kind of stuff.
It was insane.
Yeah.
So it turns out that if you fork a process, as soon as you get into that new process, Python touches like 60 to 70% of the objects and it's in the pool of memory, which basically means it now has to actually copy all of the memory from all of those objects.
And so you don't actually get to share that much memory between the child and the parent process in the first place.
So if you try to spin up, you know, a thousand processes in order to saturate 64 cores, you are wasting a lot, a lot of memory.
So that's where I kind of built this piece of AIO multiprocess, where essentially what it's doing is it's spinning up a process pool and it only spins up one per core.
And then on each child process, it then also spins up an asyncio event loop.
Right.
And rather than giving a normal synchronous function as the thing that you're mapping to a whole bunch of data points, you give a coroutine.
And in this case, what AIO multiprocess is capable of doing is essentially keeping track of how many in-flight coroutines each child process is executing.
And essentially being able to say that, like, if you wanted to have 32 in-flight coroutines per process and you had 32 processes, then of course you have whatever 32 times 32 is.
I can't do that in my head because I'm terrible at math.
Essentially, you get, you know, the cross product of those two numbers.
And that's the number of actual concurrent things that you can do on AIO multiprocess.
So the idea is like, instead of creating a whole bunch of one-off run this thing with these inputs over there, you say, well, let's create a chunk.
Like, let's go 32 here, 32 there and run them, but do that in an async way.
So you're scaling the wait times.
Yeah, exactly.
Anyway, right.
Because you're probably doing network stuff.
Yeah.
And the benefit of this is essentially like you're scaling the benefits of asyncIO with the benefits of multiprocessing.
So for math, that's easier for me to figure out.
In reality, what we've seen is that you can generally do somewhere around 256 concurrent network requests on asyncIO on a single process before you really start to overload the event loop.
Have you looked at some of the other event loop implementations like UV loop or any of those alternate event loop functions?
So UV loop can make things faster, but the things that it makes faster are the parts that process like network request headers.
The real problem at the end of the day is that the way that the asyncIO framework and event loops work is that for each task that you give them,
it basically adds it to a round robin queue of all of the things that it has to work on.
So at the end of the day, if you want to run a thousand concurrent tasks, that's a thousand things that it has to go through in order before it gets to any one task.
Right.
And it's going around asking, are you done?
Are you done?
Yeah.
Or something like that, basically.
And if you're doing anything with the result of that network request before you actually return the real result from your coroutine,
then you're almost certainly going to be starving the event loop of or starving other coroutines on the same event loop of processing power.
And so what we've seen actually is you end up with cases where you technically time out the request because it's taken too long for Python or asyncIO to get back to the network request before it hits like a TCP interrupt or something like that.
That's interesting.
Yeah.
So this way you could say like, well, throw 10 processes or 20 processes at it and make that shorter.
If you're willing to run 256 network requests per process and you have 10 processes or 10 cores, then suddenly you can run 2,500 network requests simultaneously from asyncIO and Python.
At that point, you're probably saturating your network connection unless you're talking to mostly local hosts.
At Facebook, when you're talking about a monitoring system, that's actually what you're doing is you're almost certainly talking to things that have super low latency to talk to and super high bandwidth.
And so this was essentially the answer to that is like run asyncIO event, asyncIO event loops on a whole bunch of child processes, and then do a bunch of really like smart things to balance the load of the tasks that you're trying to run across all of those different processes in order to try and make them execute as quickly as possible.
And then also, whenever possible, try to reduce the amount of times that you're serializing things back and forth.
So one of the other common things that having more processes enables you to do is actually do some of the work to process, filter, aggregate that data in those child processes, rather than pickling all the data back to the parent process and then, you know, dealing with it and aggregating it there.
Right, because you've already got the like scale out for CPU cores.
Yeah, so it kind of gives like a local version of MapReduce, where essentially you're mapping work across all these child processes, and then inside each batch or whatever, you're aggregating that data into the result that you then send back up to the parent process, which can then process and aggregate that data further.
Yeah, super cool. You gave a talk on this at PyCon in Cleveland, one of the last real actual in-person PyCons.
Yeah, first one I've ever attended and the first one that I've ever given a talk at.
Yeah, that was a good one, the one in Cleveland.
Yeah, the room was absolutely massive and terrifying, and I don't know how I managed to do it all.
Yeah, it's just kind of block it out, block it out. But no, it's all good. Cool, yeah, so I'll link to that as well. People can check that out. And it really focuses on this AIO multiprocessing part, right?
Yeah. Nice. All right, last of the AIO things at OmniLib is AIO iter tools.
Yeah, so you kind of hinted on this before, like iter tools is mostly a bunch of helpers that let you process lists of things or iterables in nicer ways. And AIO iter tools is just basically taking the built in functions, like iterating, getting the next thing from an iterable or mapping, or chaining between multiple iterables or whatever.
And essentially bringing that into an async first world. So all of the functions in AIO iter tools will accept both like normal standard iterators or lists or whatever, as well as async iterables or generators or whatever.
And essentially, it up converts everything to an async iterable, and then gives you more async iterable interfaces to work on these.
So I know how to create a generator with like yield. So I can have a function, it does a thing, and then it goes through some process and it says yield an item, like here's one of the things in the list.
That's already really good because it does like lazy loading, but it doesn't scale the waiting time, right? It just waits.
So for the async generator, what's the difference there?
In this case,
Never tried one of those.
If you just call the function async def and then have a yield statement in it, it creates an async generator, which is just an async iterable object that similar to how when you call a coroutine, it's an object, but it doesn't actually run until you await it.
With an async generator, calling it creates the generator object, but you don't actually...
Then the async part is done, right? At that point.
Well, it's like it doesn't, it still doesn't even start running it until you actually start to use the async for some other async iteration to then iterate over it.
If you're using the async iterator, you still get the lazy loading of everything like with a normal generator, but you also have the potential for your thing to be interrupted.
The common use case here or the expected use case would be if you're doing something like talking to a whole bunch of network hosts and you want to return the results as they come in,
as an async iterable, then you could use something like AIO iter tools to then do things like batch up those results or run another coroutine across every result as it comes in, things like that.
The other added benefit in here is that there's also a concurrency limited version of gather.
So as I said earlier, when you have a whole bunch of tasks, you're actually making the event loop do a whole bunch more work.
One of the common things I've seen is that people will spawn 5000 tasks and each task or they'll all have some semaphore that limits how many of them can execute at once.
But you still have 5000 tasks that the event loop is trying to service.
And so you're giving a whole bunch of overhead every time it wants to switch between things.
It's got to potentially go through up to 5000 of them before it gets to one that it can actually service.
So the concurrency limited version of gather that AIO iter tools has lets you specify some limit, like only run 64 things at a time.
And so it will, you know, try to fetch the first 64 things of all of the coroutines or awaitables that you give it.
And it will start to yield those values as they come in.
But essentially, it's making sure that the event loop would never see more than 64 active tasks at a time, at least from that specific use of it.
Yeah, yeah, they're just hanging out of memory.
They don't really get thrown into the running task.
So one of the challenges or criticisms almost I've seen around asyncio is that it doesn't allow for any back pressure or whatever, right?
Like if I'm talking to a database, it used to be that the web front end would have like some kind of performance limit.
It could only go so hard against the database.
But if you do just await it, like all the traffic just piles in until it potentially can't take it anymore.
And it sounds like this has some mechanisms to address that kind of generally speaking.
That's at least the general intent of it is to be able to use this concurrency limit to try and prevent overloading either the event loop or your network or whatever.
So even if you have 5000 items, by setting the limit to 64, you know that, you know, you're only going to be doing that many at a time.
And then you can combine that that concurrency limited gather with something like the result of that is its own async iterable.
And then you could also combine that with things like chain or other things in order to mix that in with the rest of the like iter tools functional lifestyle, if you will.
Yeah, yeah, yeah.
Super cool.
I can imagine that these might find some way to work together.
You might have some asyncio, AIO iter tools thing that then you feed off to AIO multiprocessing or something like that.
Do you put these together any?
Yeah, exactly.
These are definitely a whole bunch of tools that I've put together in various different use cases.
Yeah, very neat.
All right.
Well, we're getting quite near the end of the show.
I think we've talked about a lot about these very, very cool libraries.
So before we get out of here, though, we touched on this at the beginning, but I'll ask you this as one of the two main questions at the end of the show.
If you're going to write some Python code, what editor do you use?
The snarky answer is anything with a Vim emulation mode.
That was the thing that I learned in college.
And I specifically avoided answering that earlier when we were talking about it.
But that's what I learned when I was writing a whole bunch of PHP code.
And that's what I used for years.
And then eventually I found Sublime Text.
And I really liked that.
But it kind of seemed dead in the water.
Adam came out, but Adam was slow.
And so these days I'm using VS Code primarily because it has excellent Python integration, but also because it has a lot of like Facebook builds a lot of things that we used to have on top of Adam called Nuclide, which especially were a lot of like remote editing tools.
Okay.
We've rebuilt a lot of those on top of VS Code because VS Code is faster, nicer, you know, has better ongoing support from the community and so forth.
Nice.
Yeah.
VS Code seems like the natural successor to Adam.
Yeah.
And like I said before, it's like I had tried PyCharm at one point, but it's one of those cases where I touch just enough stuff that's not Python that I really want my tools to work and function the same way regardless.
And so VS Code has the better sort of like broader language support where it's like there's some days where I just have to write a bash script and I want it to be able to do nice things for bash or, you know, I use it as a markdown editor and it has a markdown preview.
Things like that.
Yep.
All right.
Cool.
Sounds good.
And then notable PyPI package.
I mean, I guess we spent a lot of time on four of them, right?
Yeah.
I've also talked about microsort, usort.
Yeah.
So the joke answer is I have a package called AIO Seinfeld that's built on top of AIO SQLite.
And essentially you give it a database of Seinfeld scripts and you can search for things by actor or by keyword of what they're saying.
And it will essentially give you back some elements of dialogue from a script that contains your search query.
And this is powering a site I have called Seinfeld quote.com, which is basically just a really old bootstrap template that lets you search for pieces of Seinfeld quotes.
I also implemented a chat bot in Discord for some of my friends that also uses this.
The more serious answer would be the other one that we didn't talk about from OmniLib, which is attribution, which is essentially a quick program to automate the generation of change logs and to automate the process of cutting a release for a project.
And so I use this on all of the OmniLib projects.
And essentially I type one command attribution release.
I'm sorry, attribution tag and then a version number.
And it will drop a dunder version.py in the project directory.
It will create a get tag.
It lets you then type in what you want the release notes to be.
It's assuming, you know, a markdown format.
And then once it's made that tag, then it regenerates the change log for that tag and retags it appropriately.
And so you get this really nice thing where the actual tag of the project has both the updated change log and the appropriate version number file.
So you only ever type the version in once.
You only ever type the release notes in once.
And it gives you, you know, as much help and automation around that as possible.
Oh, yeah.
Okay, very cool.
That's a good one.
All right.
Final call to action.
If people are excited about AsyncIO, maybe some of the stuff at OmniLib, they want to get started.
What do you tell them?
If they want to get started on the projects, going to OmniLib.dev is the easiest way to find the ones that are currently hosted on the project.
We're always welcoming of code review from the community.
So even if you're, you know, not a maintainer, if you are interested in reviewing pull requests and giving feedback on things, always welcoming of that.
There's never enough time in my own personal life to review everything or respond to everything.
Otherwise, if there are things in these projects that you are interested in adding, like new features or fixing bugs or whatever, either open an issue or just create a pull request.
And I am more than happy to engage in design decisions or discussions or whatever.
Make sure that ideally, if you open an issue first, make sure you're not wasting your time on a pull request that's going in the wrong direction.
Right, right.
Because people might have this idea, but you're like, this is really inconsistent where this project is going to go or whatever.
So even if it's perfect, you can't accept it, right?
So good advice.
If it's just like a bug fix or something, then, you know, probably just worth creating a pull request and I'm not going to bite your head off.
But otherwise, the only other thing I would say is that LGBTQ things are very personal to me.
And so I would ask that if you're in a position to do so, that you please donate to an LGBTQ charity that will help in the community.
There's two that I really like.
One is called Power On.
And that's a charity that donates technology to LGBTQ youth.
They're either homeless or disadvantaged.
And they have that at poweronlgbt.org.
And then the other one is the Trevor Project, which is crisis intervention and a suicide hotline for LGBTQ youth.
And that's at thetrevorproject.org.
Yeah, awesome.
Those are just two examples, but there are plenty.
Worst case, just donate to a food bank near you.
Cool.
Yeah, that's a great advice.
Great call to action.
Seems like your projects are also really open to new contributors, people getting into open source.
So participating in that way seems like a great thing.
Fantastic.
All right, John.
Well, thank you so much for being on Talk Python.
It's been great to have you here.
Thank you for having me so much.
I really appreciate it.
This has been another episode of Talk Python to Me.
Our guest on this episode was John Reese, and it's been brought to you by Linode and Talk Python Training.
Simplify your infrastructure and cut your cloud bills in half with Linode's Linux virtual machines.
Develop, deploy, and scale your modern applications faster and easier.
Visit talkpython.fm/Linode and click the Create Free Account button to get started.
Want to level up your Python?
We have one of the largest catalogs of Python video courses over at Talk Python.
Our content ranges from true beginners to deeply advanced topics like memory and async.
And best of all, there's not a subscription in sight.
Check it out for yourself at training.talkpython.fm.
Be sure to subscribe to the show.
Open your favorite podcast app and search for Python.
We should be right at the top.
You can also find the iTunes feed at /itunes, the Google Play feed at /play,
and the direct RSS feed at /rss on talkpython.fm.
We're live streaming most of our recordings these days.
If you want to be part of the show and have your comments featured on the air,
be sure to subscribe to our YouTube channel at talkpython.fm/youtube.
This is your host, Michael Kennedy.
Thanks so much for listening.
I really appreciate it.
Now get out there and write some Python code.
I'll see you next time.
Thank you.