What do web servers do most of the time? They wait. They wait on external systems while processing a
request. Think about a standard web request to an e-commerce site where you're logged in.
You send a session cookie and request to a URL. It pulls a bunch of items from the database,
maybe a Redis cache, and even talks to an external API. And that is exactly the situation
AsyncIO is built for. But to take advantage of it in the Python web frameworks, the framework
itself has to support AsyncView methods. That's what SANEC was built to do. On this episode,
you'll meet Adam Hopkins, who is leading the SANEC project. This is Talk Python to Me,
episode 188, recorded November 27th, 2018.
Welcome to Talk Python to Me, a weekly podcast on Python, the language, the libraries, the ecosystem,
and the personalities. This is your host, Michael Kennedy. Follow me on Twitter where I'm at,
mkennedy. Keep up with the show and listen to past episodes at talkpython.fm. And follow the show on
Twitter via at Talk Python. Adam, welcome to Talk Python.
Thank you for having me. I'm excited to be here.
Yeah, I'm excited to have you here. I'm really excited to talk about asynchronous web frameworks
with you, in particular, Sanic. But before we get to that story, let's start with yours. How'd you
get into programming in Python?
I started getting into programming mostly in high school, but I've sort of been around computers
all my life. I mean, my dad had computers when I was a kid in the 80s, and I really remember not
having computers. I've always been very comfortable, you know, tinking around and playing with them.
But I think I really got started into programming mainly by, I wanted to build a website. I was
interested in trying to put things online, and I started to build some stuff. And I mainly started
out working with PHP, because at the time, that was sort of the big kingpin if you wanted to put
something online, and it was easy to get started.
Yeah, it's definitely, and it's still massive on the web.
Yeah, it is. I think WordPress definitely helps get people started, and is a really good intro and a
nice tool for people. And I kind of had my own, I don't want to call them problems, but I started,
I felt like every time I was working in PHP, I sort of needed to keep one eye on the documentation,
because I could never remember.
Yeah. It wasn't obvious. It wasn't clear. It's just one of those languages, right?
Exactly. You know, one time there's a function, and the parameters are needle and haystack,
and the next time the search is haydl and e-stack, and I can never remember which way the namings
convention. And so I started looking around to try to find something else.
You're like, it's got to be better than this. Other people use other tools. What are we going to use?
Exactly. And I did know a little bit about Python, and more, you know, just, I'm going to write a script
and print out Hello World type stuff. I mean, nothing sophisticated at all. And at the time,
this was just about when Django had been released. And Django had been released, and I think Ruby on
Rails was out there. So it was a question of, like, here are these two different ways, and which way am I
going to go? And I picked one, and I'm happy that I picked Django because it led me down a nice path,
I think.
Yeah. And I think that's, you know, that's a pretty, I don't know if it was luck or just good,
but it was definitely a good choice that you went that way. I feel like Rails has definitely lost a
lot of the favor that it had. I know there's a lot of projects out there that's still built on Rails,
but it doesn't seem to have the shine that it did five years ago.
No, and kudos to the Django team. I mean, I think what they're doing is pretty amazing that they continue
to build a great product that's still very relevant and still hugely popular. I don't know, the framework's
got to be, what, 15 years old maybe at this point? Maybe not quite.
Yeah, but it's been around for a while, so that's pretty awesome. And did you practice law for a while?
I did, I did.
Tell us about that. That's far from programming, typically.
Well, yeah, so I didn't go to school as, you know, I'm not a computer science major or anything like
that. Everything that I know about programming is entirely self-taught. I went to school,
I graduated from George Washington University in DC, and I went to law school after that. After
law school, I practiced law for about five years. And all during that time, I didn't really ever lose
touch with programming, but it was a hobby for me. Building things was,
it was a hobby and was not my day-to-day job. But in 2014, my wife and I decided that, you know,
we were living in Massachusetts at the time, and we decided that we were going to move across the world
to Israel. And it was sort of as this life-changing time, and we decided, you know, we're going to make
this move. And as part of that, we're going to assess our career paths. I felt very lucky that I sort of was
able to take what had been my hobby and now turn it into a career. And I think that's a pretty fortunate
thing. I think it's great. And I definitely think it's one of the real benefits of being in technology.
We have so many ways of working that are just asynchronous already. Even when we're together,
right? We all use GitHub and Slack or email or whatever. Right. And you could be 5,000 miles away
and still do that, right? That's pretty awesome. Yeah. I mean, the team that I'm working on,
working with right now, it's a pretty small group of guys. But, you know, I'm in Israel. There's a
couple of guys in Ohio or North Carolina, Brazil. Like the team is, you know, we exist as a team. We
function day to day as if we're a team and interact, you know, like it's, you know, the guy sitting right
next door, but really he's literally halfway around the globe. Yeah. That's pretty awesome. So maybe
that's a good segue into what you do day to day. Like, are you doing, what do you work on day to day?
Sure. So my day job, I work with a company called Matrix Retail and their main product is geared towards
retail companies that have a lot of different... Like Macy's or some Target or something like that.
Exactly. The type of companies that exist in malls across the U.S. I mean, they're mostly U.S.-based.
And the idea is they're constantly getting all this data, you know, how many sales they had,
how many transactions, how many people are walking in and out of the store. And we sort of take all of
that data and try to make a projection for what their sales are going to be. And then once we have
that information, we can create a schedule for their staff. And so we determine you're predicting to have
a thousand dollars worth of sales during this particular hour on this particular day. And
how many people do you need to have in the store to be able to reach that thousand dollars of sales,
but not have too many people that you're just paying for people to stand around doing nothing?
Interesting. So you're kind of like load balancing, but for employment.
That's actually a great way to put it. I like that.
There you go.
You know, I think one of the bigger complications in that is, you know, it's not just a matter of
how do you slot people into the right place, but then you've got to deal with jurisdictional laws.
So one place, you know, shift might need to be, can't be shorter than three hours. And another
place, there can't be two shifts that are closer together and they need to have meal breaks and
have to be a certain length. And so all these sort of validation rules we have to take into account.
And it makes for a pretty complex puzzle.
I'm sure it does actually. Anytime law gets involved, it's crazy as I'm sure you know, but it
is this in Python?
The application itself runs a pretty big stack of a bunch of different technologies.
A lot of it is in Java.
Like most places, right?
Yeah, yeah, it's true. It's true. A lot of the stack is in Java, the earlier stuff. And
what I've been working on is creating sort of a new API on top of what's currently existing to
get some more features, get a little bit more performance. And actually that's a lot of how,
how I came into, to using Sanic. We can get into that a little bit later, but all this new stuff that
I'm adding is all in Python and load balanced across a pretty big cluster using Kubernetes and
some other fun tools like that.
It sounds like you're using a lot of cool technologies. And I think it's really cool using
Sanic for that microservice bit. So you said we're switching to Python because it's for speed and to
make it fast in addition to adding features, which I think is, you know, so many people who are not
deeply involved in Python, like, oh, Python's slow and there's parallels and we can't do this and
whatever. And here you are choosing it for it. And it's working out well, I would imagine.
Yeah. I think, I think anybody that's, you know, sort of a Java developer out there is yelling at
me saying, how could that possibly be true? But, but it is, I mean, you know, we're, we can benchmark
some of the stuff that we've got in Java versus some of the stuff that we have running now.
And we're definitely running a lot faster, a lot leaner. Our memory usage is, is a lot lower.
So we're able to handle a lot more, a lot more connections and a lot more request response cycles
a lot faster, which is, which is the most important thing at the end of the day.
Yeah, absolutely. You know, it's so interesting. It's just, there are so many aspects to performance
in Python. And I certainly want to go down that rabbit hole, but you've got like the concurrency,
you've got like C and Cython implementations. You've just got all these things that, that are
really interesting. And then you throw in the web app where web apps are mostly talking to external
systems and are not themselves computationally heavy, but they're more orchestration in a sense,
right? So there's just so many, many cool things in the, it sounds like you've got a cool
architecture going there.
What I was just going to say is on that point is, is it's, you know, to get that performance,
it's sort of figuring out what are the right tools that you need to be able to, to achieve
that. So we're using, we're using Sanic to sort of power our microservice architecture, but,
you know, we're also pushing stuff off to, a queue, the task you're using salary.
Uh, if you're familiar with that tool, it's, it's, it's basically, you know, we've got a server
sitting somewhere else and we're just pushing messages off to it and letting it sort of handle
all these background tasks. And, you know, instead of creating these big, large queries and running
them on Postgres, we're caching a lot of stuff inside of Redis. That is, so it's sort of like a top to
down, top to bottom, approach of, of where can we, where can we make things faster?
Right. And using the right architecture, like asynchronous queues, for example, can just
make all the difference. Yeah, absolutely. Yeah. So maybe this is a good, good place to start
talking about, some of the web frameworks and some of these ideas. And before, you know,
so Sanic is a web framework based on Python's async event loop, asyncio and the async and await
keywords and all that. And we'll get into all those details, but before we dig in, like not everybody
is using these, these newer features, right? Like asyncio only came out in three, four,
and then really, I think properly in three, five with async and await. So maybe, maybe just tell us
what is asyncio and why is it really good for web apps? Sure. Well, first I think, you know,
everybody should be maybe using it. Maybe that's too broad of a statement, but, I really, I really,
asyncio is a, is a great standard library module that was added to sort of be another answer to
how can Python achieve concurrency. So we have some of the other modules that exist, including,
uh, multi-processing and threading and stuff like that. And, and while this answers some of the same
questions, it does it in a sort of different way. Yeah. When you think of these other, all the
traditional ways we've got multi-processing, we've got threading, and then of course, you know, like
APIs built on top of those like execution pools and whatnot, but all of those involve some sort of
forking. But asyncio is inherently doesn't even necessarily use multiple threads or processes at
all. Right. Yeah. It's sort of a simpler methodology and cleaner methodology in that, you know, you're
still running everything on, on a single process, but you're basically just taking the idea of start
working on something in our case in a web framework, it's a request and you start working on it and
processing it. And you get to a point where you say, well, I need to go find some information
somewhere else. So I'm going to pause here. And while I'm pausing here, let's go see what else needs
to get done. And actually it's very much sort of like what we're, what I'm doing in my day-to-day job.
And, you know, we're trying to figure out, you know, you've got an employee and he's coming in for
a five hour shift and you want to figure out how do you keep your, your person employed during those
five hours without having just standing around doing nothing, wasting time and money is, you know,
there to do, to work. So let's figure out how to make your, your web workers produce the most you can
get for them. Yeah, absolutely. So, you know, the whole IO bit of asyncio, right? It's right there
in the name is what it's for, right? Like it doesn't help you for computational things at all,
right? You really only have one thread and if you start doing computational stuff, you know,
you're just cute. You're just doing them in order, right? Yeah. But for web apps, their main job is to
talk to a database, talk to other microservices, talk to a queue, and then they just wait, right? And so
it's the perfect time, perfect way to structure these web apps is to say all the time that you're
waiting on these external systems, we're going to go do other work, right? Other requests. And that's
basically the idea behind Santa, right? Absolutely. It's using async. It's not like a magic bolt.
It's going to make things run faster. It's just, it's going to make things run more efficient. I think
that's a better way to put it. Yeah. It's not going to be just idly waiting around. It's going to be
working on something else. It's a really good multitasker, I guess. But I'll go back to this. All right.
So what, so Sanic is one of these async Python web frameworks, but there are others. Like,
what are some of the other ones that people might compare to if they're, if they're like shopping for
this type of framework? The ones that first come to mind are AIO HTTP. That's another big one.
Jepronto, you mentioned them. I think they've sort of stopped working on developing on that project.
I don't know if that one's still with the status of that or not. Yeah. I remember coming out around the
Sanic. I feel like it did stall out. I'll have a look. It's a really interesting project. And I think
what they did with it, it's a great approach. And it's a little bit different than Sanic's approach,
because I believe Jepronto is written a lot in C, which makes it extremely fast, but it also makes it a
little bit more difficult for, I think, the everyday Python programmer to pick up and look under the hood
and make some changes or where they might need to. Yeah. And looking at the repo, it's 50% C,
50% Python, and it had a ton of contributions and work on it back up till April 2017. Then you're
right. It kind of just went on pause.
This portion of Talk Python to me is brought to you by us. Have you heard that Python is not good for
concurrent programming problems? Whoever told you that is living in the past because it's prime time
for Python's asynchronous features. With the widespread adoption of async methods and the async and await
keywords, Python's ecosystem has a ton of new and exciting frameworks based on async and await.
That's why we created a course for anyone who wants to learn all of Python's async capabilities,
async techniques and examples in Python. Just visit talkpython.fm/async and watch the intro video to see if this course is for you.
It's only $49 and you own it forever. No subscriptions. And there are discounts for teams as well.
Any others that people might consider?
Court is another one. I think you've talked with the guy that built Court on your show before.
I have. Yeah. Court's really nice as well.
And I've actually been in contact with him a little bit. He's given us some ideas on where we can add into some stuff.
ASGI to send. We could talk a little bit about that later.
But he's been a pretty helpful resource. And I think the one more that I want to mention that's out there.
It's I think the newest on the scene is called Vibora.
Vibora. Yes. Vibora looks very interesting as well. Although I haven't done anything with Vibora.
Yeah. I haven't tried it out to see what it is. It does a little bit more than what I would personally like from a web framework, from a micro framework.
I like to handle a lot of things like caching and templating and stuff like that myself.
And it comes with a lot of that. So for a lot of people, that might be a might be a plus.
So it's another another one to definitely put out there.
Yeah, it's interesting. I tried to play with it a little bit. And for some reason, it kept crashing.
Not in the sense like it said you're doing something wrong, but it would just like the process would die.
And I was a little underwhelmed with it, I guess.
But definitely looks like it has a lot of promise. It maybe just needs a little polish.
So let's get back to Sanic, which to me seems like maybe along with Cort, one of these async modern Python web frameworks that you really can use and depend upon.
You know, like I mentioned earlier, I'm using it now in production. It's definitely production worthy and operational and it's pretty reliable.
I mean, itself is still a pretty new framework, but I mean, anything that's going to be in this field is going to be fairly new.
Because like you said, in sync IO is hasn't been a part of the standard library since version 3.4.
So yeah, and it's really 3.5 where it actually became, you know, has the proper keywords to make it easy.
Yeah, exactly.
Easy to program.
So maybe just give us the quick flyover of Sanic.
Sure. It's probably probably good to mention that is Sanic does use those those keywords.
So it's place. It's Python 3.5 and up only.
I know there are some asyncio tools that you can still use 3.4 with, but you're not gonna be able to do that with Sanic.
So 3.5.
Yeah.
I think most things have 3.5 if they have 3.5 at all, all these days.
How do you feel about this Python 2, Python 3 thing? Do you feel like we've kind of, we're downhill headed towards 3.3 and beyond?
Yeah, it's done. It's over. Game set match.
Yeah, I kind of feel the same way. So yeah, this framework here, for example, like you have to have not just 3, but 3.5.
So cool. So it depends on that. It's a lot like Flask, right?
It is. And I think the original idea behind it, and I'm not the guy that developed it.
The person that developed it is a guy named Michael Hill, and he originally created the API, and it looks very much like Flask, where you've got decorators and these blueprints so that you can sort of nest your functionality.
And it looks a lot like Flask, but obviously being asyncio, it operates differently.
Yeah, absolutely. So for example, all of Vue methods can be async def instead of just def, right?
And you can use a weight in them and so on. Do you have to use, does everything have to be async? Or can you do like blocking work in there as well?
Well, you could, for example, create a function that is, you know, a regular def and doesn't have to be async.
It certainly would be better too, because that's where you're going to be able to sort of see this performance benefits.
Because by definition, just about everything that we do in Python is going to be some blocking involved.
So you can't avoid blocking code at all. But by using a tool like asyncio, you're giving yourself an opportunity to alleviate.
Yeah, exactly. So I'm guessing, what I was getting at is like, well, what if I'm using some library, and it doesn't support asyncio?
Let's say, let's say Mongo Engine, for example, or something like that, I could still make those calls in there would just be taking less advantage of its async potential, right?
Exactly, exactly. So, so if you're going to use an ORM, say SQLAlchemy or PeeWee or something like that, you could still use it and it'll still function.
But what's going to happen is when you make that call, and it goes to make that network call to your server, it's going to block up your whole process.
And you're not going to be able to start handling a new request during that timeframe.
Now, there are ways that you can get around that and use some of these other tools that are async enabled.
Right. So the primary thing that you would do, the first thing that you might do is like, say, I'm using library X to talk to, let's say, Redis.
It would be to go out and say, is there a asyncio enabled variant or extension or something to that library, right?
Absolutely.
Maybe talk us through, yeah, talk us through that.
Sure. I think one of the first things that I do when I need to do to answer that question is, are you familiar with these awesome lists that are, you know, kind of circle around GitHub?
Yes. Awesome Python and whatnot, things like that.
Exactly. I like to go take a look at those.
There is one for asyncio stuff and it lists, you know, a number of different ORMs and stuff like that that are on there.
I do a lot of work in Redis and there's two different packages in Redis that async enabled Redis that I like to use.
One of them is called a Redis, where if you did pip install Redis, you get a very popular library that's out there to interact with Redis.
And this pretty much mirrors the exact same API as that.
You just put a Redis instead of Redis.
And the second one is called a IO Redis, which also has an async enabled API to be able to access your Redis layer.
Yeah. So I think many of the different packages people are out there using, there is some variant or some alternative that they can switch to.
But I just bring it up because it's not like, you know, having an asynchronous method, view method is going to just magically make your SQLAlchemy call or your Mongo Engine call or whatever.
Just start being concurrent, right?
You're going to have to go and get a library that allows you to write, await this query, await that insert, et cetera.
Yeah. And I think if you're going to get started in doing the work in a framework like Sanic or one of these other async enabled, one of the first things I think you should do is figure out how you're going to talk to your data layer and figure out which one of these tools works best for you.
There's a number of them and they're growing every day.
And I feel like I continually get to see new ones.
And I think it's an exciting time for Python.
I think it absolutely is.
I think the async and await stuff is really unlocking a lot of the potential.
I'm especially excited for Sanic and some of the similar projects because while we've had these cool keywords in this capability, it's not until you can actually create asynchronous web methods that you can actually really break that thing loose and use it where it makes a lot of sense.
Yeah, absolutely.
So, yeah, it's an exciting time for sure.
So what are some of the ways that we might get around these dependencies?
So step one, go and find another library that does the similar or same thing that you're doing but has coroutines and async methods and whatnot.
That's option one.
If for some reason that fails, is there a way it can use threading or multiprocessing or some kind of background queue or something?
Is there something else I can do to not block up my web async I.O. loop and still work with them?
The question is, could potentially get really messy?
If you're going to take and start a new, say, a new thread inside of your loop.
Could be done.
That's a different question.
Yeah.
I think that's probably...
So you could have a background thread that's always running for each worker process and then a queue that you drop something off.
An in-memory queue.
You drop something off and it picks it up.
Like, say, subscribe this person to a mailing list.
For some reason, the API that does that doesn't support async I.O.
And I think for the most part that would work, but it's definitely not super good, is it?
If the goal is to do something like that, I would highly suggest upgrading to Python 3.7 and then you get to use async I.O. queues, which is a fairly new feature, which basically you could just start pushing off tasks to this async I.O. queue and let them handle it.
And I think the great use case for that is I take a request, I'm processing it, I need to send out an email, push that off to this queue and be done with it.
A little bit more involve method, like you said, you could do something with threading or use another package like celery, which is something that I've had good success using for a number of years.
So I'm still using it inside my async I.O. stack because it really solves a good problem for me.
Yeah, it does.
So we've got our web code, we're writing our async methods, we're awaiting all these calls, everything's working pretty well.
Beyond that, it looks quite a bit like Flask, which I think is really a good thing.
There's a couple of things I'd like to maybe call out that are either less well known about Flask and is relevant here as well, or this enables.
And one of them is blueprints.
So typically when people say, I'm going to go create a Flask app, they all, they say, well, what you do is you create the app.py and you.
You allocate the app and then you use app, you know, as a decorator app.route and you put it on these methods.
Boom, you're done with Flask, right?
And that's cool for a demo.
But if you have hundreds of view methods broken up by categories of like, here's the store views and here's the user management views and here's the reporting.
And you don't want to cram that all into one file.
And so Flask and Sanic both have this idea of blueprints to help you partition your projects into different pieces.
Do you want to talk about that?
Sure.
I think there's a big difference between, you know, looking at the tutorial, it's got 10 lines and here's how to get started.
Wow.
It's amazing.
You know, hello world.
It's such an easy thing.
But then you get to the question of how do I actually build something?
Like, how is this usable?
How does it not become a giant mess of one huge app.py file, right?
Yeah.
Maybe you like one giant file.
Maybe that's your...
But, you know...
But if you don't, then it becomes a bit of a challenge of juggling that app.route stuff around.
Exactly.
And I think it's, you know, something to do is, okay, you've decided you're going to do async.
You've decided on your data access layer and, you know, which one of these tools you're going to use.
And then the next question, I think, is how am I going to organize this?
Am I going to have one Sanic server that's going to handle all my requests?
Or maybe I need to have 10 different services and they're all going to be connected and they're all going to be responding to different things.
And within each one of those things, maybe we need to have one little unique entity called a blueprint that's going to handle certain information.
And maybe you need to have, you know, 10 of these.
So, for example, maybe you're going to have authentication be one blueprint.
And maybe you're going to have users be another blueprint.
The idea is just to organize and structure your code.
One thing that Sanic also has is what's called a blueprint group, which does exactly that.
It just takes a bunch of blueprints and it groups them together.
And one of the big benefits here is for routing purposes.
Yeah, that's cool.
So, basically, instead of saying app.route, you say blueprint.route.
And you kind of reverse it.
In your main startup file, you import all the blueprints and you register them with the app and say, here's a whole bunch of subparts of my web framework and their routes and their view methods.
Put them all together here and then run it.
The group sounds pretty interesting.
I didn't know about that.
Yeah.
And one of the big other benefits you get with blueprints is you don't have to be passing around your app object all over the place.
So, if you're going to have...
Yeah, that gets really tricky.
Yeah.
Exactly.
So, it gives you a way to sort of register these views to something.
And then when you finally have your app and you're creating your app, they all get added on at that point.
So, you don't need to create these crazy global variables or anything like that.
Yeah.
I think it's a really nice way to break apart different parts of Flask or Static apps into little sections that are more self-contained.
And then you can import them into the main app.
It's sort of a personal preference of mine.
I don't know that I necessarily say this is the way you have to do it.
But I like to have each one of my different endpoints sort of be like its own blueprint.
And at max have two or three different methods that are attached to it.
I like to try to keep things as organized as possible.
And mainly that's for development purposes.
It's when I've got an issue or I need to make an update or fix a bug or something like that.
It's a lot easier for me to navigate when I've got everything very finely grained, broken up into different files.
Yeah, that's the same way I do it.
And I really like it.
You know, if you're in this file, this is the type of stuff you're working on and just that.
So, another thing that Static makes pretty easy is WebSockets.
And that's been traditionally a little tricky in regular Python code, right?
Because it'll just block up.
How are you just going to wait on this bidirectional communication thing, right?
Yeah, I think one of the great benefits that we get from AsyncIO, besides being able to have, you know, these non-blocking access to other services and stuff like that, is to be able to do stuff like WebSockets.
And, you know, WebSockets are fun.
Yeah.
It's great when you're building, you know, a front-end application and you can have that bidirectional control between server and client.
And the way that it's achieved right now inside of Sanic is by using another package that's called WebSockets.
So, you could just do pip install WebSockets.
And that gets you to the same bit of code that's running inside of Sanic.
And Sanic is just basically putting on another routing layer on top of that to be able to get to your socket.
Yeah, it's really cool.
Really cool.
Say that now, in coming next year, there may be some ideas on how we change that and make that a little bit better.
And being able to handle different deployment strategies.
And by that, I mean using ASGI, which is sort of the asynchronous response to WSGI.
Right.
WSGI is the web service gateway interface, which is how most things like Flask, Pyramid, Django, etc., plug into larger web servers like MicroWSGI or G Unicorn or something, right?
But the fundamental problem with that API is it's a single function call per request that has no asynchronous concepts around it, right?
Exactly.
So, you're not really going to be able to run an asynchronous application off of WSGI.
And this is a problem that the guys building Django channels, I believe, were the first ones that came up with the ASGI implementation or specification, whatever you want to call it.
We're looking into it to adopting that, which will open up a greater deployment strategy for Sanic.
Yeah, that would be really nice if there was the WSGI equivalent for all these different frameworks.
That's really one of the key missing things to unlock a lot of it.
But since you brought it up, maybe we should talk a little bit about deployment on this.
So, what is different or how does deployment for regular web servers and web apps change if you're deploying something based on Sanic versus something on Flask?
The biggest change is you could just do Python app.py and just run your application straight from inside Sanic.
And Sanic has its own way to be able to operate as a web server and to interact.
And so, you could just do that and you don't have to do anything more.
There is some other deployment strategies where you could use G Unicorn and there's a worker layer that sort of enables G Unicorn to be able to run Sanic.
So, we've got that in the documentation.
So, you could do that.
I think the best way to do it, and my personally preferred way right now, is to just allow Sanic to handle its own handle itself and create Docker containers.
And the Docker container's job is to run Python.py, Python app.py.
And then stick in front of that something like Nginx or another load balancer that handles the incoming request and just kind of spits out my traffic to one of my services.
Yeah, that makes a lot of sense.
You don't have to do it that way.
Yeah, sure.
Would you set it up as like a system daemon type thing, system control, and like a service unit to run the Sanic part?
Or do you just do that?
Like, if you're using Docker containers, maybe it's just you start the container and it does its thing.
Yeah, you could do that.
And this is, again, this is my own personal preference bleeding in here.
But Docker is such a widely popular tool now.
It makes it so easy to deploy and to take something from development on my machine and to stick it up on a staging server and then eventually stick it into production.
I find Docker to be a very useful tool for that.
And, you know, one step further is to use Kubernetes to sort of manage that whole process.
But that may be a little bit too complex for a lot of people.
And I don't necessarily want to say that that's the way that you have to do it.
Right.
And regardless of whether you're using Docker or you're just running it in a separate process, you still probably want something like Nginx, which is going to handle SSL.
It's going to handle delegating back to that server.
It's going to handle, would you serve the static files through there still through Nginx?
Yeah, absolutely.
I think that's another good point to make.
And while you certainly could handle static files through Sanic, and Sanic does give you an API to be able to do that.
Your best bet most of the time is going to be just let Nginx handle your static files.
You're going to get the most performance by just serving those directly.
You don't really need to let Python handle static stuff.
You don't let that handle all your dynamic content.
Yeah, that makes sense.
That's the way I run my stuff as well.
Let's talk about testing.
So testing regular web apps is pretty straightforward.
I mean, it still requires some tricks to know how to like create a dummy request and things like that.
But then you just call it, and then you get the answer.
And then you assert things about the answer.
But with Sanic, what does testing look like?
Because typically testing async I/O stuff means there has to be an event loop.
It has to be running.
You've got to wait for it to be done.
There's like extra stuff.
What does testing one of these web apps look like?
Yeah, it's certainly not an easy thing to do.
And I think this is sort of one of those other areas where maybe you've checked off all those boxes and you've kind of are now onto the stage of how do I start testing this stuff?
And I would say don't try to invent the wheel yourself.
Go see what else is already out there.
And personally, I like to use pytest.
There's a great plugin for Sanic called pytest Sanic, which solves a lot of these problems and makes it pretty similar to what you're used to doing with pytest, where you're creating your application.
And it's a little bit different because you've got your async and await keywords in there, but it's going to handle creating the loop for you.
And so you can just kind of go about your process the way you would expect.
Yeah, it's pretty interesting.
Some of the stuff that you do.
So like your test method might be an async method, right?
Async def test something, not just a regular test method, but the pytest Sanic knows how to run it and it's all good, right?
Yeah.
And there's also an asyncio pytest.
I forget the name of the package that does sort of the same thing, just not in the confines of Sanic itself.
But that's also definitely something to look out for.
So the answer is, yeah, it's definitely possible.
There's a little bit more complexity to it, but take a look to see what other people are doing.
And I think it's a piece of the puzzle that's been solved.
So may as well go take a look to see how other people have done it.
And especially since there's a pytest plugin, it's pretty easy.
So that was one of maybe, I don't know if you call it an extension exactly, but it's one of the things that is sort of in the category of extensions.
But there's actually a whole bunch of different extensions, right?
So that's kind of a lot of different cores and all that.
Do you want to give us a quick flyover of some of the more important extensions?
Sure.
I think probably the most important extension that's out there is the pytest Sanic extension because it gives you the extension ability.
Another one that is hugely popular is Sanic OpenAPI, which adds, I don't know if you're familiar with OpenAPI or not, but it's a specification for how do you define your API endpoints.
And with this plugin, you basically just add some more decorators around your methods and it's going to create the output for you.
That's going to meet your specification of what kind of methods does it accept and what kind of intake variables might need to get passed into the method.
And eventually, what's the output?
What does your endpoint give you?
Yeah, that's really cool.
So it defines the API and gives you some Swagger UI back as well so you can actually see what methods are available and things like that, right?
Exactly.
Another one, and this one's going to be a little bit of a self-plug because it's my extension, is an authentication Sanic JWT, which adds web tokens.
And if you do pip install Sanic JWT, it gives you a blueprint that has all the authentication endpoints that you would need to log in, to create a token, to verify a token, to pass refresh tokens and all that kind of fun stuff.
That's a good one.
I see that it has Jinja 2 as one of the extensions.
What is the default template engine in Sanic?
Well, there isn't one, and this is sort of where it's a micro framework where it's not going to make a lot of the decisions for you.
And so you can kind of create your own stack.
And personally, I think while you can serve static files, I think sort of the place where frameworks like Sanic really shine is using them to power web APIs and web sockets and stuff like that.
Personally, I don't use it to handle HTML well.
It certainly is capable of providing HTML responses.
Sanic is going to say, let's you figure out what's the best way that you want to be able to template those.
And if it's Jinja 2, there's a plugin that you can use for that.
Yeah, that makes sense.
Are there any other template languages that you can use?
You can use Django ones, you can use Chameleon, Mako?
I don't know how it would work with Django ones.
Anything that's going to work in Python, you can use that.
Because at the end of the day, what you need to do to respond with Sanic is you just need to be able to pass it a string.
So use whatever template engine you want.
I see.
So you would basically just create the thing and tell it to here's your template file, here's your model data, render it, and just have Sanic return the HTML.
So you kind of plug in that layer.
Exactly.
So you could use whatever you needed to there.
It would work.
Yeah, it seems like one of the first things you'd want to do is create your own little decorator that takes a template file and then just automatically does that.
You just wrap up your view methods, right?
Yeah, don't do that more than twice.
Creating decorators is a huge thing, I think, in frameworks like Sanic and Flask and stuff like that.
And there is some information in tutorials on how you would go about that when you realize that you're going to need something like that.
Right.
Yeah, very, very cool.
So one of the things that happened not too long ago, maybe a year ago, is Sanic moved from being this framework created by, you said Michael Hill?
Yes.
Yeah, created by Michael Hill and has now become more of a community project.
Do you want to talk about that whole process, what that means for Sanic and why it was done?
Sure.
So I kind of got involved with Sanic.
First, I got it mainly because it was something that I wanted to just check out.
You know, one of the things I like to do in my spare time is see what else is out there that I don't know and try playing with it.
And Sanic was kind of how I started playing around with these async enabled frameworks.
I sort of realized that there was a missing hole.
And this is before I was involved as one of the maintainers of the Sanic project.
I saw where there was a hole for authentication.
So I created the JWT extension.
And then when I started using it for work, I sat down and I said to myself, OK, this is a pretty new framework.
You know, one of the risks with something that's new is how long is it going to last and how long is people going to be maintaining it?
That is really one of the concerns with any of the one of the frameworks that you mentioned at the beginning is they're also new, but web apps are often long lived, right?
You don't want to build your thing on, you know, if you'd built it on Jepranta, you'd be really bummed that contributions stopped, you know, a year ago.
Yeah, absolutely.
When I go to GitHub to find some package, that's one of the first things I do is look to see how active it's been.
Because if I'm going to integrate this into something that I'm putting into production, if it stops getting maintained, then I need to step up to the plate and start maintaining it if something breaks.
Yeah.
And do you want to do that, right?
You don't want to do that for everything out there.
Sometimes you just want to use something.
Exactly.
And when I was making that decision with Sanic, I said to myself, okay, you know, what happens if these guys stop maintaining it?
And I looked at it and I said, okay, you know, I mean, if that happens, then, you know, I realize this is a risk and maybe I'll get myself, maybe then I'll just take it out of my own wing and just fork it and do what I need to with it.
And lo and behold, you know, some time goes by and I start noticing that all these issues are piling up and there's pull requests that are piling up and, you know, the release cycle is a little bit slower than it had been at first.
And I started talking with some other people that are, that I know that are users of Sanic and I noticed that they also had the same concerns.
And so we're saying, okay, well, maybe this is a long-lived project or not.
And so I started talking with these people and I started talking with the guys that were the maintainers of the project and everybody that had used the package and had worked on it, you know, really liked it and was enjoying doing it and nobody wanted to see it just go away.
So we said, okay, well, then let's take this package that was on GitHub as a repository for just a single user.
Let's turn it into a community package, a community organization and invite whoever wants to be involved.
Let's get everybody together and start working on this together.
Let's make decisions together.
Let's not have just one person hold the key to this.
And, you know, we're still in this process of figuring out what that means to be a community run organization.
And it's not like we have this answered yet, but, you know, we've been doing this for, I guess, about, I don't know, five or six months now.
And it's sort of been a fun ride so far.
And I think we're headed in a good direction.
It looks like a good move in it.
You definitely have some folks involved there.
So I think it's very positive.
I also just checked out Vibora and it looks like it's stalled out as well.
So it's looking more and more like Sanic is the one that still has the momentum of the ones that we've been talking about.
I don't want to be the one to make that claim, but.
No, you let me say it.
But, I mean, I think Hort is still doing really well and AIoHDP as well.
But like some of the other ones that came out at the same time, I kind of feel like, I don't know, they just, they had a lot of activity, but not so much.
So it's good that you're taking these steps to try to solidify that base and, you know, not make it just dependent upon one person's interest in pursuing this sort of tweak of the Flask API, right?
Yeah, absolutely.
And that's, we've sort of tried to make sure that there's some people that are involved, that have been involved in the project for, since it's beginning early days, and some people that are just getting involved now.
And we try to have a mix of as many voices as we can.
So we have consistency and we're not breaking any backwards compatibility issues.
But I'm pretty excited because there definitely has been a good response and been a lot of community action in pushing this project.
Yeah, that's great.
So what are some of the plans going forward?
You guys decided to issue the zero versioning style and go with something different for versioning, right?
The current version is 0.8.3.
And that's going to be the last zero versioned release.
And coming up in about another month, we're going to switch over to calendar versioning.
And the idea behind this is we've got a fast framework.
Let's make our development cycles, you know, somewhat fast as well.
So we can respond to things quickly.
And so we decided to go with four releases per year.
We're going to do one in March, one in June, September and December with the idea that the December release will be a long-term support release, an LTS release.
That's pretty cool.
I think it makes things more clear because a lot of folks have been involved in open source for a long time.
They see 0.12 and they're like, oh, great, that's fine.
But many people, especially coming from like the enterprise-y space and the commercial space, they see 0.something and they're like, oh, it's not ready yet.
It's in some kind of beta or something.
That's sort of what we're trying to achieve with moving to a community organization and moving to having an LTS release is to make this known that this is a safe project to rely upon.
And you can use this in production.
It's not going anywhere and it's going to perform.
That's cool.
What does a version number look like in calendar versioning?
Is it like 2018 dot or like, yeah, what are you doing there?
You could do that.
What we're going to do is it's going to be 18.12.
So that's going to be the next release.
And then after that, it'll be 19.3, 19.6, 19.9.
Yeah.
And then like little patches and stuff would be like dot something on the end.
Exactly.
So if we need to have inside that release cycle, then yeah, it'll be 18.12.1.
I imagine that those type of patches, you'll see more of those with the LTS releases because the idea behind that is we're going to lock in the API.
We're going to lock in the feature set in December of 2018.
And that's going to be on its own branch inside of GitHub.
And for the next year, if we see any security issues or bugs, we're going to go and we're going to fix those and we're going to push those out to the 18.12 package.
But while we're doing that, we may move on to additional feature sets inside 19.3 and moving forward.
Yeah.
That wouldn't necessarily get put into that 18.12.
That's really cool.
And you know, one thing I definitely like about this is I can open up a requirements file or I can do a pip list on a virtual environment or something and just look and know, oh, that library is a year and a half old.
Or that library is pretty new.
Like if I do, you know, a pip list and I see requests is that I don't even know what it's at.
Like, let's say I see pyramid at 1.8.7.
Right.
Like that's out of date.
It should be 1.9.
That's something.
But how out of date, right?
Like, was it just updated?
You need to know the package.
You need to be involved in the community and have an understanding of when all those releases are happening to know, you know, whether your version is out of date and by how much.
With your model, it's super clear.
Oh, that's six months old.
We should probably consider upgrading that.
Yeah.
I mean, I don't want to say it's our model.
Like it's a thing that's out there.
Sorry, the model.
Let me rephrase it.
The model you've adopted.
It lets me see that.
Yeah, that's really nice.
And I think pip is now on to that model as well.
And there's some other.
I've noticed a lot more repositories using calendar versioning, I'd say, in the last year than I had, you know, before that.
It's cool.
Well, I'm definitely a fan.
Now, one of the things I saw looking through the documentation is you have on the various issues and items in GitHub is you have help wanted labels.
And it sounds like you're looking for folks who would be willing to be contributors and participate, right?
That's sort of the goal is this is a community built project.
You know, let's get everybody involved that wants to be.
And one of the things that, you know, I'm not the only one doing this, but one of the things that I do, you know, when I check GitHub and I see that there are new issues open, I try to, you know, make an assessment, you know, of how difficult would it be to implement this feature?
Or, you know, is this a bug or whatever it is?
And use GitHub to provide the context.
If I have a project and I want to help out and say, you know, I don't really know, you know, where can my efforts go?
It's a little bit easier for me to see, okay, well, you know, this is an easy task for me to handle.
So let me try this one first, as opposed to trying to add an ASGI or something like that, which would be more difficult.
Yeah, the community is still working on that one as a whole.
That's cool.
So, but if people are keen to do it, there's all sorts of stuff I'm sure they could pick up, like even triaging issues, working on documentation tutorials.
There's being a relatively new web framework.
There's probably not as hard or solidified as say something like Django to make changes.
Absolutely.
And I think, you know, the best thing there is if you have interest in it, the best way to get involved is to just go into GitHub or we have, we have, we just opened up a community forum using Discourse.
And just look to see what else, what the conversations are and just start responding, you know, just put in some, some thoughts.
You know, there's a conversation about something and, you know, something that strikes your interest, you know, put in your own thoughts.
And it's sort of a good way to just get yourself involved in, in your face known.
So you don't even, you don't necessarily need to get involved by making a pull request and, you know, solving, you know, you know, bug issue number one, two, three, four.
It's, I think just making your presence known is, is a great way to contribute yourself in your, in your, in your thoughts.
Yeah, it's cool.
A lot of people are looking to get into their first open source project for one reason or another.
And I always like to highlight opportunities to do that.
So it sounds like this is a good one.
Anyone's looking to contribute, you know, we're definitely, you know, we'd love to have help with documentation and testing.
And I think one of the big things right now would be examples is, you know, I used Sanic to solve this problem and this is how I did it.
That kind of stuff.
Yeah.
Cool.
That sounds really great.
So I guess we're probably about out of time.
So we should leave it there for Sanic.
Let me ask you the two questions I always ask my guests at the end of the show.
If you're going to work on Sanic or any other Python code, what editor do you use?
I'm a sublime text guy.
I really like when I can open up my code editor and just see code and I don't like to have too many distractions.
So I like simplicity.
Yeah, that's cool.
And notable package on PyPI, not necessarily the most popular, but you're like, oh, you should know about this.
And I'll go and throw a pip install Sanic out there.
So you don't have to say that one, but maybe something else like maybe an extension or something or async ORM.
I'll leave, maybe I'll leave all that stuff for the listeners to go and explore and find, you know, maybe one of those awesome async lists or something.
Maybe someone wants to create an awesome Sanic list.
Maybe that's a good contribution.
That would be cool.
How about, but as another notable package, I've got another GUI that I don't think you guys have necessarily covered yet.
We've been on a real kick, but we haven't hit them all, I'm sure.
Well, how about pip install eel?
I think that's another one that you can, it's a fun little project, which is.
Eel's a good one.
The idea is somewhat similar to what Electron is doing, but doing it with Python instead.
Yeah.
So Electron JS allows you to take and create what looks like a desktop application, but it's actually, you know, a hidden Chrome running HTML defined front end.
Maybe it's something like Ember or Angular and then a Node JS backend for much of the logic.
Right.
And so eel does basically that, but for Python, right?
Exactly.
And one of the benefits besides giving you the entire Python ecosystem is it's a little bit less intensive than say Electron might be.
It's also a little simpler, isn't it?
If I remember looking through.
Yeah.
It's a fun, it's a fun little project.
Nice.
Yeah.
People should check that out.
All right.
Awesome.
So final call to action.
People are excited about Sanic and Async Web frameworks.
What would you say to them?
What I'd like to most stress is get yourself involved in open source work however you can.
I think especially for people that are maybe newer developers and just getting started out, open source is a great way to get experience and to get exposure and most importantly, start learning.
And so I'd say come say hi to us over in the Sanic community.
We'd love to have you.
But if not, go find another community that you're passionate about and make yourself known.
And this whole web framework, Async Web framework in Python, you think it's time?
It's time to come?
Don't quote me on this, but I believe eventually Django's planning on adding Async several release sites, a couple years, two years down the low.
And it's coming.
I mean, it's definitely the way to go.
So the future is now.
Yeah.
You spoke about Django channels and I think it was Andrew Godin that wrote this.
Right.
Put together a cool thing called a Django Async roadmap that lays out their plans for basically doing something similar.
It's a really great article.
I'll throw it in the show notes for people who are on the Django side of the fence.
But yeah, definitely.
I agree with you.
And thanks for being on the show, Adam.
It was great to talk with you.
And best of luck on Sanic.
It's a cool project.
Thank you very much.
You bet.
Bye.
Take care.
This has been another episode of Talk Python to Me.
The guest on this episode was Adam Hopkins.
And it's been brought to you by us over at Talk Python Training.
Want to level up your Python?
If you're just getting started, try my Python Jumpstart by Building 10 Apps course.
Or if you're looking for something more advanced, check out our new Async course that digs into all the different types of Async programming you can do in Python.
And of course, if you're interested in more than one of these, be sure to check out our Everything Bundle.
It's like a subscription that never expires.
Be sure to subscribe to the show.
Open your favorite podcatcher and search for Python.
We should be right at the top.
You can also find the iTunes feed at /itunes, the Google Play feed at /play, and the direct RSS feed at /rss on talkpython.fm.
This is your host, Michael Kennedy.
Thanks so much for listening.
I really appreciate it.
Now get out there and write some Python code.
Thank you.