WEBVTT

00:00:00.020 --> 00:00:03.280
<v Michael Kennedy>For years, Django and Async came with an asterisk.

00:00:04.000 --> 00:00:06.160
<v Michael Kennedy>The docs themselves warned you off it.

00:00:06.580 --> 00:00:07.580
<v Michael Kennedy>Scary performance notes.

00:00:08.180 --> 00:00:09.800
<v Michael Kennedy>A story that felt half-finished.

00:00:10.120 --> 00:00:12.560
<v Michael Kennedy>Well, that story just got rewritten, literally.

00:00:12.860 --> 00:00:16.540
<v Michael Kennedy>And the person who rewrote it is here to tell you why that old framing was wrong.

00:00:17.140 --> 00:00:21.860
<v Michael Kennedy>Carlton Gibson is a former Django fellow, sat on the security team for eight years,

00:00:22.180 --> 00:00:23.300
<v Michael Kennedy>and he's on the steering council.

00:00:24.080 --> 00:00:26.960
<v Michael Kennedy>On this episode, we get into the Async topic doc rewrite.

00:00:27.020 --> 00:00:33.000
<v Michael Kennedy>what actually remains versus what was just fear and the new tasks framework in 6.0,

00:00:33.560 --> 00:00:36.140
<v Michael Kennedy>DB-level cascades and fetch models landing in 6.1,

00:00:36.340 --> 00:00:40.160
<v Michael Kennedy>and why free threading is the bet it's about to pay off big for Django.

00:00:41.260 --> 00:00:43.960
<v Michael Kennedy>If you've been told that Django's async story isn't ready,

00:00:44.580 --> 00:00:46.160
<v Michael Kennedy>this episode puts that myth to bed.

00:00:46.760 --> 00:00:51.840
<v Michael Kennedy>This is Talk Python To Me, episode 556, recorded July 2nd, 2026.

00:01:09.720 --> 00:01:14.040
<v Michael Kennedy>Welcome to Talk Python To Me, the number one Python podcast for developers and data scientists.

00:01:14.590 --> 00:01:19.880
<v Michael Kennedy>This is your host, Michael Kennedy. I'm a PSF fellow who's been coding for over 25 years.

00:01:20.540 --> 00:01:21.580
<v Michael Kennedy>Let's connect on social media.

00:01:22.020 --> 00:01:25.040
<v Michael Kennedy>You'll find me and Talk Python on Mastodon, Bluesky, and X.

00:01:25.400 --> 00:01:27.220
<v Michael Kennedy>The social links are all in your show notes.

00:01:27.980 --> 00:01:31.500
<v Michael Kennedy>You can find over 10 years of past episodes at talkpython.fm.

00:01:31.680 --> 00:01:34.880
<v Michael Kennedy>And if you want to be part of the show, you can join our recording live streams.

00:01:35.260 --> 00:01:35.700
<v Michael Kennedy>That's right.

00:01:35.960 --> 00:01:39.160
<v Michael Kennedy>We live stream the raw uncut version of each episode on YouTube.

00:01:39.760 --> 00:01:44.180
<v Michael Kennedy>Just visit talkpython.fm/youtube to see the schedule of upcoming events.

00:01:44.370 --> 00:01:48.060
<v Michael Kennedy>Be sure to subscribe there and press the bell so you'll get notified anytime we're recording.

00:01:48.760 --> 00:01:50.420
<v Michael Kennedy>This episode is brought to you by Sentry.

00:01:50.640 --> 00:01:54.880
<v Michael Kennedy>Don't let those errors go unnoticed. Use Sentry like we do here at Talk Python. Sign up at

00:01:55.040 --> 00:02:01.020
<v Michael Kennedy>Talk Python.fm slash Sentry. Carlton, welcome back to Talk Python To Me. Amazing to have you here.

00:02:01.360 --> 00:02:02.840
<v Michael Kennedy>Thank you, Michael. Thank you for having me on again.

00:02:03.320 --> 00:02:08.679
<v Michael Kennedy>Always fun to talk with you. Fun to have you back. We're going to talk about one of my favorite

00:02:08.840 --> 00:02:15.260
<v Michael Kennedy>topics, async, both for the reasons it's amazing and the reasons it's like, why? Why is it this way?

00:02:15.740 --> 00:02:18.320
<v Carlton Gibson>Why do we do this to ourselves is the thing, right?

00:02:18.960 --> 00:02:19.220
<v Michael Kennedy>Exactly.

00:02:20.480 --> 00:02:22.620
<v Michael Kennedy>Wait, I want to say, oh, but I didn't want it like that.

00:02:22.740 --> 00:02:23.280
<v Michael Kennedy>No, just kidding.

00:02:24.440 --> 00:02:29.000
<v Michael Kennedy>Specifically with regard to Django, you did some really interesting updates with Django

00:02:29.180 --> 00:02:30.160
<v Michael Kennedy>that we're going to talk about.

00:02:30.460 --> 00:02:33.360
<v Michael Kennedy>But first, your teams, they're still in the World Cup.

00:02:33.460 --> 00:02:33.920
<v Michael Kennedy>How are we doing?

00:02:34.220 --> 00:02:34.720
<v Carlton Gibson>Oh, yeah.

00:02:34.760 --> 00:02:37.280
<v Carlton Gibson>Well, so I'm British by birth, so English by birth.

00:02:37.440 --> 00:02:39.860
<v Carlton Gibson>So I was supposed to support England, but I live in Spain.

00:02:40.080 --> 00:02:43.900
<v Carlton Gibson>So depending on who's winning, who's doing better, I get to flip my allegiance.

00:02:44.900 --> 00:02:50.160
<v Michael Kennedy>you know i think that's really that's really amazing because you if one of your team's not

00:02:50.220 --> 00:02:54.820
<v Michael Kennedy>doing well you can just like i'm gonna just cheer for the other one it's it's great you get like two

00:02:54.980 --> 00:02:59.340
<v Carlton Gibson>shots at the thing yeah i mean in the last uh european championships we had the england spain

00:02:59.520 --> 00:03:06.400
<v Carlton Gibson>final so i couldn't lose really yeah that's amazing yeah so the u.s team um advanced yesterday as well

00:03:06.430 --> 00:03:10.979
<v Michael Kennedy>they played really well so i'm very very excited for that yeah it's lovely to see new new nation

00:03:11.000 --> 00:03:12.720
<v Michael Kennedy>emerging nations do well in the World Cup.

00:03:12.980 --> 00:03:15.300
<v Michael Kennedy>You know, a long time ago, that would have been like some kind of joke.

00:03:15.410 --> 00:03:17.280
<v Michael Kennedy>People are like, ah, surely you're joking.

00:03:17.780 --> 00:03:18.980
<v Michael Kennedy>Surely you're joking about this.

00:03:19.120 --> 00:03:24.700
<v Michael Kennedy>But honestly, soccer or football, non-American football, is popular in America.

00:03:25.000 --> 00:03:25.740
<v Michael Kennedy>So super cool.

00:03:26.140 --> 00:03:27.020
<v Michael Kennedy>Glad to hear it.

00:03:27.240 --> 00:03:28.680
<v Michael Kennedy>And you're back.

00:03:29.260 --> 00:03:32.040
<v Michael Kennedy>So tell people just what have you been up to?

00:03:32.440 --> 00:03:34.360
<v Michael Kennedy>You know, not everyone listens every episode.

00:03:34.600 --> 00:03:35.960
<v Michael Kennedy>So a quick introduction for yourself, maybe.

00:03:36.240 --> 00:03:36.440
<v Carlton Gibson>Okay.

00:03:36.510 --> 00:03:37.680
<v Carlton Gibson>So my name is Carlton.

00:03:38.300 --> 00:03:39.780
<v Carlton Gibson>I've long time worked with Django.

00:03:40.360 --> 00:03:42.760
<v Carlton Gibson>I'm a former Django fellow, did that for five years.

00:03:42.960 --> 00:03:44.440
<v Carlton Gibson>I'm on the Django steering council.

00:03:44.570 --> 00:03:46.260
<v Carlton Gibson>I was on the security team for eight years.

00:03:46.370 --> 00:03:48.480
<v Carlton Gibson>I maintain a whole load of packages in the ecosystem.

00:03:48.740 --> 00:03:50.540
<v Carlton Gibson>So, you know, I'm quite vested there.

00:03:51.420 --> 00:04:00.040
<v Carlton Gibson>And, yes, you know, basically I use Django and have done for, I don't know, however many years to build web applications to pay the bills.

00:04:01.240 --> 00:04:02.100
<v Carlton Gibson>Had a busy spring.

00:04:02.610 --> 00:04:04.360
<v Carlton Gibson>I was everywhere I went to.

00:04:04.670 --> 00:04:09.060
<v Carlton Gibson>There was a Pi TV event organized by JetBrains in Amsterdam in March that I went to.

00:04:09.620 --> 00:04:10.540
<v Carlton Gibson>That was all on YouTube.

00:04:10.810 --> 00:04:12.960
<v Carlton Gibson>There was, I don't know, 11 hours of live streaming or something.

00:04:13.080 --> 00:04:13.840
<v Carlton Gibson>It was amazing.

00:04:14.700 --> 00:04:18.859
<v Carlton Gibson>Then I was in DjangoCon Europe in Athens in, I think, May.

00:04:19.350 --> 00:04:19.959
<v Carlton Gibson>No, April.

00:04:20.340 --> 00:04:23.260
<v Carlton Gibson>And then in May, I was at Paikon Italia, which is in Bologna.

00:04:23.720 --> 00:04:25.980
<v Carlton Gibson>Paikon Italia really is a lovely conference.

00:04:26.130 --> 00:04:27.680
<v Carlton Gibson>You should make your way there.

00:04:27.840 --> 00:04:30.480
<v Carlton Gibson>And now I'm just home and I'm exhausted.

00:04:30.600 --> 00:04:33.860
<v Carlton Gibson>I had a quick holiday with the family, but now I'm hiding at home until September,

00:04:34.110 --> 00:04:36.820
<v Carlton Gibson>where we've got Django on the Med, which is a three-day sprint event,

00:04:37.080 --> 00:04:40.040
<v Carlton Gibson>which is going to be in Pescara, Italy, which we're looking forward to.

00:04:40.340 --> 00:04:43.000
<v Michael Kennedy>That sounds really, really nice.

00:04:43.160 --> 00:04:47.600
<v Michael Kennedy>I love traveling, but the jet lag kind of blows me out eventually.

00:04:47.800 --> 00:04:50.200
<v Michael Kennedy>I'm like, ah, this is 11 hours jet lag.

00:04:50.240 --> 00:04:51.620
<v Michael Kennedy>I don't really want to do this anymore.

00:04:52.040 --> 00:04:56.960
<v Michael Kennedy>It sounds to me like you're kind of able to keep it in the broader Euro neighborhood, which is nice.

00:04:57.580 --> 00:04:58.060
<v Carlton Gibson>Yes, I was.

00:04:58.240 --> 00:05:01.400
<v Carlton Gibson>I was, which it does help because there isn't that massive time change.

00:05:01.740 --> 00:05:03.220
<v Carlton Gibson>But it's still hard work, right?

00:05:03.840 --> 00:05:06.800
<v Carlton Gibson>Even though you have fun at these events, even though you're seeing your friends,

00:05:06.960 --> 00:05:09.400
<v Carlton Gibson>even though it's, you know, doing what you're vested in.

00:05:09.620 --> 00:05:14.860
<v Carlton Gibson>It's their long days and you have to be full on, you know, in action all day, each day.

00:05:15.150 --> 00:05:17.840
<v Carlton Gibson>And so by the time you get home, you are, you know, kind of exhausted.

00:05:17.890 --> 00:05:22.320
<v Carlton Gibson>And then obviously you've been away and you get home and, you know, you need to be present for your family too.

00:05:22.450 --> 00:05:24.800
<v Carlton Gibson>So there's a limit to how many of these I can do.

00:05:24.850 --> 00:05:28.540
<v Michael Kennedy>So I won't do any other events this year because it's just too much.

00:05:28.540 --> 00:05:28.600
<v Michael Kennedy>Yeah, that's good.

00:05:28.880 --> 00:05:29.460
<v Michael Kennedy>Just too much.

00:05:29.640 --> 00:05:29.820
<v Carlton Gibson>Yeah.

00:05:29.870 --> 00:05:33.640
<v Michael Kennedy>I remember traveling when we had little kids and that was a whole nother level because you'd come back.

00:05:33.760 --> 00:05:36.800
<v Michael Kennedy>It's like, you've been gone for a week, so I need some help.

00:05:37.540 --> 00:05:38.400
<v Michael Kennedy>But I'm exhausted.

00:05:38.500 --> 00:05:38.980
<v Michael Kennedy>You don't understand.

00:05:39.460 --> 00:05:41.700
<v Michael Kennedy>But at the same time, it's a fair request, you know?

00:05:41.960 --> 00:05:42.840
<v Carlton Gibson>Yeah, no, absolutely.

00:05:43.120 --> 00:05:43.240
<v Carlton Gibson>Absolutely.

00:05:43.620 --> 00:05:44.500
<v Carlton Gibson>I mean, our kids are older now.

00:05:45.200 --> 00:05:50.460
<v Carlton Gibson>The eldest is 18, or will be 18 next week, just about to go to university, and then 16,

00:05:50.960 --> 00:05:51.540
<v Carlton Gibson>and 13, 14.

00:05:51.680 --> 00:05:53.320
<v Carlton Gibson>So they are a lot easier than they used to be.

00:05:53.460 --> 00:05:57.820
<v Carlton Gibson>But still, it's a big ask on my partner to go away and they're alone for the whole week.

00:05:58.940 --> 00:05:59.260
<v Carlton Gibson>Absolutely.

00:05:59.620 --> 00:06:02.440
<v Carlton Gibson>In that point position, we might call it.

00:06:03.000 --> 00:06:03.460
<v Michael Kennedy>Yes, exactly.

00:06:04.520 --> 00:06:09.660
<v Michael Kennedy>so uh tell people you mentioned django on the med yes i talked to paulo about this a couple episodes

00:06:09.850 --> 00:06:13.700
<v Michael Kennedy>ago but yeah tell me okay you're going to be part of this too yeah yeah yeah so paulo and i

00:06:14.040 --> 00:06:19.500
<v Carlton Gibson>this was paulo's idea several years ago to have core developer sprints for django um because at the

00:06:19.630 --> 00:06:23.160
<v Carlton Gibson>the end of jank on europe or the end of jank on yes we always have a couple of days of sprints

00:06:23.480 --> 00:06:28.100
<v Carlton Gibson>and they're some of the best parts of the conference um but also it's around the time

00:06:28.310 --> 00:06:32.579
<v Carlton Gibson>people are leaving and your people are already kind of exhausted you know having the three days

00:06:32.600 --> 00:06:39.040
<v Carlton Gibson>of talks plus then of the sprints so it's like we wanted more and um we decided to to do this event

00:06:39.270 --> 00:06:44.100
<v Carlton Gibson>and we were like yeah this is a good idea but it didn't happen and then i think dublin a year a

00:06:44.210 --> 00:06:47.640
<v Carlton Gibson>year and a half ago now uh we were like no come on let's like it so we said we're going to have

00:06:47.640 --> 00:06:52.580
<v Carlton Gibson>it in palo vijay which is my hometown um we did that last september and all last october sorry

00:06:52.740 --> 00:06:57.560
<v Carlton Gibson>beginning of october and we're doing it in paulo's hometown pescara this september and it's as i say

00:06:57.560 --> 00:06:59.440
<v Carlton Gibson>It's three days to get together and work on Django.

00:06:59.760 --> 00:07:02.200
<v Carlton Gibson>And we had some really big features landed.

00:07:02.350 --> 00:07:05.880
<v Carlton Gibson>Like, for instance, Django 6.1 is in pre-release now.

00:07:06.260 --> 00:07:10.860
<v Carlton Gibson>And it's adding an ability to have DB-level cascade options for the on-delete.

00:07:10.980 --> 00:07:16.300
<v Carlton Gibson>So on-delete is, well, you've got a foreign key, so I've got a user, and that user has a bookmark.

00:07:16.560 --> 00:07:18.100
<v Carlton Gibson>Well, what happens when I delete the user?

00:07:18.150 --> 00:07:19.180
<v Carlton Gibson>What happens to the bookmark?

00:07:19.440 --> 00:07:20.900
<v Carlton Gibson>It's got a parent relationship.

00:07:21.380 --> 00:07:23.600
<v Carlton Gibson>And so normally you would cascade that.

00:07:23.880 --> 00:07:28.540
<v Carlton Gibson>Now, Django's always offered this option, but historically, it's done that in memory.

00:07:28.600 --> 00:07:36.980
<v Carlton Gibson>In Python, it's pulled, it's walked the tree, and it's got the objects, and then it's deleted the leaves, and then the stems, and only finally down to the branches.

00:07:37.200 --> 00:07:40.040
<v Carlton Gibson>And now, that can be very costly at scale.

00:07:41.220 --> 00:07:46.380
<v Michael Kennedy>I'm going to delete this one record that references several tables with a million entries.

00:07:46.700 --> 00:07:47.740
<v Michael Kennedy>That could be a lot of money.

00:07:47.980 --> 00:07:48.300
<v Carlton Gibson>Exactly.

00:07:48.500 --> 00:07:53.320
<v Carlton Gibson>You've got an organization that's got team members and projects, and you delete the organization.

00:07:53.600 --> 00:07:59.380
<v Carlton Gibson>it just brings down your application basically you get out of memory memory issues so the database

00:07:59.550 --> 00:08:04.720
<v Carlton Gibson>still has to do that work but the database is much much much better at doing that work than python is

00:08:05.070 --> 00:08:10.640
<v Carlton Gibson>um and so that's so anyway we've got this new option this ticket had been open for i don't know

00:08:10.860 --> 00:08:15.680
<v Carlton Gibson>so many years because it's it was really hard and we had marish feliciak who's a former django fellow

00:08:15.690 --> 00:08:21.919
<v Carlton Gibson>and one of the core um orm contributors simon charette is one of the core open source contributors

00:08:21.940 --> 00:08:26.420
<v Carlton Gibson>Lily Foote, Jacob Tyler-Wells, who's the new Django fellow. We had the four of them just there working

00:08:27.280 --> 00:08:32.380
<v Carlton Gibson>together and they were able to get this over the line. And it's literally something which would

00:08:32.760 --> 00:08:37.979
<v Carlton Gibson>never have happened except for the time that was available at Django on the Met. So we've got

00:08:38.140 --> 00:08:41.940
<v Carlton Gibson>exciting plans for this time. We're hoping to get people again, people from around the community to

00:08:42.080 --> 00:08:46.900
<v Carlton Gibson>come. And I know, I'm really excited about it. The first one was a proof of concept. It went really

00:08:47.080 --> 00:08:50.899
<v Carlton Gibson>well. The second one's lining up to be amazing too. And once we've done two, well, it's kind of

00:08:50.860 --> 00:08:55.920
<v Michael Kennedy>established and you know we're yeah well it's tradition you gotta do it yeah yeah exactly you

00:08:56.200 --> 00:09:01.280
<v Michael Kennedy>gotta go hang out with the mediterranean so i've done i've done a lot of remote work in my life

00:09:01.530 --> 00:09:06.480
<v Michael Kennedy>and i find you know when we do in-person meetups get the people who are all distributed together

00:09:06.770 --> 00:09:11.820
<v Michael Kennedy>it just changes the energy you know there's something about the bandwidth of face-to-face

00:09:12.450 --> 00:09:18.339
<v Carlton Gibson>communication which just isn't rec replicable record you which you can't replicate replica

00:09:18.360 --> 00:09:25.660
<v Carlton Gibson>online yeah thank you mike thank you my but yeah it's it's i don't know i mean look it works online

00:09:25.780 --> 00:09:30.740
<v Carlton Gibson>but as well we had another it was a good session as well so with um django we've got this um we've

00:09:30.740 --> 00:09:35.880
<v Carlton Gibson>had recently this new features board which was an attempt to take people suggesting features away

00:09:35.990 --> 00:09:40.700
<v Carlton Gibson>from django's track instance which is kind of old and scary and is a place for for the work to be

00:09:40.710 --> 00:09:45.839
<v Carlton Gibson>done and put it into a GitHub project and where we could have a project board and we could have

00:09:45.760 --> 00:09:50.260
<v Carlton Gibson>upvotes and we could have a Kanban board to see what were being advanced to try and give a bit

00:09:50.340 --> 00:09:54.600
<v Carlton Gibson>more visibility into the process. And sometimes there's disagreements and those disagreements

00:09:54.700 --> 00:09:58.260
<v Carlton Gibson>online, they can go on forever and ever, but we were able to have a session and there were

00:09:58.360 --> 00:10:02.980
<v Carlton Gibson>different voices from different opinions. And we were able to come to agreements in five minutes

00:10:03.400 --> 00:10:10.200
<v Carlton Gibson>on topics which had been rolling on the boards for months. And that kind of, look, I can see

00:10:10.680 --> 00:10:15.720
<v Carlton Gibson>why you're saying this. It's really hard to get that online. You know, there's money emojis

00:10:15.740 --> 00:10:20.480
<v Michael Kennedy>we use as much as we try and communicate openly it's just as much slack as you want to throw out

00:10:20.620 --> 00:10:27.100
<v Michael Kennedy>there and you know i mean since 2020 it's it's kind of been just oh yeah obviously remote is better

00:10:27.380 --> 00:10:31.900
<v Michael Kennedy>and so on and there are mega advantages don't get me wrong but it's not always better and in lots of

00:10:31.960 --> 00:10:37.400
<v Michael Kennedy>ways so yeah so i guess for this good gone you you mentioned 2020 and it's covid it's one thing that

00:10:37.630 --> 00:10:43.259
<v Carlton Gibson>um i was um django fellow over over that period and um you know it was a really hard time for

00:10:43.280 --> 00:10:48.820
<v Carlton Gibson>because i think that the pandemic really knocked the back out of the community not we had we

00:10:48.900 --> 00:10:52.460
<v Carlton Gibson>had older people stepping away we had young roots that were starting to come and then the put the

00:10:52.880 --> 00:10:58.120
<v Carlton Gibson>the pandemic came and it was kind of like this really really really tough period and it's now

00:10:58.520 --> 00:11:02.080
<v Carlton Gibson>five six years later that we're looking at it we're thinking actually now the community you know the

00:11:02.180 --> 00:11:06.560
<v Carlton Gibson>activity is going on with things like jang and on space and the new community the new members that

00:11:06.580 --> 00:11:11.819
<v Carlton Gibson>are coming through and the people who who are active in the community think well next year when the

00:11:11.840 --> 00:11:16.700
<v Carlton Gibson>steering council is re-elected. There's obvious candidates who weren't there two years ago.

00:11:18.020 --> 00:11:24.420
<v Carlton Gibson>And it's taken us, I think, half a decade to recover from the real knock that was COVID.

00:11:24.720 --> 00:11:30.920
<v Michael Kennedy>Wow. I believe it. I believe it. So is this for just for core devs or is this open to people who

00:11:30.920 --> 00:11:35.660
<v Michael Kennedy>just want to contribute? No, it's open to everybody. Obviously, if you're an established

00:11:36.160 --> 00:11:41.800
<v Carlton Gibson>contributor, it's going to be easy to get going. But it's also about bringing on new

00:11:41.800 --> 00:11:43.480
<v Carlton Gibson>and teaming people up.

00:11:43.760 --> 00:11:45.060
<v Carlton Gibson>And, you know, if you come

00:11:46.600 --> 00:11:48.820
<v Carlton Gibson>and you've got a particular interest in Django,

00:11:48.920 --> 00:11:50.200
<v Carlton Gibson>if you're a total Django novice,

00:11:50.280 --> 00:11:52.740
<v Carlton Gibson>it would be strange to come to choose Django on the Met

00:11:52.740 --> 00:11:53.740
<v Carlton Gibson>as your first event.

00:11:54.080 --> 00:11:56.560
<v Carlton Gibson>But if you're into Django and you know it

00:11:56.640 --> 00:11:57.480
<v Carlton Gibson>and you want to get contributing,

00:11:57.780 --> 00:11:59.880
<v Carlton Gibson>it's, yeah, that's part of the plan too,

00:12:00.200 --> 00:12:02.940
<v Carlton Gibson>is that we have both experienced and new contributors

00:12:03.100 --> 00:12:05.100
<v Carlton Gibson>and we can team people up and get people working together.

00:12:05.440 --> 00:12:09.500
<v Carlton Gibson>And, you know, part of the contributing to Django problem

00:12:09.900 --> 00:12:10.660
<v Carlton Gibson>is the learning curve.

00:12:10.760 --> 00:12:12.620
<v Carlton Gibson>So it feels like it's a big, scary thing.

00:12:12.750 --> 00:12:17.640
<v Carlton Gibson>Whereas if you sit down with somebody in a room and you work on what they're working on, you're like, actually, I could work on this side ticket.

00:12:17.800 --> 00:12:19.200
<v Carlton Gibson>And all of a sudden people are off.

00:12:19.300 --> 00:12:23.000
<v Michael Kennedy>Well, it suffers from what many of the successful projects out there do.

00:12:23.420 --> 00:12:24.880
<v Michael Kennedy>It's super polished.

00:12:25.280 --> 00:12:27.260
<v Michael Kennedy>It's used by millions and millions of projects.

00:12:27.980 --> 00:12:33.900
<v Michael Kennedy>It's hard to just look at it and go, oh, I'll just change this without regard to all that knock on effect, right?

00:12:33.960 --> 00:12:45.740
<v Carlton Gibson>Yes. And that's where part of the new features repo was about giving a space where people could put their ideas forward without having them immediately closed as won't fix.

00:12:45.890 --> 00:12:51.840
<v Carlton Gibson>Because if you went and, you know, oh, can we integrate React? No, we're not going to do that.

00:12:52.020 --> 00:12:57.080
<v Carlton Gibson>You can use React and here's how we use React, but we're not going to integrate it into JankRusate as an example.

00:12:57.620 --> 00:12:59.860
<v Carlton Gibson>And that would be open, it'd be no, no, close, won't fix.

00:13:00.010 --> 00:13:01.939
<v Carlton Gibson>And people felt that that was a little bit too...

00:13:03.060 --> 00:13:04.140
<v Carlton Gibson>A little too stack over flowy.

00:13:04.660 --> 00:13:06.020
<v Carlton Gibson>Well, yeah, it wasn't ideal.

00:13:06.180 --> 00:13:10.880
<v Carlton Gibson>So the new features repo is to allow space to develop for a conversation.

00:13:11.060 --> 00:13:14.120
<v Carlton Gibson>And then those ideas which have a bit of traction, we can say, no, do you know what?

00:13:14.340 --> 00:13:17.520
<v Carlton Gibson>A lot of people have said they support this one or we can advance it.

00:13:18.400 --> 00:13:18.940
<v Michael Kennedy>Hey, one second.

00:13:19.399 --> 00:13:23.180
<v Michael Kennedy>Normally, this would be an ad break from Sentry, but not this time.

00:13:23.560 --> 00:13:27.120
<v Michael Kennedy>Let's just thank them for supporting the show and get right back to the conversation.

00:13:27.620 --> 00:13:31.000
<v Michael Kennedy>Also, visit talkpython.fm/sentry after the show.

00:13:31.340 --> 00:13:31.800
<v Michael Kennedy>Thanks, Sentry.

00:13:32.280 --> 00:13:35.440
<v Michael Kennedy>You have another project coming along, The Mantle.

00:13:35.710 --> 00:13:36.400
<v Michael Kennedy>Tell us what this is.

00:13:37.000 --> 00:13:37.600
<v Carlton Gibson>Django Mantle.

00:13:38.600 --> 00:13:42.840
<v Carlton Gibson>So it's my take on a modern serialization story for Django.

00:13:43.020 --> 00:13:46.900
<v Carlton Gibson>So the big topic we've had is about REST Framework, and REST Framework is aging.

00:13:46.950 --> 00:13:49.280
<v Carlton Gibson>And REST Framework is wonderful, and it has been wonderful.

00:13:49.480 --> 00:13:51.860
<v Carlton Gibson>But the serializers in particular, they're kind of last generation.

00:13:52.190 --> 00:14:01.040
<v Carlton Gibson>And there are much more modern approaches available now, involved in Pydantic, in msgspec, in Attrs and Catters, which is what I use,

00:14:01.180 --> 00:14:03.820
<v Carlton Gibson>are much, much faster than they used to be.

00:14:04.650 --> 00:14:06.580
<v Carlton Gibson>They generate code which is almost as quick

00:14:07.550 --> 00:14:09.780
<v Carlton Gibson>as you would see if you hand-wrote it.

00:14:09.940 --> 00:14:12.580
<v Carlton Gibson>If you wrote the code to literally pull this attribute,

00:14:12.820 --> 00:14:13.680
<v Carlton Gibson>this attribute, this attribute,

00:14:13.920 --> 00:14:16.300
<v Carlton Gibson>the code that's generated for you by, say, caters

00:14:16.550 --> 00:14:19.440
<v Carlton Gibson>is as quick as that code, which is amazing.

00:14:19.610 --> 00:14:21.680
<v Carlton Gibson>Whereas REST Framework serializes,

00:14:21.730 --> 00:14:24.420
<v Carlton Gibson>they go through and get the field object from the model.

00:14:24.580 --> 00:14:26.100
<v Carlton Gibson>They get the value from it.

00:14:26.500 --> 00:14:28.400
<v Carlton Gibson>It's like, hang on, all of those loopings

00:14:28.680 --> 00:14:30.200
<v Carlton Gibson>are very slow by comparison.

00:14:31.300 --> 00:14:33.880
<v Carlton Gibson>So then the question is, well, why don't we just use Pydantic?

00:14:34.120 --> 00:14:37.440
<v Carlton Gibson>Well, Pydantic's fantastic, but it's not ORM aware.

00:14:37.870 --> 00:14:42.000
<v Carlton Gibson>It doesn't have any insight into how Django works or anything.

00:14:42.050 --> 00:14:44.360
<v Carlton Gibson>So Django Mantle is my take on that.

00:14:44.540 --> 00:14:49.540
<v Carlton Gibson>It's the type-save wrapper around Django's liquid core.

00:14:49.550 --> 00:14:54.320
<v Carlton Gibson>So you write these atters classes, and then you just instantiate a query object,

00:14:54.390 --> 00:15:00.260
<v Carlton Gibson>and you go fetch, and it fully instantiates your object to the shape you define,

00:15:00.520 --> 00:15:02.140
<v Carlton Gibson>and it fetches all the data all in one go,

00:15:02.540 --> 00:15:03.900
<v Carlton Gibson>and it can do that for nested objects,

00:15:04.040 --> 00:15:06.300
<v Carlton Gibson>and you can do writes with validation, all these things.

00:15:07.460 --> 00:15:10.880
<v Carlton Gibson>And it's ORM aware in that it only selects the fields that it needs,

00:15:11.260 --> 00:15:14.280
<v Carlton Gibson>and it does the pre-fetch relateds for you

00:15:14.420 --> 00:15:16.900
<v Carlton Gibson>so that you don't get this N plus one query problem, right?

00:15:17.140 --> 00:15:20.520
<v Carlton Gibson>And it's very good for performant fetches

00:15:20.660 --> 00:15:22.400
<v Carlton Gibson>of just the exact data you want.

00:15:23.580 --> 00:15:26.460
<v Carlton Gibson>You end up with your objects being fully Python typed.

00:15:27.199 --> 00:15:28.700
<v Carlton Gibson>They're at its classes, so they're typed.

00:15:28.880 --> 00:15:30.240
<v Carlton Gibson>Your type checker knows the way around them.

00:15:30.800 --> 00:15:35.620
<v Carlton Gibson>mypy or PyRefly is now adding support for Atis classes too, I see this week.

00:15:36.380 --> 00:15:36.760
<v Carlton Gibson>Oh, cool.

00:15:36.960 --> 00:15:43.120
<v Carlton Gibson>So anyway, it's an attempt to show what a serialization story using these modern approaches

00:15:43.320 --> 00:15:43.840
<v Carlton Gibson>can look like.

00:15:44.080 --> 00:15:48.360
<v Carlton Gibson>And then from there, it's like, well, okay, what does an API framework in 2026 look?

00:15:48.460 --> 00:15:48.980
<v Michael Kennedy>I love it.

00:15:49.030 --> 00:15:49.680
<v Michael Kennedy>I love this idea.

00:15:49.850 --> 00:15:55.979
<v Michael Kennedy>I think there's a lot of value in taking some of these very focused, lightweight, dedicated

00:15:56.440 --> 00:16:03.600
<v Michael Kennedy>like data structures right like c attrs or pydantic or just straight data classes and somehow

00:16:03.880 --> 00:16:07.320
<v Michael Kennedy>integrating them in here without just this heavyweight stuff like you talked about the

00:16:07.320 --> 00:16:12.180
<v Michael Kennedy>eager loading that's and some of the other things like the typing can be weird as well like sql

00:16:12.380 --> 00:16:18.420
<v Michael Kennedy>alchemy you kind of have to lie to it yes somehow do you either lie about it being a database thing

00:16:18.420 --> 00:16:24.320
<v Michael Kennedy>or do you lie about it being an integer when it actually gets the value right like yeah yeah yeah

00:16:24.320 --> 00:16:28.300
<v Carlton Gibson>Yeah, because the actual type is a Django deferred field, right?

00:16:28.400 --> 00:16:30.460
<v Carlton Gibson>For a Django modifier, it's a deferred field.

00:16:30.540 --> 00:16:31.940
<v Carlton Gibson>What on earth is a deferred field?

00:16:32.160 --> 00:16:34.900
<v Carlton Gibson>Where it's an object that has the descriptive protocol

00:16:35.120 --> 00:16:37.660
<v Carlton Gibson>so that when you access it, it gives you the converted value.

00:16:38.640 --> 00:16:39.420
<v Carlton Gibson>I don't want to type that.

00:16:40.380 --> 00:16:40.680
<v Michael Kennedy>Exactly.

00:16:41.100 --> 00:16:43.080
<v Michael Kennedy>Even if you do, then you try to add it to an integer.

00:16:43.240 --> 00:16:44.460
<v Michael Kennedy>It's like, you can't add that to an integer.

00:16:44.640 --> 00:16:45.400
<v Michael Kennedy>That's not the same thing.

00:16:45.440 --> 00:16:46.460
<v Michael Kennedy>You're like, but it's going to be.

00:16:46.780 --> 00:16:47.860
<v Carlton Gibson>Yeah, it's going to be.

00:16:47.940 --> 00:16:51.680
<v Carlton Gibson>So this was the topic of my talk that I gave in Amsterdam

00:16:52.020 --> 00:16:53.600
<v Carlton Gibson>and Django in Europe and PyCon in Thailand.

00:16:53.820 --> 00:16:57.420
<v Carlton Gibson>It's slightly different versions because I had more time at DjangoCon Europe.

00:16:57.470 --> 00:17:02.500
<v Carlton Gibson>So I went more into the way we grow from, say, an active record pattern,

00:17:02.550 --> 00:17:03.640
<v Carlton Gibson>which is like a Django model, right?

00:17:03.640 --> 00:17:04.319
<v Carlton Gibson>It's an active record.

00:17:04.500 --> 00:17:09.300
<v Carlton Gibson>It's your database row, but it's also an object that's got logical behaviors

00:17:09.680 --> 00:17:12.439
<v Carlton Gibson>to a more separated approach where you're like, hang on,

00:17:12.470 --> 00:17:16.300
<v Carlton Gibson>I don't want my display logic and my aggregation logic

00:17:16.500 --> 00:17:20.060
<v Carlton Gibson>and my user validation logic all living on this one database class.

00:17:20.699 --> 00:17:21.959
<v Carlton Gibson>So how do we separate those off?

00:17:22.040 --> 00:17:23.819
<v Carlton Gibson>That's kind of where mantle came from originally.

00:17:24.240 --> 00:17:32.800
<v Carlton Gibson>But the idea was that, and my take is that Django's ORM is built on these very powerful dynamic patterns that Python allows.

00:17:33.260 --> 00:17:36.660
<v Carlton Gibson>And retrofitting static typing onto that is going to be a real problem.

00:17:36.710 --> 00:17:38.700
<v Carlton Gibson>So instead of that, that's also a dynamic C.

00:17:38.840 --> 00:17:44.260
<v Carlton Gibson>On top of that, we have these static islands where we can have all the nice type safety features of modern Python,

00:17:44.550 --> 00:17:49.940
<v Carlton Gibson>but we're not trying to eliminate the very powerful dynamic features that we have underneath.

00:17:50.520 --> 00:17:55.180
<v Carlton Gibson>And so the mantles, the shape classes that you define, they're your static islands.

00:17:55.290 --> 00:17:58.580
<v Carlton Gibson>And there you can add all your domain logic that you want to, and you can have it fully

00:17:58.860 --> 00:17:59.660
<v Carlton Gibson>separated from the database.

00:18:00.030 --> 00:18:01.500
<v Carlton Gibson>And they are just pure Python classes.

00:18:01.720 --> 00:18:03.860
<v Carlton Gibson>There's no data mutability in there.

00:18:03.960 --> 00:18:09.120
<v Carlton Gibson>There's no like, say, oh, I modified a property and saved, and now I've updated something.

00:18:09.310 --> 00:18:11.580
<v Carlton Gibson>No, there's none of that active record stuff going on.

00:18:11.630 --> 00:18:17.199
<v Carlton Gibson>So it's a kind of way of getting some of the benefits of this sort of separation layers

00:18:17.220 --> 00:18:22.520
<v Michael Kennedy>type approach whilst still keeping everything you love from the aura uh spoken like someone who

00:18:23.260 --> 00:18:28.760
<v Michael Kennedy>appreciates some nice architecture and layers are very cool well the issue is as your application

00:18:28.940 --> 00:18:35.100
<v Carlton Gibson>grows the complexity grows with it and you need more structure to manage that complexity like you

00:18:35.240 --> 00:18:40.440
<v Carlton Gibson>simply do like it the issue isn't that you started with a jango model it's that your jango model grew

00:18:40.640 --> 00:18:44.999
<v Carlton Gibson>and then grew and it grew and all of a sudden it's got it's got methods on it which are several

00:18:45.020 --> 00:18:50.060
<v Carlton Gibson>distinct responsibilities. And then you think, oh, I'm getting a lot of code here. So you start

00:18:50.480 --> 00:18:55.300
<v Carlton Gibson>doing dry, you start abstracting methods. But that abstracted method wasn't for a single

00:18:55.580 --> 00:19:01.140
<v Carlton Gibson>responsibility. So it's not clean so that you've got, ah. And everybody who's grown a Django app

00:19:01.150 --> 00:19:05.460
<v Carlton Gibson>has seen this. And it's perfectly natural and it's perfectly normal. It's just how applications grow.

00:19:05.840 --> 00:19:10.819
<v Carlton Gibson>And Mantle is kind of my take on, well, how do we deal with that? How do we grow from the active

00:19:10.840 --> 00:19:15.080
<v Carlton Gibson>record pattern that we began to, to something that's more robust. And it's not that you should

00:19:15.300 --> 00:19:19.960
<v Carlton Gibson>start with the more robust patterns because you're over-engineering them. There's a nice road we have

00:19:20.020 --> 00:19:25.600
<v Carlton Gibson>to walk between these two problems. And that's the joy of crafting the application over time,

00:19:25.700 --> 00:19:35.580
<v Michael Kennedy>right? That's why we're programmed. I love it. I love it. All right. Let's talk about Django 6.1.

00:19:35.940 --> 00:19:36.020
<v Michael Kennedy>Yes.

00:19:36.500 --> 00:19:38.820
<v Michael Kennedy>I think it's coming out.

00:19:38.960 --> 00:19:39.740
<v Michael Kennedy>It's not quite finished,

00:19:39.900 --> 00:19:41.940
<v Michael Kennedy>but there's a lot of features we're going to talk about

00:19:42.010 --> 00:19:43.800
<v Michael Kennedy>that are pretty exciting and related here, right?

00:19:43.900 --> 00:19:45.640
<v Michael Kennedy>A lot of new things coming out of the database, for example.

00:19:45.920 --> 00:19:46.160
<v Carlton Gibson>Yes.

00:19:46.270 --> 00:19:47.640
<v Carlton Gibson>So I think it's in beta now.

00:19:47.730 --> 00:19:49.860
<v Carlton Gibson>So it's going to be a release candidate in a couple of weeks,

00:19:49.930 --> 00:19:52.000
<v Carlton Gibson>and then we're about a month away from the final release.

00:19:52.080 --> 00:19:54.440
<v Carlton Gibson>Oh, by the way, I see you've got the page up there.

00:19:54.470 --> 00:19:56.480
<v Carlton Gibson>The Django Developer Survey is on right now.

00:19:56.650 --> 00:19:57.360
<v Carlton Gibson>Do take it, folks.

00:19:57.390 --> 00:19:58.640
<v Carlton Gibson>We want as many replies as we can.

00:19:58.750 --> 00:20:01.840
<v Carlton Gibson>It really does help drive forward the development of Django.

00:20:01.890 --> 00:20:05.000
<v Carlton Gibson>So do give that five minutes there of your time to answer the questions.

00:20:05.640 --> 00:20:06.840
<v Carlton Gibson>But what's exciting?

00:20:07.400 --> 00:20:09.940
<v Carlton Gibson>Well, we already talked about database-level cascades, right?

00:20:10.140 --> 00:20:13.660
<v Carlton Gibson>That's my big feature because that's something that you just couldn't work around.

00:20:13.850 --> 00:20:16.300
<v Carlton Gibson>And there's no in Python solution to that.

00:20:16.350 --> 00:20:17.080
<v Carlton Gibson>We wanted that.

00:20:17.660 --> 00:20:23.000
<v Carlton Gibson>You'd have to, well, in the past, I have created a custom database backend and changed the way it creates.

00:20:23.240 --> 00:20:25.200
<v Carlton Gibson>No, that's not a sustainable way.

00:20:25.970 --> 00:20:28.200
<v Carlton Gibson>So database-level cascades are my favorite one.

00:20:28.200 --> 00:20:32.860
<v Carlton Gibson>The one that I think more people are excited about is called fetch modes, which are on query sets.

00:20:33.280 --> 00:20:37.100
<v Carlton Gibson>So we talked about the prefetch related with the way Mantle handles it.

00:20:37.600 --> 00:20:43.900
<v Carlton Gibson>But the idea is you've got an author and so you fetch a user and they've got a list of bookmarks.

00:20:43.940 --> 00:20:50.960
<v Carlton Gibson>And so you get the author, you get the user, and then you iterate for, bookmark in users.bookmark, do display the bookmark.

00:20:51.120 --> 00:20:57.480
<v Carlton Gibson>And so what happens is it goes through and you get this N plus one query where it goes and fetches the lazy related object.

00:20:57.660 --> 00:21:05.260
<v Carlton Gibson>And that's great at small scales and when you're getting going, but it's a performance nightmare, basically.

00:21:05.600 --> 00:21:09.740
<v Carlton Gibson>And this N plus one problem that comes up is something that people hit all and all again.

00:21:10.160 --> 00:21:13.800
<v Carlton Gibson>So fetch modes let you say, look, fetch peers, which is the exciting one.

00:21:13.800 --> 00:21:19.760
<v Carlton Gibson>It lets you say, if I access the first element in a list of related objects, go and get them all at once.

00:21:20.220 --> 00:21:22.480
<v Carlton Gibson>And so it's like a deferred pre-fetch related.

00:21:22.700 --> 00:21:32.820
<v Carlton Gibson>And that's going to be a lot easier to manage in a lot of cases than having to manually add the pre-fetch related in each case, because those often become difficult to do by hand.

00:21:33.560 --> 00:21:36.320
<v Carlton Gibson>They're all right to add the first time, but they become difficult to keep updated.

00:21:36.500 --> 00:21:39.720
<v Carlton Gibson>And then you're suddenly pre-fetch relating something which you're no longer using.

00:21:40.060 --> 00:21:41.500
<v Carlton Gibson>And so fetch mode.

00:21:41.510 --> 00:21:44.800
<v Michael Kennedy>Yeah, your data access layer, maybe you're right, says get me all the books.

00:21:45.290 --> 00:21:49.840
<v Michael Kennedy>But then you might have to add like some kind of flag that says and the authors.

00:21:50.300 --> 00:21:53.560
<v Michael Kennedy>Because sometimes you want to call that function and you want the authors, but sometimes you

00:21:53.620 --> 00:21:55.780
<v Michael Kennedy>want to just list the books, you know, and you don't need that.

00:21:55.920 --> 00:22:00.040
<v Michael Kennedy>And it's like, yeah, so this way kind of just lets you says, if you're ever going to do

00:22:00.480 --> 00:22:04.040
<v Michael Kennedy>this lazy loading, then basically upgrade it to an eager load.

00:22:04.260 --> 00:22:04.700
<v Carlton Gibson>Yes.

00:22:04.980 --> 00:22:09.080
<v Carlton Gibson>And there's, I don't think the videos are quite out yet, but Jacob Tyler was, you found

00:22:09.160 --> 00:22:12.400
<v Carlton Gibson>a great talk at DjangoCon Europe on the, on how fetch modes work.

00:22:12.840 --> 00:22:13.760
<v Carlton Gibson>They'll, they'll be out very soon.

00:22:13.760 --> 00:22:17.340
<v Carlton Gibson>I think they'll be on YouTube, but do look out for them in the interest, but how, how,

00:22:17.480 --> 00:22:22.000
<v Carlton Gibson>how fetchmates work and what the limitations were and you know where it's not you know it's not a

00:22:22.250 --> 00:22:27.660
<v Carlton Gibson>magic solution it's not just oh let's enable fetch peers in all cases you know without thinking about

00:22:27.660 --> 00:22:32.860
<v Carlton Gibson>it no but it it's a good change and um when i've spoken to people about what they're excited about

00:22:32.860 --> 00:22:39.020
<v Carlton Gibson>in 6.1 most people have pointed to this as their headline feature nice nice so that's good and then

00:22:39.090 --> 00:22:43.420
<v Carlton Gibson>i guess um a little bit yeah go ahead well i was going to say about the other thing that i'm

00:22:43.520 --> 00:22:47.440
<v Carlton Gibson>excited about that you know there's an old list of features the thing with django releases every

00:22:47.460 --> 00:22:50.540
<v Carlton Gibson>there's this big long release pack go and read it go and pick your favorite features but there's

00:22:50.620 --> 00:22:57.460
<v Carlton Gibson>always lots in there for everything but you we can now customize the multi-part parser class for

00:22:57.900 --> 00:23:03.640
<v Carlton Gibson>on the request object now whoever wants to do that well it turns out but that you can when you're

00:23:03.640 --> 00:23:08.780
<v Carlton Gibson>getting multi-part requests if you're getting big ones with lots of multi-parts in swapping out a

00:23:08.860 --> 00:23:14.679
<v Carlton Gibson>different multi-parts class could save you maybe 30 on your passing time which is 30 cpu on pass it

00:23:14.580 --> 00:23:15.880
<v Carlton Gibson>on request for a passing time,

00:23:16.320 --> 00:23:18.080
<v Carlton Gibson>that may be a non-negagable game.

00:23:18.610 --> 00:23:19.560
<v Carlton Gibson>So I'm excited about that.

00:23:19.590 --> 00:23:23.240
<v Carlton Gibson>But I'm excited because it starts to open the pathway

00:23:23.500 --> 00:23:25.340
<v Carlton Gibson>to more customizations on the request.

00:23:25.390 --> 00:23:27.960
<v Carlton Gibson>And so, you know, Django's got this implementation,

00:23:28.400 --> 00:23:30.720
<v Carlton Gibson>but can we swap out that implementation and compare it

00:23:31.070 --> 00:23:32.900
<v Carlton Gibson>and see if we can get some performance gains

00:23:33.030 --> 00:23:36.220
<v Carlton Gibson>by being able to turn some of these nods under the hood.

00:23:36.440 --> 00:23:37.100
<v Michael Kennedy>That's really nice.

00:23:37.500 --> 00:23:39.820
<v Michael Kennedy>And having it be customizable is really great.

00:23:39.940 --> 00:23:40.460
<v Michael Kennedy>I'm working on this.

00:23:40.740 --> 00:23:43.440
<v Michael Kennedy>I'm working with this project that I just want to use.

00:23:43.560 --> 00:23:48.000
<v Michael Kennedy>I don't want to be an author of it, but it has no real extensibility.

00:23:48.120 --> 00:23:52.540
<v Michael Kennedy>So anything I want to do, I have to write little custom bits and change the source code and run it.

00:23:52.600 --> 00:23:55.980
<v Michael Kennedy>And I'm like, as soon as there's a new version, how am I going to pay for this?

00:23:56.040 --> 00:23:56.560
<v Michael Kennedy>You know what I mean?

00:23:56.640 --> 00:23:58.580
<v Michael Kennedy>And so this is really nice.

00:23:58.820 --> 00:23:59.060
<v Carlton Gibson>Yeah.

00:24:00.200 --> 00:24:04.040
<v Carlton Gibson>Again, this is customizing the multi-part pass class.

00:24:04.140 --> 00:24:05.920
<v Carlton Gibson>I don't think that's for the majority of people.

00:24:06.220 --> 00:24:08.320
<v Carlton Gibson>But when you need it, it's going to be there.

00:24:08.320 --> 00:24:10.340
<v Carlton Gibson>And it's like, okay, actually, there's a big win.

00:24:10.520 --> 00:24:11.600
<v Carlton Gibson>So those are my favorites.

00:24:12.040 --> 00:24:12.160
<v Michael Kennedy>Yeah.

00:24:12.280 --> 00:24:12.380
<v Michael Kennedy>Okay.

00:24:13.020 --> 00:24:14.100
<v Michael Kennedy>What's the timeline?

00:24:14.520 --> 00:24:15.620
<v Michael Kennedy>It's in beta, you say?

00:24:15.860 --> 00:24:16.800
<v Carlton Gibson>It's in beta now.

00:24:16.890 --> 00:24:18.420
<v Carlton Gibson>So the pre-release is three months.

00:24:19.620 --> 00:24:21.160
<v Carlton Gibson>So the alpha was a month ago.

00:24:21.250 --> 00:24:22.240
<v Carlton Gibson>The beta is just now.

00:24:23.300 --> 00:24:25.720
<v Carlton Gibson>And then the release candidate will be in a week or so.

00:24:25.830 --> 00:24:27.480
<v Carlton Gibson>And then a couple of weeks after that will be the final.

00:24:27.590 --> 00:24:29.240
<v Carlton Gibson>So I guess August will be.

00:24:29.640 --> 00:24:30.960
<v Carlton Gibson>So it's coming fast.

00:24:31.640 --> 00:24:32.580
<v Carlton Gibson>Oh, yes, yes, yes.

00:24:32.960 --> 00:24:34.140
<v Carlton Gibson>It's very nearly there.

00:24:34.360 --> 00:24:34.860
<v Carlton Gibson>Excellent.

00:24:35.620 --> 00:24:36.940
<v Carlton Gibson>So just a call to action.

00:24:37.240 --> 00:24:41.400
<v Carlton Gibson>If you're using Django and you could just install it against your application and run the beta,

00:24:41.740 --> 00:24:43.040
<v Carlton Gibson>run your test suite against the beta.

00:24:43.400 --> 00:24:45.840
<v Carlton Gibson>That would be amazing because if it passes, then you're all good.

00:24:46.000 --> 00:24:46.500
<v Carlton Gibson>You're good to update.

00:24:46.940 --> 00:24:48.560
<v Carlton Gibson>If it doesn't pass, then we want to know that.

00:24:48.580 --> 00:24:50.080
<v Carlton Gibson>We want to know, well, what changed?

00:24:51.480 --> 00:24:55.140
<v Michael Kennedy>It's got to be such a challenge to work on a project that's used,

00:24:55.700 --> 00:24:58.760
<v Michael Kennedy>not just consumed by people, but as a programmer,

00:24:59.500 --> 00:25:01.200
<v Michael Kennedy>code integrated with this project, right?

00:25:01.440 --> 00:25:01.960
<v Carlton Gibson>Yeah, yeah, yeah.

00:25:02.000 --> 00:25:03.520
<v Carlton Gibson>And there's the whole ecosystem, right?

00:25:03.600 --> 00:25:06.100
<v Carlton Gibson>So, I mean, why do people use Django?

00:25:06.180 --> 00:25:09.140
<v Carlton Gibson>One of the big reasons they use Django is for the third-party packages

00:25:09.160 --> 00:25:14.160
<v Carlton Gibson>that are available, CMSs or image processing packages or whatever it is that you want to do.

00:25:14.640 --> 00:25:16.140
<v Carlton Gibson>Did we break any integrations with them?

00:25:16.360 --> 00:25:17.100
<v Carlton Gibson>Ah, well, we hope not.

00:25:17.280 --> 00:25:19.540
<v Carlton Gibson>But unless people test and tell us, we don't find out.

00:25:19.740 --> 00:25:24.480
<v Carlton Gibson>And when I was a fellow for five years and in every release, we would have on release

00:25:24.780 --> 00:25:30.080
<v Carlton Gibson>day bug reports saying, oh, there's some regression here when we could have had those a week before

00:25:30.080 --> 00:25:31.360
<v Carlton Gibson>and fix them before the final release.

00:25:31.700 --> 00:25:31.980
<v Michael Kennedy>Yeah.

00:25:32.740 --> 00:25:35.640
<v Michael Kennedy>Have you looked lately what the...

00:25:36.080 --> 00:25:36.960
<v Michael Kennedy>I'll just do it like this.

00:25:37.180 --> 00:25:39.340
<v Michael Kennedy>Have you looked at what it says used by?

00:25:39.720 --> 00:25:40.640
<v Michael Kennedy>On GitHub here?

00:25:41.060 --> 00:25:43.680
<v Michael Kennedy>You know, it says, does it say how many projects?

00:25:43.710 --> 00:25:44.080
<v Michael Kennedy>I don't know.

00:25:44.110 --> 00:25:44.760
<v Michael Kennedy>Oh yeah, here we go.

00:25:45.020 --> 00:25:47.000
<v Michael Kennedy>This is just the public projects on GitHub,

00:25:47.300 --> 00:25:48.500
<v Michael Kennedy>but two million?

00:25:48.720 --> 00:25:49.420
<v Michael Kennedy>Couple of million.

00:25:49.530 --> 00:25:49.820
<v Michael Kennedy>Yeah, right.

00:25:49.990 --> 00:25:50.540
<v Michael Kennedy>Okay, good.

00:25:51.740 --> 00:25:52.880
<v Michael Kennedy>There's so many that are private.

00:25:53.040 --> 00:25:54.380
<v Michael Kennedy>There's so many that have nothing to do with GitHub.

00:25:54.600 --> 00:25:56.500
<v Michael Kennedy>It's like a very small sample.

00:25:56.660 --> 00:25:57.760
<v Carlton Gibson>That's still crazy.

00:25:58.140 --> 00:26:00.820
<v Carlton Gibson>Jay, a couple of years ago at DjangoCon US,

00:26:01.030 --> 00:26:01.900
<v Carlton Gibson>I think maybe 2024,

00:26:02.480 --> 00:26:04.360
<v Carlton Gibson>Jacob Kaplan Moss gave a sort of best estimate

00:26:04.490 --> 00:26:06.280
<v Carlton Gibson>about the user base, about 4 million,

00:26:06.420 --> 00:26:12.360
<v Carlton Gibson>something like that as a kind of you know four million in action django projects in in the wild

00:26:12.900 --> 00:26:18.920
<v Michael Kennedy>so okay you know if you count all the internal apps given that two million public repos on gmail

00:26:19.010 --> 00:26:23.120
<v Michael Kennedy>if you count all the internal private things that people just threw up for like a little internal

00:26:23.380 --> 00:26:28.320
<v Michael Kennedy>company app i bet it's way more than four million now i obviously couldn't prove it but yeah i mean

00:26:28.680 --> 00:26:31.820
<v Carlton Gibson>i'd have to ask jacob how he came up with the figures but it was in terms of downloads and

00:26:31.890 --> 00:26:36.380
<v Michael Kennedy>looking at this and looking yeah yeah exactly yeah probably the big query data pipe yeah or

00:26:36.400 --> 00:26:40.600
<v Michael Kennedy>Yeah. Honestly, I feel like that stuff's kind of broken these days. Like 10 years ago,

00:26:40.710 --> 00:26:46.460
<v Michael Kennedy>that was pretty representative. But right now, I think it's kind of a little weird on both ends.

00:26:46.530 --> 00:26:51.240
<v Michael Kennedy>I think it's weird because there's so many Docker builds, right? This project can say,

00:26:51.240 --> 00:26:56.120
<v Michael Kennedy>well, it got a hundred downloads. Well, that's because we just built something in CI a bunch

00:26:56.120 --> 00:27:00.740
<v Michael Kennedy>of times today. We didn't really intend to download it, that many projects. But also at the same time,

00:27:01.100 --> 00:27:05.320
<v Michael Kennedy>there's a bunch of caching layers. Like you might grab a Docker image that has it or build a Docker

00:27:05.340 --> 00:27:11.360
<v Michael Kennedy>image and never reload that or use uv caching like it's just so opaque i think i mean a lot of people

00:27:11.480 --> 00:27:17.280
<v Carlton Gibson>run little local pi pi pi cache or pip caches sorry so when you do pip install you're not you

00:27:17.420 --> 00:27:22.140
<v Carlton Gibson>fetched it once you but you might have installed it a hundred times from that so yeah exactly what's

00:27:22.150 --> 00:27:27.640
<v Michael Kennedy>that meant to be counted as one or i i think it's why it's so hard i don't have no idea so anyway

00:27:28.080 --> 00:27:33.159
<v Michael Kennedy>um incredible and this obviously is right i was gonna say once upon a time there was a an idea

00:27:33.180 --> 00:27:35.720
<v Carlton Gibson>to add some kind of telemetry into Django.

00:27:35.800 --> 00:27:36.660
<v Carlton Gibson>And obviously that didn't go anywhere

00:27:36.690 --> 00:27:38.360
<v Carlton Gibson>because no one wants to go anywhere near that.

00:27:38.540 --> 00:27:42.400
<v Carlton Gibson>But we still sometimes would like some idea

00:27:42.640 --> 00:27:44.140
<v Carlton Gibson>about how we estimate

00:27:44.290 --> 00:27:46.720
<v Carlton Gibson>because the sort of visible Django community

00:27:46.860 --> 00:27:48.020
<v Carlton Gibson>is like a thousandth of a percent.

00:27:48.180 --> 00:27:51.580
<v Carlton Gibson>So say Jacob's 4 million users isn't too far off.

00:27:51.920 --> 00:27:53.240
<v Carlton Gibson>The sort of visible community,

00:27:53.270 --> 00:27:54.800
<v Carlton Gibson>the people who subscribe to the newsletter

00:27:54.980 --> 00:27:57.300
<v Carlton Gibson>or do the survey or come to Django cons

00:27:57.330 --> 00:27:58.140
<v Carlton Gibson>or all these things,

00:27:58.620 --> 00:28:00.120
<v Carlton Gibson>they're about a thousandth of that,

00:28:00.180 --> 00:28:02.360
<v Carlton Gibson>like 4,000 or 5,000 people

00:28:02.380 --> 00:28:03.880
<v Carlton Gibson>that we can sort of point to and go,

00:28:03.920 --> 00:28:04.580
<v Carlton Gibson>I know who you are.

00:28:04.590 --> 00:28:05.320
<v Carlton Gibson>I know who you are.

00:28:05.600 --> 00:28:06.560
<v Carlton Gibson>Yes, I've heard of you.

00:28:06.680 --> 00:28:07.760
<v Carlton Gibson>Yes, I know you're on LinkedIn.

00:28:08.640 --> 00:28:10.500
<v Carlton Gibson>But that's a fraction of the total user base.

00:28:10.610 --> 00:28:12.380
<v Carlton Gibson>It would be lovely if we had some way

00:28:12.380 --> 00:28:14.260
<v Michael Kennedy>of just reaching more.

00:28:14.860 --> 00:28:15.060
<v Michael Kennedy>I know.

00:28:15.140 --> 00:28:16.420
<v Michael Kennedy>I have a new term for you, Carlton.

00:28:16.800 --> 00:28:17.740
<v Michael Kennedy>Dark matter Django.

00:28:18.200 --> 00:28:18.760
<v Carlton Gibson>Yeah, brilliant.

00:28:18.980 --> 00:28:19.500
<v Carlton Gibson>I'm having that.

00:28:19.510 --> 00:28:20.220
<v Carlton Gibson>Let me write that down.

00:28:21.740 --> 00:28:22.680
<v Michael Kennedy>Dark matter Django.

00:28:22.980 --> 00:28:24.000
<v Michael Kennedy>Well, we're talking about stuff

00:28:24.110 --> 00:28:26.380
<v Michael Kennedy>that is not dark matter.

00:28:27.100 --> 00:28:30.700
<v Michael Kennedy>So you recently did a bunch of work

00:28:31.060 --> 00:28:39.400
<v Michael Kennedy>on the documentation and positioning of async in general for django yeah yes yeah so the topic

00:28:39.660 --> 00:28:43.580
<v Carlton Gibson>guide which is here that you've got up on the page the asynchronous support topic guide it was first

00:28:43.780 --> 00:28:51.480
<v Carlton Gibson>written when we very first added um async support to django back in like that jango 3. something or

00:28:51.580 --> 00:28:58.360
<v Carlton Gibson>i can't remember exactly but i think it was um python 3.6 it was was when it is and um async's

00:28:58.380 --> 00:29:04.760
<v Carlton Gibson>come an awful long way since then and it was very negative it was like oh we still haven't finished

00:29:04.900 --> 00:29:10.500
<v Carlton Gibson>this it's still in development it's still not ready and the performance implications are terrible

00:29:10.650 --> 00:29:17.580
<v Michael Kennedy>unless you do it 100 perfectly but you and there was regular back in um go back in um python three

00:29:17.740 --> 00:29:24.400
<v Michael Kennedy>five three six days it async and await and this async concept really had kind of a two to three

00:29:24.560 --> 00:29:30.020
<v Michael Kennedy>issue like i would even if i wanted to do it i got to call this library which is talking to the

00:29:30.200 --> 00:29:36.260
<v Michael Kennedy>internet or into the database but it's not async so what's the point yes yes and the costs of so

00:29:36.540 --> 00:29:42.020
<v Carlton Gibson>the the big thing we'll come and talk about a lot is threats so in um django uh we use a lot of what

00:29:42.080 --> 00:29:48.820
<v Carlton Gibson>of a helper um from the asgirf library called sync to async where we um you take async function and

00:29:48.920 --> 00:29:53.040
<v Carlton Gibson>you you wrap it into a call routine so that you can wait into a coroutine so you can await that

00:29:53.100 --> 00:29:58.800
<v Carlton Gibson>and it gets run in a thread and async io has the two thread thing which is the same um but essentially

00:29:58.860 --> 00:30:05.000
<v Carlton Gibson>the same option um sync to async has some ability to track which thread it was called from so for

00:30:05.100 --> 00:30:11.180
<v Carlton Gibson>instance database connections they're tied to a thread um and the transaction state is all tied

00:30:11.180 --> 00:30:17.360
<v Carlton Gibson>to that thread so it's important that database um queries are run on the same thread as the connection

00:30:17.420 --> 00:30:23.960
<v Carlton Gibson>came from. Otherwise, you're going to have problems. Sync2AdsAsync adds a few niceties

00:30:24.070 --> 00:30:29.800
<v Carlton Gibson>around that, but it's essentially the same as AsyncIO2 thread. Go and run this in an executor

00:30:29.920 --> 00:30:36.020
<v Carlton Gibson>on a separate thread. That's great, but that used to cost orders of milliseconds to bootstrap and

00:30:36.140 --> 00:30:41.600
<v Carlton Gibson>set up. With latest pythons, it's tens of nanoseconds, or tens of microseconds, sorry.

00:30:42.720 --> 00:30:46.460
<v Carlton Gibson>Tens of microseconds when you're doing milliseconds requests, they're not a significant thing,

00:30:46.620 --> 00:30:49.800
<v Carlton Gibson>But the documentation was like, no, this is going to slow down your requests.

00:30:49.900 --> 00:30:52.600
<v Carlton Gibson>And it's like, hang on, it's really not.

00:30:53.559 --> 00:30:55.280
<v Carlton Gibson>And the API is complete.

00:30:55.860 --> 00:31:01.520
<v Carlton Gibson>The ORM, the cache framework, authentication, session, signals, it's all got async API.

00:31:01.640 --> 00:31:03.600
<v Michael Kennedy>So from a user perspective, it really is all there.

00:31:03.900 --> 00:31:05.500
<v Carlton Gibson>It doesn't mean there's no work to be done.

00:31:06.040 --> 00:31:07.080
<v Carlton Gibson>It's an ongoing project.

00:31:07.780 --> 00:31:09.100
<v Carlton Gibson>We keep developing it.

00:31:09.100 --> 00:31:10.180
<v Carlton Gibson>We keep fixing new things.

00:31:10.200 --> 00:31:11.080
<v Carlton Gibson>We keep adding new things.

00:31:11.500 --> 00:31:19.280
<v Carlton Gibson>But this idea that the topic guy presented that somehow the story wasn't finished, well, that's not quite, that wasn't quite right.

00:31:19.380 --> 00:31:20.500
<v Carlton Gibson>So we wanted to update that.

00:31:20.560 --> 00:31:24.200
<v Carlton Gibson>And we wanted to take some of the warnings about performance and just downgrade them to notes.

00:31:24.300 --> 00:31:27.260
<v Carlton Gibson>You know, they're a bit overplayed in this latter day.

00:31:27.420 --> 00:31:28.600
<v Michael Kennedy>I agree.

00:31:28.780 --> 00:31:32.220
<v Michael Kennedy>I think the sync to async and back and forth is pretty interesting.

00:31:32.360 --> 00:31:38.620
<v Michael Kennedy>And more broadly, you know, one of the things people use as a, I don't really want to use async.

00:31:38.740 --> 00:31:39.420
<v Michael Kennedy>It's kind of bad.

00:31:39.600 --> 00:31:43.380
<v Michael Kennedy>they'll call it like a virus or something and a lot of times what they mean is like well you've

00:31:43.480 --> 00:31:47.540
<v Michael Kennedy>got some low level function it's async so in order to call that the thing above it has to be async

00:31:47.570 --> 00:31:51.720
<v Michael Kennedy>which means like the rest of your app like it just cascades you know it's sort of viral in the gpl

00:31:51.880 --> 00:31:56.620
<v Michael Kennedy>sense right um but yes that's a you don't have to live with that you can yeah go ahead sorry well

00:31:56.820 --> 00:32:01.300
<v Carlton Gibson>they call it function coloring problem right is that once you've got a blue function or red

00:32:01.450 --> 00:32:07.800
<v Carlton Gibson>function all the functions are called it have to be the same color yes but but you there's nothing

00:32:07.820 --> 00:32:13.440
<v Michael Kennedy>to stop you from going in a synchronous function just async to sync boom and like kind of just stop

00:32:13.440 --> 00:32:19.920
<v Michael Kennedy>the propagation there or create your own async io event loop and just run that one bit down below and

00:32:20.000 --> 00:32:24.540
<v Michael Kennedy>so you could have a synchronous function but it maybe calls five external endpoints and you could

00:32:24.740 --> 00:32:29.780
<v Michael Kennedy>do all that with async in parallel and just like run that but cap it right there i think there's a

00:32:29.900 --> 00:32:34.080
<v Michael Kennedy>lot more creativity that we can bring just just just say like oh it's virus either it's amazing or

00:32:34.060 --> 00:32:35.060
<v Michael Kennedy>we don't want it at all, you know?

00:32:35.140 --> 00:32:35.240
<v Michael Kennedy>Yes.

00:32:35.640 --> 00:32:37.040
<v Carlton Gibson>No, that example,

00:32:37.340 --> 00:32:38.200
<v Carlton Gibson>that example you gave

00:32:38.220 --> 00:32:38.860
<v Carlton Gibson>of calling five,

00:32:39.240 --> 00:32:42.040
<v Carlton Gibson>you know, five external things concurrently,

00:32:42.260 --> 00:32:44.860
<v Carlton Gibson>that's one of the easiest ways

00:32:45.040 --> 00:32:45.820
<v Carlton Gibson>into AsyncVagenda.

00:32:45.820 --> 00:32:46.920
<v Carlton Gibson>So you've got your WSGI app,

00:32:46.960 --> 00:32:47.560
<v Carlton Gibson>it's deployed,

00:32:47.800 --> 00:32:48.740
<v Carlton Gibson>it's got its,

00:32:49.300 --> 00:32:50.020
<v Carlton Gibson>you've run Gunicorn,

00:32:50.280 --> 00:32:50.940
<v Carlton Gibson>it's got, you know,

00:32:51.140 --> 00:32:51.880
<v Carlton Gibson>four workers running,

00:32:52.100 --> 00:32:53.160
<v Carlton Gibson>saying handling requests,

00:32:53.400 --> 00:32:54.440
<v Carlton Gibson>each one in a single thread,

00:32:54.660 --> 00:32:55.520
<v Carlton Gibson>nice and separated,

00:32:55.960 --> 00:32:56.740
<v Carlton Gibson>so they're not getting

00:32:56.880 --> 00:32:57.520
<v Carlton Gibson>in each other's ways,

00:32:58.180 --> 00:32:59.320
<v Carlton Gibson>sorry, in a separate process,

00:32:59.540 --> 00:33:00.860
<v Carlton Gibson>each kind of single threaded,

00:33:01.120 --> 00:33:01.880
<v Carlton Gibson>doing its thing.

00:33:02.320 --> 00:33:02.860
<v Carlton Gibson>And then you're like,

00:33:02.940 --> 00:33:03.520
<v Carlton Gibson>I want to make some,

00:33:04.760 --> 00:33:05.640
<v Carlton Gibson>API requests.

00:33:06.130 --> 00:33:09.540
<v Carlton Gibson>So you can define a single async view

00:33:09.920 --> 00:33:12.820
<v Carlton Gibson>and Django's handlers will run that

00:33:12.980 --> 00:33:14.520
<v Carlton Gibson>in an event loop for you

00:33:14.740 --> 00:33:17.900
<v Carlton Gibson>and you can run, say, your five HTTP requests

00:33:17.970 --> 00:33:19.240
<v Carlton Gibson>and you can aggregate them together

00:33:19.380 --> 00:33:20.520
<v Carlton Gibson>and you can return a single response.

00:33:21.060 --> 00:33:23.500
<v Carlton Gibson>And that can just be done in a fraction of the time

00:33:24.580 --> 00:33:26.860
<v Carlton Gibson>of calling one and then the other.

00:33:26.940 --> 00:33:27.660
<v Michael Kennedy>Yeah, yeah, yeah.

00:33:27.850 --> 00:33:31.220
<v Michael Kennedy>And it's just so clearly a win, right?

00:33:31.580 --> 00:33:37.580
<v Michael Kennedy>Do you have to switch to an ASGI, an ASGI server mechanism if you're going to use async views in Django?

00:33:37.920 --> 00:33:40.860
<v Carlton Gibson>No, you can run async views under WSGI.

00:33:41.280 --> 00:33:45.520
<v Carlton Gibson>But what you don't get, this is a good question.

00:33:46.000 --> 00:33:51.320
<v Carlton Gibson>So say I'm deployed, say I'm a Django developer, when should I switch to ASCII?

00:33:51.460 --> 00:33:52.700
<v Carlton Gibson>When should I switch to async?

00:33:53.100 --> 00:33:56.080
<v Carlton Gibson>And the real question is if you want long-lived requests.

00:33:56.700 --> 00:34:06.100
<v Carlton Gibson>If you want to do, say, something where you hold open a connection and you, say, do service center events where every five seconds you get an update.

00:34:06.700 --> 00:34:08.320
<v Carlton Gibson>Like a stock ticker is a good example, right?

00:34:08.379 --> 00:34:11.040
<v Carlton Gibson>You get what was the latest stock price.

00:34:11.460 --> 00:34:13.280
<v Carlton Gibson>Okay, it keeps coming down the wire at you.

00:34:13.620 --> 00:34:19.940
<v Carlton Gibson>Well, you don't want to do that on the WSGI model because each connection is a worker on WSGI.

00:34:19.980 --> 00:34:21.300
<v Carlton Gibson>And so you say you've got four workers.

00:34:21.480 --> 00:34:22.200
<v Carlton Gibson>Well, that's great.

00:34:22.280 --> 00:34:23.379
<v Carlton Gibson>But then, well, hang on.

00:34:23.520 --> 00:34:25.659
<v Carlton Gibson>I've got three of those held open by clients.

00:34:25.860 --> 00:34:27.960
<v Carlton Gibson>well, now I've only got one serving requests.

00:34:28.500 --> 00:34:30.260
<v Carlton Gibson>Well, okay, double your workers.

00:34:30.350 --> 00:34:32.220
<v Carlton Gibson>But then you're going to double your users.

00:34:32.659 --> 00:34:35.639
<v Carlton Gibson>And if you're using pre-forked workers

00:34:35.669 --> 00:34:37.300
<v Carlton Gibson>where each worker lives in a separate process,

00:34:37.520 --> 00:34:39.399
<v Carlton Gibson>that's a lot of times that your Django application

00:34:39.760 --> 00:34:43.560
<v Carlton Gibson>is loaded into RAM and say the cache is primed

00:34:43.639 --> 00:34:46.520
<v Carlton Gibson>or the database connections are set up.

00:34:47.280 --> 00:34:49.580
<v Carlton Gibson>There's a lot of overhead to running multiple processes.

00:34:50.030 --> 00:34:51.600
<v Carlton Gibson>Whereas if you can do it with a NASGI worker,

00:34:51.720 --> 00:34:54.200
<v Carlton Gibson>you can hold open that connection

00:34:54.600 --> 00:34:56.620
<v Carlton Gibson>and it can just be served on the event loop.

00:34:56.620 --> 00:34:58.880
<v Carlton Gibson>You can have many connections just for one worker.

00:34:58.880 --> 00:35:00.740
<v Michael Kennedy>It's basically tied to an event loop

00:35:00.790 --> 00:35:04.060
<v Michael Kennedy>and then that can share that thread or that worker

00:35:04.280 --> 00:35:06.480
<v Michael Kennedy>as it interlaces across that one event loop, right?

00:35:06.490 --> 00:35:07.200
<v Michael Kennedy>Yeah, exactly.

00:35:07.390 --> 00:35:09.260
<v Carlton Gibson>And that's where the exciting things come.

00:35:09.320 --> 00:35:11.180
<v Carlton Gibson>If you've got long-lived connections,

00:35:12.310 --> 00:35:13.240
<v Carlton Gibson>even if you're just doing polling,

00:35:13.370 --> 00:35:16.400
<v Carlton Gibson>you can do polling quite quickly, once a second,

00:35:16.710 --> 00:35:19.660
<v Carlton Gibson>and you're not really going to overload your whiskey workers.

00:35:20.140 --> 00:35:21.660
<v Carlton Gibson>But if you're holding a connection open,

00:35:22.040 --> 00:35:26.260
<v Carlton Gibson>Ah, at that point, Async and ASCII is really called for.

00:35:26.790 --> 00:35:27.680
<v Carlton Gibson>Yeah, that's cool.

00:35:27.800 --> 00:35:30.260
<v Carlton Gibson>So you can kind of have this progressive scale.

00:35:30.620 --> 00:35:31.100
<v Carlton Gibson>Yeah, yeah.

00:35:31.850 --> 00:35:39.400
<v Carlton Gibson>I mean, standardly, the people that, I mean, we have an awful lot of people doing very exciting things, you know, with Async.

00:35:39.720 --> 00:35:48.720
<v Carlton Gibson>They're either running small projects where a single ASCII worker is enough and it can handle enough connections and you don't actually need multiple processes.

00:35:49.320 --> 00:35:53.460
<v Carlton Gibson>Or they tend to be running at the moment a WSGI base for their app,

00:35:53.560 --> 00:35:56.600
<v Carlton Gibson>so a standard WSGI base, which is handling most of the work.

00:35:56.980 --> 00:36:00.600
<v Carlton Gibson>And then the async endpoints are just running in a single ASCII process

00:36:00.690 --> 00:36:01.720
<v Carlton Gibson>as a sort of sidecar.

00:36:01.930 --> 00:36:06.240
<v Carlton Gibson>And they do this because something we'll get into is you get what's called

00:36:06.910 --> 00:36:10.560
<v Carlton Gibson>you get kind of contention for CPU-bound work on the event.

00:36:11.520 --> 00:36:12.440
<v Carlton Gibson>And we can talk about that in a bit.

00:36:12.560 --> 00:36:16.580
<v Carlton Gibson>But I think that's the way people are deploying most successfully

00:36:17.110 --> 00:36:18.680
<v Michael Kennedy>at the slightly larger applications.

00:36:19.020 --> 00:36:36.040
<v Michael Kennedy>Yeah, that makes sense. And this contention, it is very interesting. It certainly is. I want it actually, I searched for the wrong thing. Hold on. That is, I wanted to actually show. I recently went through this whole process of like, how could I make my server use less memory?

00:36:36.420 --> 00:36:44.220
<v Michael Kennedy>brought and i had some of my web apps were running purely synchronous as whiskey and some of them i

00:36:44.300 --> 00:36:51.340
<v Michael Kennedy>had upgraded to like true async running on asgi on granian right and i'm just i was looking the

00:36:51.360 --> 00:36:56.720
<v Michael Kennedy>server i'm like it's using i don't know nine gigs of ram for all these problems like 27 containers

00:36:56.900 --> 00:37:01.799
<v Michael Kennedy>running but still it's a lot i'm like is there a way that i could just maybe reduce the workers

00:37:01.800 --> 00:37:03.380
<v Michael Kennedy>a little bit and make it chill.

00:37:03.690 --> 00:37:06.080
<v Michael Kennedy>And so I wrote this article about like going through

00:37:06.190 --> 00:37:07.180
<v Michael Kennedy>all these different things.

00:37:07.630 --> 00:37:10.820
<v Michael Kennedy>But one of the things that I found really helpful was

00:37:11.180 --> 00:37:14.900
<v Michael Kennedy>if I could switch to more of the processes,

00:37:15.440 --> 00:37:19.020
<v Michael Kennedy>more of the requests being processed by an ASGI server

00:37:19.300 --> 00:37:21.440
<v Michael Kennedy>with like truly async calls

00:37:21.530 --> 00:37:23.740
<v Michael Kennedy>and down to the database layer and so on,

00:37:23.980 --> 00:37:25.420
<v Michael Kennedy>maybe I can use fewer workers.

00:37:25.980 --> 00:37:29.940
<v Michael Kennedy>And so I like, just off of like one of my web apps,

00:37:29.980 --> 00:37:36.020
<v Michael Kennedy>I cut like 500 megs off because I could just reduce the number of workers because each one was more capable and didn't get bound up.

00:37:36.160 --> 00:37:42.540
<v Michael Kennedy>There's still this computational problem you have, but at least you're not doing like I'm waiting a second for this API endpoint, you know?

00:37:42.760 --> 00:37:43.860
<v Michael Kennedy>Yes, yes, yes.

00:37:44.120 --> 00:37:45.100
<v Michael Kennedy>I thought that was pretty interesting.

00:37:45.600 --> 00:37:46.340
<v Carlton Gibson>No, it is.

00:37:46.360 --> 00:37:49.400
<v Carlton Gibson>And it's like people, it's one reason why people run.

00:37:49.640 --> 00:37:59.900
<v Carlton Gibson>So I always run, not always, but I default to running, say, Gun & Corn with multiple workers, multiple processes as workers because it's separate and you don't get the guild contention issues.

00:38:00.280 --> 00:38:08.380
<v Carlton Gibson>But there is a great approach, depending on how much of this CPU work that you're busy doing and how the guild contention comes into play.

00:38:08.920 --> 00:38:12.800
<v Carlton Gibson>But if you run with the threaded worker, you've just got one process.

00:38:13.380 --> 00:38:18.160
<v Carlton Gibson>And you've got much less memory overhead from running one process than you have from running four.

00:38:18.160 --> 00:38:20.300
<v Carlton Gibson>Shared initialization and all that.

00:38:20.780 --> 00:38:21.680
<v Carlton Gibson>Connection pooling, everything.

00:38:22.000 --> 00:38:22.800
<v Carlton Gibson>Yeah, all of it.

00:38:23.760 --> 00:38:26.820
<v Carlton Gibson>But it's never as simple as, oh, this is the way to deploy.

00:38:27.880 --> 00:38:29.340
<v Carlton Gibson>How do you deploy your application?

00:38:29.600 --> 00:38:33.560
<v Carlton Gibson>how long you got and how many developers are we asking because we'll all give different answers

00:38:33.940 --> 00:38:38.720
<v Carlton Gibson>for all these really interesting reasons that trade off against each other yeah something i

00:38:38.790 --> 00:38:43.920
<v Michael Kennedy>would like to get your thoughts on is i feel like sometimes people say like why are we making python

00:38:43.970 --> 00:38:49.760
<v Michael Kennedy>so complicated like the magic of python was that it was simple and yet there's also another side of

00:38:49.760 --> 00:38:55.800
<v Michael Kennedy>that story where people's like well i have to switch to go because i can't i can't do this and so you

00:38:55.820 --> 00:39:00.500
<v Michael Kennedy>got to walk this tight balance of there has to be a simple story but you don't want there to not be

00:39:00.620 --> 00:39:05.640
<v Michael Kennedy>a nice upper bound or high upper bound because you're going to lose maybe your your biggest

00:39:05.900 --> 00:39:11.000
<v Michael Kennedy>proponents or your your biggest users right yeah what do you think about that um no i think look

00:39:11.200 --> 00:39:17.200
<v Carlton Gibson>python so you know i this fits my brain idea i think it's something i'm very into and one of my

00:39:17.340 --> 00:39:23.780
<v Carlton Gibson>sort of grumbles about the type about typing is that it becomes not pipe it fits your brain i mean

00:39:23.800 --> 00:39:26.660
<v Carlton Gibson>And, you know, I was told a story at PyCon Italia,

00:39:26.750 --> 00:39:32.320
<v Carlton Gibson>I showed five typing experts sat around a table trying to work out the correct type in.

00:39:32.530 --> 00:39:34.640
<v Carlton Gibson>And these are the absolute gurus, right?

00:39:34.780 --> 00:39:36.580
<v Carlton Gibson>And that's not fit your brain, is it?

00:39:38.680 --> 00:39:42.620
<v Carlton Gibson>So part of me is like, ah, on that front.

00:39:42.860 --> 00:39:44.460
<v Carlton Gibson>But we have to be performant.

00:39:44.490 --> 00:39:48.780
<v Carlton Gibson>You don't want to have to change language just because you need to increase your throughput,

00:39:49.000 --> 00:39:51.360
<v Carlton Gibson>just because you're doing a more serious application.

00:39:51.740 --> 00:39:56.440
<v Carlton Gibson>That's not good for, that wouldn't be good for long-term sustainability of the project.

00:39:56.440 --> 00:39:57.540
<v Michael Kennedy>No, it definitely wouldn't.

00:39:58.020 --> 00:40:03.160
<v Michael Kennedy>And the fact that you don't have to add typing, you don't have to write async views.

00:40:03.600 --> 00:40:07.500
<v Michael Kennedy>I think that's part of where it's really good because you can grow into that.

00:40:07.630 --> 00:40:08.620
<v Michael Kennedy>There is space to grow into.

00:40:08.980 --> 00:40:15.740
<v Carlton Gibson>There is, though, a kind of pressure to the original DEP, which added type hints to Python.

00:40:16.020 --> 00:40:23.760
<v Carlton Gibson>said, we don't intend to make type hints compulsory, even by convention. But I think there is very much

00:40:23.800 --> 00:40:29.240
<v Carlton Gibson>people coming into the Python landscape. There is a cultural pressure to adopt typing, even if perhaps

00:40:29.320 --> 00:40:35.240
<v Carlton Gibson>it's not relevant to you. It's like, oh, no, but this is how you do it. And there is a you should,

00:40:35.560 --> 00:40:40.320
<v Carlton Gibson>which perhaps doesn't respect that original promise in the dead.

00:40:40.500 --> 00:40:45.160
<v Michael Kennedy>Fair. And I think AI throws a whole spanner into it as well. And let's not go too deeply

00:40:45.180 --> 00:40:49.000
<v Michael Kennedy>down this because we can spend the next three hours on but you know if you ask an ai to write

00:40:49.140 --> 00:40:55.500
<v Michael Kennedy>python a lot of times it'll do typed python now sure without asking yes and um the big issue

00:40:55.780 --> 00:41:02.840
<v Carlton Gibson>with um ai output is can you verify it and so you know type safety any type safety that you get from

00:41:02.960 --> 00:41:09.060
<v Carlton Gibson>the type checker is going to help the ai um verify the output or help you verify the output 100 i

00:41:09.220 --> 00:41:13.180
<v Michael Kennedy>think it's a big bonus to ai as well whether yes people care about that you know that's a different

00:41:13.200 --> 00:41:21.400
<v Michael Kennedy>deal. Indeed. So what's left? Where else do people need to focus? Okay, so performance realities is,

00:41:22.470 --> 00:41:28.820
<v Carlton Gibson>you know, DB parallelism is your limit, right? So each DB connection is a ordered, like,

00:41:29.310 --> 00:41:34.200
<v Carlton Gibson>it takes, you know, it takes a query and it gets a response. And so whether you're coming,

00:41:34.640 --> 00:41:39.200
<v Carlton Gibson>whether you're doing that in threads, the way Django does it, or if you're doing it with greenlets,

00:41:39.380 --> 00:41:43.580
<v Carlton Gibson>the way some other ORMs do it, you're still limited by the size of your connection pool.

00:41:45.590 --> 00:41:49.340
<v Carlton Gibson>Again, and that's whether you're doing it with WSGI or ASCII, the limit always has

00:41:49.430 --> 00:41:53.620
<v Carlton Gibson>been the number of database connections you could hold open at a time, basically.

00:41:54.150 --> 00:41:55.580
<v Carlton Gibson>So that doesn't really change anything.

00:41:55.630 --> 00:41:59.920
<v Carlton Gibson>If you want to scale up, you need to increase the size of your connection pool or handle

00:41:59.970 --> 00:42:00.740
<v Carlton Gibson>that more efficiently.

00:42:01.920 --> 00:42:02.500
<v Carlton Gibson>That's one thing.

00:42:02.780 --> 00:42:10.600
<v Carlton Gibson>Async transactions is something that, because of the way Django's connections are tied to a thread local,

00:42:11.900 --> 00:42:19.180
<v Carlton Gibson>we're never going to have async transactions in the sense that you can call await inside an atomic block.

00:42:19.320 --> 00:42:24.600
<v Carlton Gibson>Why not? Because that await call may not be on the same thread as the connection, right?

00:42:24.980 --> 00:42:28.700
<v Carlton Gibson>So the transaction state is held on the connection, which is on a thread,

00:42:29.140 --> 00:42:31.720
<v Carlton Gibson>and if that await call handles in a different thread, you're going to be in big trouble.

00:42:32.000 --> 00:42:36.300
<v Carlton Gibson>So it's always going to be the case with Django's choice that you need to take if you want.

00:42:36.470 --> 00:42:38.800
<v Carlton Gibson>It doesn't mean you can't have transactions in async.

00:42:38.880 --> 00:42:45.860
<v Carlton Gibson>You just wrap your transaction in a single function, which you then call within a single sync to async.

00:42:47.220 --> 00:42:48.960
<v Carlton Gibson>So that's always going to be the case.

00:42:49.380 --> 00:42:50.860
<v Carlton Gibson>Then there's CPU band work.

00:42:51.820 --> 00:42:58.060
<v Carlton Gibson>Look, if you do work on CPU, you either block the event loop or you throw it to a thread.

00:42:58.360 --> 00:43:02.660
<v Carlton Gibson>And then, you know, if there's a lot of CPU work going on, you've got GIL contention that we'll talk about momentarily.

00:43:03.280 --> 00:43:05.240
<v Carlton Gibson>Or you have to throw it out to a separate process.

00:43:05.600 --> 00:43:08.780
<v Carlton Gibson>But throwing work out to a separate process is just crazily heavy.

00:43:08.940 --> 00:43:10.080
<v Carlton Gibson>You've got to bootstrap that process.

00:43:10.600 --> 00:43:11.840
<v Carlton Gibson>You've got to marshal your data.

00:43:11.930 --> 00:43:12.780
<v Carlton Gibson>You've got to work out returns.

00:43:13.220 --> 00:43:16.880
<v Carlton Gibson>So, you know, async frameworks have tools to throw out to a separate process.

00:43:17.030 --> 00:43:19.520
<v Carlton Gibson>But it's not something you want to address lightly.

00:43:19.720 --> 00:43:22.280
<v Carlton Gibson>Really, really, threading was the way you wanted to go.

00:43:22.700 --> 00:43:25.720
<v Michael Kennedy>But you are going to hit the gill, essentially.

00:43:25.940 --> 00:43:26.340
<v Michael Kennedy>Yeah, yeah.

00:43:26.500 --> 00:43:35.120
<v Michael Kennedy>For people who don't know, if by default, when you use async and await on a standard event loop, that's single threaded.

00:43:35.470 --> 00:43:38.140
<v Michael Kennedy>Even though stuff is happening, it's literally single threaded.

00:43:38.340 --> 00:43:48.620
<v Michael Kennedy>And the way that it time slices it or preemptively shares it is anytime you hit an await block, like on a database call or whatever, it's like, okay, that's waiting.

00:43:48.630 --> 00:43:50.900
<v Michael Kennedy>So now the same thread can go process something else.

00:43:51.040 --> 00:43:57.960
<v Michael Kennedy>But soon as you're back in regular Python, just jamming on something, well, there's no way to time slice that, right?

00:43:58.020 --> 00:44:00.340
<v Michael Kennedy>So everything else is waiting on that computationally.

00:44:00.620 --> 00:44:02.460
<v Carlton Gibson>And you end up with really horrible things.

00:44:03.020 --> 00:44:04.260
<v Carlton Gibson>And I'm going to say it's horrible.

00:44:04.340 --> 00:44:05.540
<v Carlton Gibson>I think it really is horrible, right?

00:44:05.680 --> 00:44:08.560
<v Carlton Gibson>It's where you've got a loop of CPU-heavy work.

00:44:08.680 --> 00:44:14.060
<v Carlton Gibson>I know you're rendering it and you add in an asyncio sleep zero just to release work.

00:44:14.200 --> 00:44:15.800
<v Carlton Gibson>And it's like, why are we doing that?

00:44:15.900 --> 00:44:19.540
<v Carlton Gibson>We're doing that because we're running CPU work on the event loop.

00:44:19.660 --> 00:44:21.280
<v Carlton Gibson>The event loop wasn't for that.

00:44:21.320 --> 00:44:23.820
<v Carlton Gibson>It was for our handling I.O., by async I.O.

00:44:25.420 --> 00:44:30.140
<v Carlton Gibson>And we really want to throw CPU bound work to a thread.

00:44:30.280 --> 00:44:31.240
<v Carlton Gibson>We just want to.

00:44:31.540 --> 00:44:32.820
<v Carlton Gibson>But then we've got Python's guild.

00:44:32.940 --> 00:44:34.760
<v Carlton Gibson>So this is where I'm super excited.

00:44:34.760 --> 00:44:37.840
<v Michael Kennedy>It effectively says, well, you have as many threads as you want, but you still can run

00:44:37.880 --> 00:44:38.420
<v Michael Kennedy>one at a time.

00:44:39.060 --> 00:44:39.640
<v Carlton Gibson>Yeah, exactly.

00:44:40.420 --> 00:44:40.740
<v Carlton Gibson>Either way.

00:44:41.000 --> 00:44:46.759
<v Carlton Gibson>So either you do the CPU work on your one thread, which is running the event loop, or you throw

00:44:46.780 --> 00:44:51.100
<v Carlton Gibson>it out to different threads and they get GIL contention and so you end up with essentially

00:44:51.400 --> 00:44:57.040
<v Carlton Gibson>one cpu or one threads worth of clock time no matter which way you cut it which is python's

00:44:57.270 --> 00:45:02.080
<v Carlton Gibson>real problem and it has you know it it's not actually about switching threads it's not actually

00:45:02.090 --> 00:45:06.460
<v Carlton Gibson>about creating threads it's not actually about spinning up event loops those are relatively quick

00:45:06.660 --> 00:45:13.579
<v Carlton Gibson>things it's about the inability to genuinely run parallel work in in multiple threads in python up

00:45:13.600 --> 00:45:19.340
<v Michael Kennedy>until very recently. Yeah, it's another axis of that. You probably don't need it, but it's a low

00:45:19.600 --> 00:45:24.520
<v Michael Kennedy>upper bound that shouldn't be there for people who need to grow into it. Yes, entirely. And you

00:45:25.040 --> 00:45:32.760
<v Carlton Gibson>shouldn't need it. It can come in at quite moderate scales. Okay, my Hello World blog application,

00:45:33.020 --> 00:45:38.920
<v Carlton Gibson>I'm not going to hit gil contention with my 10 requests a minute. It's not a problem if I'm lucky.

00:45:39.800 --> 00:45:42.440
<v Carlton Gibson>Yeah, that's when it gets popular, yeah, in the article.

00:45:42.540 --> 00:45:43.060
<v Carlton Gibson>Yeah, yeah, yeah.

00:45:43.600 --> 00:45:44.420
<v Carlton Gibson>Oh, I hit Hacker News.

00:45:44.600 --> 00:45:45.480
<v Carlton Gibson>I've got 10 requests in a minute.

00:45:45.660 --> 00:45:49.080
<v Carlton Gibson>You know, you're not going to hit GitGil contention there.

00:45:49.120 --> 00:45:53.560
<v Carlton Gibson>But if you're, again, serious about using Python, a bigger application,

00:45:54.000 --> 00:45:56.420
<v Carlton Gibson>yes, you are going to have to do CPU work at some point.

00:45:56.620 --> 00:45:58.100
<v Carlton Gibson>Otherwise, the computer's not doing anything, right?

00:45:58.480 --> 00:45:59.880
<v Carlton Gibson>The program, if it's going to do something,

00:46:00.080 --> 00:46:01.660
<v Carlton Gibson>it's going to have to use the CPU at some point.

00:46:02.040 --> 00:46:03.120
<v Carlton Gibson>And so if you're doing that at scale,

00:46:04.420 --> 00:46:06.980
<v Carlton Gibson>the Git has been an impediment to that.

00:46:07.040 --> 00:46:08.640
<v Carlton Gibson>And people are, oh, I'm going to switch to Rast.

00:46:08.780 --> 00:46:09.540
<v Carlton Gibson>I'm going to switch to Go.

00:46:09.580 --> 00:46:11.680
<v Carlton Gibson>So do we have to switch language again?

00:46:12.040 --> 00:46:13.340
<v Carlton Gibson>Can we not do that?

00:46:13.580 --> 00:46:14.760
<v Michael Kennedy>Yeah, let's make it so we don't have to.

00:46:15.160 --> 00:46:18.540
<v Michael Kennedy>And you don't have to have an insane amount of requests

00:46:18.760 --> 00:46:19.540
<v Michael Kennedy>to benefit from this.

00:46:19.640 --> 00:46:21.980
<v Michael Kennedy>Kind of like my little example I just showed with the memory thing.

00:46:22.160 --> 00:46:26.160
<v Michael Kennedy>If you have to run enough workers that you needed two servers,

00:46:26.740 --> 00:46:31.500
<v Michael Kennedy>if this free threading thing and the async stuff

00:46:31.720 --> 00:46:34.520
<v Michael Kennedy>let you shrink that back down to one more manageable server,

00:46:34.560 --> 00:46:35.400
<v Michael Kennedy>it saves you on DevOps.

00:46:36.000 --> 00:46:37.160
<v Michael Kennedy>It saves you on complexity.

00:46:37.160 --> 00:46:37.840
<v Michael Kennedy>It saves your money.

00:46:38.080 --> 00:46:38.860
<v Michael Kennedy>It saves your money.

00:46:38.880 --> 00:46:40.960
<v Michael Kennedy>So if you want money, all these things.

00:46:41.140 --> 00:46:45.740
<v Michael Kennedy>So you don't have to be Instagram level of Django requests

00:46:46.010 --> 00:46:48.700
<v Michael Kennedy>to benefit from some of these ideas, right?

00:46:48.960 --> 00:46:49.780
<v Michael Kennedy>No, absolutely not.

00:46:49.960 --> 00:46:53.960
<v Carlton Gibson>I mean, it turns out, and I've always said for years,

00:46:54.240 --> 00:46:59.100
<v Carlton Gibson>with a medium-sized box, you can run a very high-traffic server

00:47:00.660 --> 00:47:01.520
<v Carlton Gibson>in realistic terms.

00:47:02.680 --> 00:47:06.100
<v Carlton Gibson>But the more you can do on the smaller box, absolutely the better.

00:47:06.320 --> 00:47:06.560
<v Carlton Gibson>Absolutely.

00:47:07.580 --> 00:47:09.940
<v Carlton Gibson>So three threading is the big win.

00:47:10.180 --> 00:47:15.020
<v Carlton Gibson>It's the big exciting thing because Django kind of looked clumsy almost when it said,

00:47:15.030 --> 00:47:16.880
<v Carlton Gibson>no, let's just put all the sync work into a thread.

00:47:18.620 --> 00:47:26.000
<v Carlton Gibson>But actually, as three threading becomes more established and it works through the ecosystem,

00:47:26.500 --> 00:47:28.280
<v Carlton Gibson>it looks as if Django's bet's going to pay off.

00:47:29.060 --> 00:47:30.640
<v Carlton Gibson>That bet will have aged very well.

00:47:31.380 --> 00:47:35.040
<v Carlton Gibson>Because we always really wanted to do CPU work in the thread.

00:47:36.600 --> 00:47:37.320
<v Carlton Gibson>I think it's great.

00:47:37.460 --> 00:47:39.880
<v Michael Kennedy>Yeah, I mean, you can have a thread pool.

00:47:40.250 --> 00:47:46.660
<v Michael Kennedy>Your threads could each have sort of a hanging around database connection for any time you need it, ready to go.

00:47:46.990 --> 00:47:50.120
<v Michael Kennedy>And yeah, you just delegate it out and you actually get parallelism.

00:47:50.310 --> 00:47:51.940
<v Michael Kennedy>Yeah, no, and that's the thing.

00:47:52.300 --> 00:47:55.240
<v Carlton Gibson>Actual parallelism within Python is really exciting.

00:47:56.190 --> 00:47:56.920
<v Carlton Gibson>And that's for everybody.

00:47:57.050 --> 00:48:01.000
<v Carlton Gibson>I mean, you know, to go back, why did we update the async topic docs?

00:48:01.140 --> 00:48:05.780
<v Carlton Gibson>It's because there are problems that remain with async, but they're not Django-specific problems.

00:48:06.120 --> 00:48:15.420
<v Carlton Gibson>The same GIL contention problems you have, you type FastAPI concurrency in, or scaling problems, and you read the same sort of problems.

00:48:15.820 --> 00:48:17.200
<v Carlton Gibson>And so it's not Django specific.

00:48:17.730 --> 00:48:19.840
<v Carlton Gibson>It's not a Django problem there.

00:48:20.000 --> 00:48:21.820
<v Carlton Gibson>It's a Python structural problem.

00:48:22.460 --> 00:48:23.600
<v Carlton Gibson>Same database scaling.

00:48:23.670 --> 00:48:24.140
<v Carlton Gibson>How do we do it?

00:48:24.140 --> 00:48:25.000
<v Carlton Gibson>What do database pools?

00:48:25.140 --> 00:48:27.240
<v Carlton Gibson>Well, same using Flask.

00:48:27.400 --> 00:48:29.540
<v Carlton Gibson>You're going to scale it that way, et cetera.

00:48:29.820 --> 00:48:33.200
<v Carlton Gibson>It's nice that these are general problems that we all share, I think.

00:48:33.420 --> 00:48:34.300
<v Michael Kennedy>Yes, absolutely.

00:48:34.640 --> 00:48:42.380
<v Michael Kennedy>yeah i mean and let's see i think let's open it no there we go i think it's just so easy these days

00:48:42.720 --> 00:48:48.360
<v Michael Kennedy>to try out free threaded python too it used to be oh you got to get this weird thing and build it or

00:48:48.390 --> 00:48:54.280
<v Michael Kennedy>whatever but now with uv it's just literally uv python install like what do you want do you want

00:48:54.330 --> 00:49:00.660
<v Michael Kennedy>three fourteen six t yeah that's all you're saying in two seconds you've got free threaded

00:49:00.680 --> 00:49:01.840
<v Michael Kennedy>Python to play with and try out.

00:49:02.060 --> 00:49:02.200
<v Carlton Gibson>Yes.

00:49:03.140 --> 00:49:05.540
<v Carlton Gibson>The thing here, there's still work to be done here.

00:49:06.900 --> 00:49:10.080
<v Carlton Gibson>Most of it is about the ecosystem, not about the architecture.

00:49:10.220 --> 00:49:11.660
<v Carlton Gibson>The architecture is already in Plex.

00:49:12.310 --> 00:49:19.020
<v Carlton Gibson>So if you're at an app level, because people have been running threaded Django for a long

00:49:19.200 --> 00:49:19.260
<v Carlton Gibson>time.

00:49:19.580 --> 00:49:22.760
<v Carlton Gibson>We're talking about people deploying Whiskey with threaded workers.

00:49:23.280 --> 00:49:24.200
<v Carlton Gibson>They've been doing that for a long time.

00:49:25.460 --> 00:49:30.220
<v Carlton Gibson>But standardly, at the app level, you don't touch things which are multi-thread.

00:49:30.500 --> 00:49:33.040
<v Carlton Gibson>You might have, I don't know, some module level caching.

00:49:33.420 --> 00:49:35.420
<v Carlton Gibson>You might have a property at the module level.

00:49:35.520 --> 00:49:36.140
<v Carlton Gibson>Well, hang on.

00:49:36.460 --> 00:49:37.480
<v Carlton Gibson>Is that really thread safe?

00:49:37.600 --> 00:49:38.620
<v Carlton Gibson>Do you need to wrap that in a lock?

00:49:38.980 --> 00:49:41.540
<v Carlton Gibson>But it's extensions, it's libraries.

00:49:42.020 --> 00:49:43.220
<v Carlton Gibson>Are they doing these kind of patterns?

00:49:43.300 --> 00:49:46.180
<v Carlton Gibson>And we just need to work through them and run all our test suites against them

00:49:46.180 --> 00:49:49.360
<v Carlton Gibson>and make sure that we're compatible with re-threading.

00:49:49.640 --> 00:49:54.520
<v Carlton Gibson>And then I think we're actually going to unlock an awful lot of performance for everybody.

00:49:55.260 --> 00:49:58.660
<v Carlton Gibson>So instead of having to run multiple processes to get parallelism,

00:49:58.960 --> 00:50:00.780
<v Carlton Gibson>you'll be able to run it all within one process.

00:50:01.360 --> 00:50:03.260
<v Michael Kennedy>You know, it's a really exciting thing.

00:50:03.540 --> 00:50:05.040
<v Michael Kennedy>Yeah, I have some really interesting experiments

00:50:05.150 --> 00:50:06.520
<v Michael Kennedy>I want to put together with free threading,

00:50:06.680 --> 00:50:08.820
<v Michael Kennedy>but the more important work is still out there.

00:50:09.300 --> 00:50:09.520
<v Michael Kennedy>Yeah.

00:50:10.079 --> 00:50:12.560
<v Michael Kennedy>With AI, I'm not ready to, like, go and work on these.

00:50:12.780 --> 00:50:15.360
<v Michael Kennedy>Like, I don't, I can't do another open source project yet.

00:50:15.680 --> 00:50:17.420
<v Carlton Gibson>Yeah, no, and you must avoid it

00:50:17.600 --> 00:50:18.540
<v Carlton Gibson>because the trouble is that,

00:50:19.440 --> 00:50:21.060
<v Carlton Gibson>I maintain a lot of packages, right?

00:50:21.320 --> 00:50:23.580
<v Carlton Gibson>And it's a burden.

00:50:23.880 --> 00:50:25.620
<v Carlton Gibson>It's a thing that once you've taken it on,

00:50:25.620 --> 00:50:26.680
<v Carlton Gibson>you can't just put that.

00:50:26.680 --> 00:50:28.080
<v Carlton Gibson>And you might think, oh, it's just a casual thing.

00:50:28.200 --> 00:50:32.920
<v Carlton Gibson>must be very conscious and careful about what you package up and put out there in the world because

00:50:32.920 --> 00:50:38.160
<v Michael Kennedy>you're making a promise to people like yeah carlton i'm thinking this is going to be a really good gist

00:50:38.280 --> 00:50:43.780
<v Michael Kennedy>not a repo you know what i mean yes right exactly put up a gist do a blog post put up a gist say

00:50:43.920 --> 00:50:48.900
<v Carlton Gibson>here it is but don't put it up as a as a package and unless you mean it if you mean it then it's a

00:50:49.060 --> 00:50:53.660
<v Carlton Gibson>package on pypi and you know all the rest people can install pip installs directly from github

00:50:53.680 --> 00:50:55.320
<v Carlton Gibson>You don't have often version control.

00:50:55.360 --> 00:50:58.900
<v Carlton Gibson>You don't have to put it on PyPI for people to be able to play with it.

00:50:59.080 --> 00:50:59.820
<v Carlton Gibson>Yeah, absolutely.

00:51:00.240 --> 00:51:04.500
<v Michael Kennedy>So one thing I want to talk to you about, I know you're excited about, is the built-in

00:51:04.820 --> 00:51:05.340
<v Michael Kennedy>task framework.

00:51:05.900 --> 00:51:11.200
<v Michael Kennedy>Because all of this stuff about threading and all of that, you know, parallelism and, you

00:51:11.200 --> 00:51:13.220
<v Michael Kennedy>know, event loops, it's fine.

00:51:13.240 --> 00:51:17.320
<v Michael Kennedy>But if you could just say, let's just run that somewhere else.

00:51:17.820 --> 00:51:18.820
<v Michael Kennedy>Don't hassle me.

00:51:19.120 --> 00:51:19.600
<v Michael Kennedy>Tell us about this.

00:51:19.780 --> 00:51:20.080
<v Carlton Gibson>Okay.

00:51:20.340 --> 00:51:26.040
<v Carlton Gibson>So this is a great question, because when we first added async to Django, people were like, oh, this is background tasks.

00:51:26.320 --> 00:51:27.800
<v Carlton Gibson>And it's like, no, no, no, it's not.

00:51:27.920 --> 00:51:32.020
<v Carlton Gibson>It's all about I.O. and multiple connections and long-lived requests and things like that.

00:51:32.040 --> 00:51:37.420
<v Carlton Gibson>It's nothing to do with having a queue where you want to do some asynchronous task.

00:51:37.540 --> 00:51:43.060
<v Carlton Gibson>So asynchronous is the word has appeared, but it means other than making people wait.

00:51:43.220 --> 00:51:48.700
<v Carlton Gibson>So the standard web, you've got a request and you send a response, and you want that to be as quick as you can.

00:51:49.000 --> 00:51:57.020
<v Carlton Gibson>But let's say somebody has a password reset and you want to send them an email so that they can click on the password reset link and they can reset their password.

00:51:57.380 --> 00:52:05.440
<v Carlton Gibson>Well, historically, you would have made the request, sent the email, wait for the email to send, which can take almost any amount of time.

00:52:06.000 --> 00:52:09.640
<v Carlton Gibson>And then only then do they get the HTTP response.

00:52:09.960 --> 00:52:11.120
<v Carlton Gibson>Well, that's a long time to wait.

00:52:11.240 --> 00:52:12.480
<v Carlton Gibson>That's a bad user experience, right?

00:52:12.800 --> 00:52:20.160
<v Carlton Gibson>The much better is to make the request, say, yes, we've accepted it, put it, we'll talk about where we put it in a moment, and just send the response quickly saying, look, it's on its way.

00:52:20.200 --> 00:52:21.300
<v Carlton Gibson>Give it a few seconds to arrive.

00:52:21.900 --> 00:52:22.840
<v Michael Kennedy>It's already email.

00:52:23.100 --> 00:52:23.880
<v Michael Kennedy>It's already a single.

00:52:23.960 --> 00:52:25.540
<v Michael Kennedy>There's nothing you can do to wait for delivery.

00:52:26.180 --> 00:52:26.560
<v Carlton Gibson>Yeah, yeah, yeah.

00:52:26.780 --> 00:52:28.580
<v Carlton Gibson>So all you can do, though, is send.

00:52:28.680 --> 00:52:34.440
<v Carlton Gibson>So what you do now is you quickly write a task to, say, the database with the Django task framework.

00:52:34.760 --> 00:52:42.420
<v Carlton Gibson>And you would send that, there'd be a worker process running totally separately, which could then send out the email, say, for instance.

00:52:42.920 --> 00:52:48.860
<v Carlton Gibson>Now, people have historically done this with either Celery or RQ or Django Q or then Django Q2.

00:52:49.090 --> 00:52:53.000
<v Carlton Gibson>And there are Huey, there's a billion of these task queues out there.

00:52:53.580 --> 00:52:58.520
<v Carlton Gibson>And problem there is that, say, Django itself couldn't rely on them.

00:52:58.810 --> 00:53:01.020
<v Carlton Gibson>And neither could any third party package rely on them.

00:53:01.080 --> 00:53:05.120
<v Carlton Gibson>Well, I could, but then I'd have to commit to, we're using Django Q2 for this.

00:53:05.500 --> 00:53:08.900
<v Carlton Gibson>And all the people who were using Celery or RQ or Huey, they'd be excluded.

00:53:09.110 --> 00:53:26.740
<v Carlton Gibson>And so what Django Tasks does is to provide a kind of back-end API that says, look, any package can use this API to enqueue a task and to check results and do the core activities that you need to do around asynchronous tasks.

00:53:27.120 --> 00:53:34.720
<v Carlton Gibson>And then just by the user, just by configuring in the settings what their back end is, can either go for Celery or RQ or it depends.

00:53:35.340 --> 00:53:42.140
<v Carlton Gibson>As long as those packages have implemented a Django tasks adapter, you can use those for the back end.

00:53:42.220 --> 00:53:47.200
<v Carlton Gibson>And so I think, for instance, Celery hasn't yet done it, but there's an open issue and it will just it will become.

00:53:47.440 --> 00:53:49.680
<v Carlton Gibson>But there are a couple of back ends that are available.

00:53:49.840 --> 00:53:55.320
<v Carlton Gibson>There's a Redis one, there's one that uses the ORM to just store in your database and you can get going with those now.

00:53:55.580 --> 00:54:06.220
<v Carlton Gibson>And for me as a third-party package maintainer, that's fantastic because now I can add tasks to my third-party package and users can benefit from those without me having to know anything about which backend they're using.

00:54:06.420 --> 00:54:06.600
<v Carlton Gibson>Yeah.

00:54:06.800 --> 00:54:06.920
<v Michael Kennedy>Yeah.

00:54:07.620 --> 00:54:15.260
<v Michael Kennedy>Your pip install or uv install instructions for your package aren't like bracket celery, bracket, whatever.

00:54:15.700 --> 00:54:17.820
<v Michael Kennedy>And if none of those are there, it's going to throw an acceptance.

00:54:18.000 --> 00:54:20.060
<v Michael Kennedy>Well, you didn't install any of the queuing options.

00:54:20.280 --> 00:54:23.400
<v Michael Kennedy>So now we're busted, you know, like, yeah, you can just delegate.

00:54:23.660 --> 00:54:23.800
<v Michael Kennedy>Yeah.

00:54:24.280 --> 00:54:25.500
<v Carlton Gibson>And Cellerie's wonderful.

00:54:26.060 --> 00:54:29.000
<v Carlton Gibson>It really is a truly wonderful piece of software.

00:54:29.110 --> 00:54:33.200
<v Carlton Gibson>But it's literally overkill for 99% of projects.

00:54:33.410 --> 00:54:35.800
<v Carlton Gibson>And it's operationally difficult to keep going.

00:54:36.160 --> 00:54:37.800
<v Carlton Gibson>You end up running three or four different things.

00:54:37.850 --> 00:54:42.160
<v Carlton Gibson>You need a monitor, a beat server, the actual worker.

00:54:43.560 --> 00:54:48.700
<v Carlton Gibson>It's designed for a level of sophistication that most applications really don't need.

00:54:48.840 --> 00:54:53.160
<v Michael Kennedy>I love that you call out the operational complexity as well.

00:54:53.340 --> 00:55:05.080
<v Michael Kennedy>Because like you say, it would be nice to just let people who don't have something complicated use this feature without going, yeah, look, I know you used to, look, that was fun when you could just put it on a platform as a service thing and let it run.

00:55:05.510 --> 00:55:14.780
<v Michael Kennedy>But now what we're doing is we're setting up a private network and now we've got poison messages and we've got downtime because the celery version had to change.

00:55:14.880 --> 00:55:15.940
<v Michael Kennedy>Oh my gosh.

00:55:16.000 --> 00:55:17.740
<v Michael Kennedy>It's a step jump, right?

00:55:17.830 --> 00:55:19.800
<v Michael Kennedy>And just to be able to avoid that is great.

00:55:20.080 --> 00:55:22.340
<v Carlton Gibson>No, it's a serious, serious.

00:55:22.820 --> 00:55:28.160
<v Carlton Gibson>If you are taking on celery, then there is a serious question to ask.

00:55:28.420 --> 00:55:34.160
<v Carlton Gibson>You know, if you're an established team, but if you're growing and suddenly it's like, shall we add celery?

00:55:34.420 --> 00:55:44.740
<v Carlton Gibson>It's like, well, hang on, look at your ops capacity and look at your ability to, you know, do you have someone on call for when the workers break down at two o'clock in the morning and page notifications and all those things?

00:55:44.810 --> 00:55:50.600
<v Carlton Gibson>Because it's a world up from self-hosting, you know, your bog standard Django app.

00:55:51.100 --> 00:55:56.460
<v Michael Kennedy>As someone who does a lot of DevOps without a lot of support, I hear that.

00:55:56.530 --> 00:56:02.300
<v Michael Kennedy>So, you know, Carl, I think this could have been like a three-hour show easily, I'm starting to realize.

00:56:03.370 --> 00:56:07.060
<v Michael Kennedy>But let's sort of get some advice to kind of wrap things up.

00:56:07.520 --> 00:56:09.460
<v Michael Kennedy>We're on a glide path here for people.

00:56:09.840 --> 00:56:11.720
<v Michael Kennedy>Should they write async views today?

00:56:11.850 --> 00:56:13.440
<v Michael Kennedy>What do you think about this?

00:56:14.080 --> 00:56:14.820
<v Michael Kennedy>Yes, right.

00:56:14.860 --> 00:56:21.740
<v Carlton Gibson>So if I'm writing a small application, like just my local blog application, I want to add a little bit of async sprinkle to it.

00:56:21.740 --> 00:56:24.600
<v Carlton Gibson>I might just deploy a single ASGI worker by itself.

00:56:24.940 --> 00:56:25.940
<v Carlton Gibson>It's probably big enough.

00:56:26.240 --> 00:56:28.560
<v Carlton Gibson>You're not going to hit scaling problems, you know, with low traffic.

00:56:28.780 --> 00:56:31.320
<v Carlton Gibson>So, you know, just a single ASGI application.

00:56:31.790 --> 00:56:42.660
<v Carlton Gibson>I think the, as I say, the people who are doing this at a bigger level than that, I think they're running their core application as WSGI and the standard ways because the scaling patterns there are so established and known.

00:56:42.740 --> 00:56:44.860
<v Carlton Gibson>and it's almost trivial by comparison.

00:56:45.780 --> 00:56:47.540
<v Carlton Gibson>But then they're running a little sidecar

00:56:47.780 --> 00:56:48.820
<v Carlton Gibson>to handle their app.

00:56:49.190 --> 00:56:50.020
<v Carlton Gibson>So just have, you know,

00:56:50.570 --> 00:56:52.180
<v Carlton Gibson>engine XA can route by the path

00:56:52.210 --> 00:56:53.500
<v Carlton Gibson>or it can route by other things.

00:56:53.660 --> 00:56:54.040
<v Carlton Gibson>But, you know,

00:56:54.230 --> 00:56:55.220
<v Carlton Gibson>if it's got connection header,

00:56:55.350 --> 00:56:55.680
<v Carlton Gibson>keep route,

00:56:55.840 --> 00:56:56.800
<v Carlton Gibson>but that's a little advanced.

00:56:56.810 --> 00:56:57.520
<v Carlton Gibson>Just route by path,

00:56:57.590 --> 00:56:59.300
<v Carlton Gibson>put everything under an A prefix.

00:56:59.570 --> 00:57:00.260
<v Carlton Gibson>And then, right,

00:57:00.380 --> 00:57:02.180
<v Carlton Gibson>those are your async ones for,

00:57:02.540 --> 00:57:02.800
<v Carlton Gibson>I don't know,

00:57:02.830 --> 00:57:04.760
<v Carlton Gibson>your long-lib service center events requests.

00:57:05.010 --> 00:57:06.220
<v Carlton Gibson>And then send everything else back

00:57:06.360 --> 00:57:06.740
<v Carlton Gibson>by your whiskey.

00:57:06.880 --> 00:57:07.340
<v Carlton Gibson>I mean, just do that.

00:57:07.740 --> 00:57:08.460
<v Carlton Gibson>Get the hang of it.

00:57:08.720 --> 00:57:09.540
<v Carlton Gibson>You'll find the scaling,

00:57:10.280 --> 00:57:11.620
<v Carlton Gibson>that scales very well.

00:57:11.890 --> 00:57:12.600
<v Carlton Gibson>And then you can say,

00:57:12.700 --> 00:57:17.600
<v Carlton Gibson>okay, well, now if I move this to async, oh, okay, I start to see some of the complications of async appear.

00:57:17.810 --> 00:57:23.840
<v Carlton Gibson>And I think that's the danger I see is when people dive feet first into async without really understanding

00:57:24.190 --> 00:57:31.700
<v Carlton Gibson>that it is an order of magnitude more complicated to correctly handle async code than to just code synchronously.

00:57:32.400 --> 00:57:36.120
<v Carlton Gibson>For all these reasons about blocking the event loop and you get deadlocks.

00:57:37.300 --> 00:57:41.540
<v Carlton Gibson>These are really complex patterns that grow into them rather than jumping headphones.

00:57:41.620 --> 00:57:47.320
<v Michael Kennedy>yeah yeah but django is ready yes no django's async story is there the problems that i see come up

00:57:47.480 --> 00:57:52.860
<v Carlton Gibson>time and time and time again are problems with python and python's async story that we all share

00:57:53.240 --> 00:57:58.880
<v Carlton Gibson>not django specific problems yeah or even that asynchronous programming in general yes asynchronous

00:57:58.920 --> 00:58:06.019
<v Carlton Gibson>yeah they are yeah they are async or guild problems not um django specific django specific

00:58:06.040 --> 00:58:12.500
<v Michael Kennedy>problems. Absolutely. I want to just put this out in the universe. If you're looking at your web app

00:58:12.660 --> 00:58:17.120
<v Michael Kennedy>and you're like, ah, these are a little bit slow, these requests. So we're going to need to set up

00:58:17.120 --> 00:58:21.820
<v Michael Kennedy>some parallelism so that we can handle multiple slow requests at a time. Like just take an hour

00:58:22.180 --> 00:58:29.060
<v Michael Kennedy>and look at your database indexes, please. Database indexes are magic. And I'm not saying it's not a

00:58:29.160 --> 00:58:35.140
<v Michael Kennedy>benefit to go async or whatever, but if slow is a problem, please just do, you know, look at the

00:58:35.080 --> 00:58:40.400
<v Michael Kennedy>query plans, look and make sure every hot path is using a database index and so on.

00:58:40.600 --> 00:58:42.260
<v Michael Kennedy>Yes, yes, exactly.

00:58:42.500 --> 00:58:46.300
<v Carlton Gibson>I mean, and if it, so when you make a database, we always talk about the database, right?

00:58:46.440 --> 00:58:51.780
<v Carlton Gibson>But if Django makes a database request, there's a little bit of CPU time to compile the request.

00:58:52.320 --> 00:58:55.220
<v Carlton Gibson>And then there's the IO bit where it's talking to the database.

00:58:55.500 --> 00:58:59.340
<v Carlton Gibson>And, you know, PsychoPG already releases the gil in that context.

00:58:59.600 --> 00:59:03.800
<v Carlton Gibson>So you're already getting, you know, what concurrency or parallelism you would have at that point.

00:59:04.880 --> 00:59:09.320
<v Carlton Gibson>And then there's CPU time to sort of marshal the return into,

00:59:09.750 --> 00:59:13.540
<v Carlton Gibson>or structure the returned data into ORM objects.

00:59:13.790 --> 00:59:15.320
<v Carlton Gibson>So those bits are CPU bound.

00:59:15.660 --> 00:59:18.260
<v Carlton Gibson>But that core bit is already releasing the gil.

00:59:18.320 --> 00:59:19.960
<v Carlton Gibson>You're already going to get parallelism there.

00:59:20.760 --> 00:59:24.140
<v Carlton Gibson>So the chances are you're not going to get much more

00:59:24.440 --> 00:59:26.460
<v Carlton Gibson>by jumping to Async or adding more workers.

00:59:26.530 --> 00:59:30.160
<v Carlton Gibson>It was your database all along that was the thing that was taking the time.

00:59:30.460 --> 00:59:32.420
<v Michael Kennedy>These things that we've been talking about,

00:59:32.520 --> 00:59:34.460
<v Carlton Gibson>about CPU contention and all the rest.

00:59:34.680 --> 00:59:38.140
<v Carlton Gibson>It's, ah, they come up in only specialized cases.

00:59:38.360 --> 00:59:41.240
<v Carlton Gibson>They're kind of advanced topics, if that makes sense.

00:59:42.220 --> 00:59:43.800
<v Carlton Gibson>They're often not the problem.

00:59:44.380 --> 00:59:47.620
<v Carlton Gibson>Amazing when you need them, but it's not a panacea, yeah.

00:59:47.900 --> 00:59:50.320
<v Carlton Gibson>Yeah, and we, I mean, how would I phrase it?

00:59:50.380 --> 00:59:53.820
<v Carlton Gibson>We spend so much time thinking about the use cases

00:59:53.950 --> 00:59:56.380
<v Carlton Gibson>for the advanced cases that we sometimes forget

00:59:56.490 --> 00:59:58.420
<v Carlton Gibson>that the simpler ones are much more simple stories.

00:59:58.610 --> 01:00:00.580
<v Carlton Gibson>And yeah, should I be writing async views?

01:00:00.660 --> 01:00:02.740
<v Carlton Gibson>well, do it gently.

01:00:03.140 --> 01:00:05.980
<v Carlton Gibson>Don't jump into it thinking it's some cure-all.

01:00:06.560 --> 01:00:07.360
<v Carlton Gibson>Yeah, but Django's ready.

01:00:07.520 --> 01:00:07.820
<v Carlton Gibson>That's cool.

01:00:08.160 --> 01:00:08.560
<v Carlton Gibson>Yeah, yeah, yeah.

01:00:08.570 --> 01:00:09.020
<v Carlton Gibson>No, absolutely.

01:00:09.460 --> 01:00:14.300
<v Carlton Gibson>And that's why we made the effort to update the topic,

01:00:14.480 --> 01:00:17.440
<v Carlton Gibson>because we realized we were giving users bad advice.

01:00:17.920 --> 01:00:18.900
<v Carlton Gibson>Yeah, you're scaring them off.

01:00:19.220 --> 01:00:20.220
<v Carlton Gibson>Yeah, no, we really were.

01:00:20.220 --> 01:00:22.000
<v Carlton Gibson>We were like, oh, it's still incomplete.

01:00:22.170 --> 01:00:22.560
<v Carlton Gibson>It's not ready.

01:00:22.800 --> 01:00:23.800
<v Carlton Gibson>No, it's there.

01:00:24.160 --> 01:00:26.940
<v Carlton Gibson>And yes, you will run into problems as you scale up,

01:00:27.290 --> 01:00:28.240
<v Carlton Gibson>but that's not Django.

01:00:28.460 --> 01:00:29.220
<v Carlton Gibson>That's async.

01:00:29.380 --> 01:00:29.960
<v Michael Kennedy>Yeah, indeed.

01:00:30.280 --> 01:00:30.940
<v Michael Kennedy>Final call to action.

01:00:31.400 --> 01:00:33.120
<v Michael Kennedy>People consider Django.

01:00:33.819 --> 01:00:36.760
<v Michael Kennedy>Maybe we're considering async for their existing Django projects

01:00:37.080 --> 01:00:40.460
<v Michael Kennedy>or other things like the Django tasks, background tasks and so on.

01:00:40.520 --> 01:00:40.940
<v Michael Kennedy>What do you tell them?

01:00:41.900 --> 01:00:44.340
<v Carlton Gibson>Yeah, I mean, I would get going with Django tasks.

01:00:44.510 --> 01:00:47.360
<v Carlton Gibson>You install the database backend for that,

01:00:47.430 --> 01:00:49.840
<v Carlton Gibson>which is all the Redis ones work perfectly as well.

01:00:49.890 --> 01:00:51.140
<v Carlton Gibson>If you've already got Redis in your stack

01:00:51.539 --> 01:00:53.340
<v Carlton Gibson>and you want a little bit more,

01:00:53.620 --> 01:00:54.880
<v Carlton Gibson>but most people don't need that.

01:00:55.000 --> 01:00:57.400
<v Carlton Gibson>Just use the ORM database queue.

01:00:57.740 --> 01:00:58.480
<v Carlton Gibson>It's a great storage.

01:00:58.880 --> 01:01:01.340
<v Carlton Gibson>That's where you want to throw most of your work.

01:01:01.560 --> 01:01:03.420
<v Carlton Gibson>You don't really want to be...

01:01:03.860 --> 01:01:07.980
<v Carlton Gibson>With Async.io, you can, yes, throw something off to an executor.

01:01:08.840 --> 01:01:11.840
<v Carlton Gibson>No, but don't do that because you've got no queue controls.

01:01:11.880 --> 01:01:12.700
<v Carlton Gibson>You've got no result.

01:01:12.800 --> 01:01:15.700
<v Carlton Gibson>You can't say, did it fail and then have logic to retry,

01:01:15.760 --> 01:01:16.800
<v Carlton Gibson>any of those sorts of things.

01:01:17.320 --> 01:01:18.940
<v Carlton Gibson>It's just running on the event loop.

01:01:19.000 --> 01:01:21.000
<v Carlton Gibson>And if it crashes, well, what do you do about that?

01:01:21.000 --> 01:01:21.940
<v Carlton Gibson>How do you recover from that?

01:01:22.020 --> 01:01:23.160
<v Carlton Gibson>There's no elegance to that.

01:01:23.820 --> 01:01:26.860
<v Carlton Gibson>So, yes, Django tasks is probably where I would point people first.

01:01:27.340 --> 01:01:29.220
<v Carlton Gibson>And then do I need long-lived requests?

01:01:29.340 --> 01:01:31.560
<v Carlton Gibson>Do I want to do things like service and events or WebSockets?

01:01:31.740 --> 01:01:32.680
<v Carlton Gibson>Yeah, that's where.

01:01:32.890 --> 01:01:34.740
<v Carlton Gibson>Or streaming, streaming responses.

01:01:35.320 --> 01:01:36.500
<v Carlton Gibson>Async, great, great example.

01:01:37.180 --> 01:01:37.240
<v Carlton Gibson>Yeah.

01:01:37.730 --> 01:01:42.120
<v Michael Kennedy>You want to adopt Datastar or one of these frameworks that really leverages service and events.

01:01:42.500 --> 01:01:42.580
<v Michael Kennedy>Yeah.

01:01:42.840 --> 01:01:42.960
<v Michael Kennedy>Yeah.

01:01:43.280 --> 01:01:48.480
<v Carlton Gibson>Or HTMX version 4, which is in beta whatever now, which I think is the risk.

01:01:48.480 --> 01:01:49.680
<v Michael Kennedy>Oh, is that coming with that as well?

01:01:50.360 --> 01:01:50.900
<v Carlton Gibson>Yeah, yeah, yeah.

01:01:51.000 --> 01:01:53.240
<v Carlton Gibson>So they basically were like, right, it's okay.

01:01:53.640 --> 01:01:57.200
<v Carlton Gibson>So I think Carson was like, we're never going to do an HTMX version 3.

01:01:57.300 --> 01:02:02.860
<v Carlton Gibson>and he's kept his promise because he's jumped straight from two to four um but the it

01:02:02.870 --> 01:02:07.680
<v Carlton Gibson>was um it was inspired by some of the things from um data star and the surrounding projects it's

01:02:07.680 --> 01:02:11.120
<v Michael Kennedy>like hang on we can have a go at that as well i think said goss you have to get him on and he can

01:02:11.190 --> 01:02:15.420
<v Michael Kennedy>talk you through it yeah i i love carson i think what he's doing is great and i've had him on before

01:02:15.540 --> 01:02:20.480
<v Michael Kennedy>i'm a fan of hdmx so that's cool maybe we'll do another no other round yeah so when i stepped

01:02:20.530 --> 01:02:26.419
<v Carlton Gibson>down as a fellow in 23 i started a new application been building with hdmx and alpine since day one

01:02:26.440 --> 01:02:27.820
<v Carlton Gibson>and just absolutely adore it.

01:02:29.460 --> 01:02:30.820
<v Carlton Gibson>It's combined with template partials,

01:02:30.900 --> 01:02:31.880
<v Carlton Gibson>which we added into Django,

01:02:31.960 --> 01:02:33.900
<v Carlton Gibson>it's really changed the way I write web applications.

01:02:34.200 --> 01:02:34.580
<v Carlton Gibson>It's amazing.

01:02:34.820 --> 01:02:35.680
<v Carlton Gibson>All for the better as well.

01:02:35.800 --> 01:02:38.040
<v Michael Kennedy>It's like JavaScript magic fairy dust.

01:02:38.400 --> 01:02:39.780
<v Michael Kennedy>Yeah, it's how it should be.

01:02:40.540 --> 01:02:40.980
<v Michael Kennedy>Yeah, exactly.

01:02:41.500 --> 01:02:42.240
<v Michael Kennedy>Sprinkle it on there.

01:02:42.240 --> 01:02:43.300
<v Michael Kennedy>It doesn't have to be a spa.

01:02:43.740 --> 01:02:44.720
<v Michael Kennedy>Everyone calm down.

01:02:44.780 --> 01:02:47.620
<v Michael Kennedy>We could still do some stuff on templates and so on.

01:02:47.980 --> 01:02:50.960
<v Carlton Gibson>But when I learned JavaScript, that was how it was.

01:02:51.540 --> 01:02:53.000
<v Carlton Gibson>You detach a few event listeners

01:02:53.340 --> 01:02:54.460
<v Carlton Gibson>and just do your little thing.

01:02:54.620 --> 01:02:55.940
<v Michael Kennedy>And that was enough, right?

01:02:56.260 --> 01:02:58.580
<v Michael Kennedy>hashtag dollar document ready.

01:02:58.740 --> 01:02:59.020
<v Michael Kennedy>Let's go.

01:03:00.940 --> 01:03:02.640
<v Michael Kennedy>A little jQuery back in the day, right?

01:03:02.800 --> 01:03:02.900
<v Michael Kennedy>Yeah.

01:03:03.180 --> 01:03:03.800
<v Michael Kennedy>That's how it was.

01:03:04.080 --> 01:03:04.380
<v Michael Kennedy>Yeah, yeah.

01:03:04.600 --> 01:03:04.700
<v Michael Kennedy>Cool.

01:03:04.800 --> 01:03:05.760
<v Michael Kennedy>Select something, do something.

01:03:06.360 --> 01:03:06.540
<v Michael Kennedy>Exactly.

01:03:07.280 --> 01:03:10.240
<v Michael Kennedy>Carlton, thanks for first all the stuff on Django for everybody

01:03:10.620 --> 01:03:12.140
<v Michael Kennedy>and second, coming on the show to share.

01:03:12.420 --> 01:03:12.800
<v Carlton Gibson>No, thanks.

01:03:12.940 --> 01:03:14.220
<v Carlton Gibson>Thanks for the chat.

01:03:14.280 --> 01:03:14.780
<v Carlton Gibson>It's been really good.

01:03:14.820 --> 01:03:18.140
<v Michael Kennedy>It's really lovely because you pick on the topics

01:03:18.440 --> 01:03:19.680
<v Carlton Gibson>and I'm like, yeah, yeah, this is right.

01:03:19.880 --> 01:03:20.040
<v Michael Kennedy>Thanks.

01:03:20.280 --> 01:03:20.480
<v Michael Kennedy>See you later.

01:03:20.760 --> 01:03:21.160
<v Michael Kennedy>Thank you, Michael.

01:03:21.900 --> 01:03:24.140
<v Michael Kennedy>This has been another episode of Talk Python To Me.

01:03:24.440 --> 01:03:25.220
<v Michael Kennedy>Thank you to our sponsors.

01:03:25.540 --> 01:03:26.720
<v Michael Kennedy>Be sure to check out what they're offering.

01:03:26.910 --> 01:03:28.300
<v Michael Kennedy>It really helps support the show.

01:03:28.920 --> 01:03:30.300
<v Michael Kennedy>Take some stress out of your life.

01:03:30.700 --> 01:03:33.880
<v Michael Kennedy>Get notified immediately about errors and performance issues

01:03:34.010 --> 01:03:36.080
<v Michael Kennedy>in your web or mobile applications with Sentry.

01:03:36.620 --> 01:03:41.000
<v Michael Kennedy>Just visit talkpython.fm/sentry and get started for free.

01:03:41.560 --> 01:03:44.000
<v Michael Kennedy>Be sure to use our code, talkpython26.

01:03:44.720 --> 01:03:48.340
<v Michael Kennedy>That's Talk Python, the numbers two, six, all one word.

01:03:48.980 --> 01:03:50.860
<v Michael Kennedy>If you or your team needs to learn Python,

01:03:51.070 --> 01:03:54.579
<v Michael Kennedy>we have over 270 hours of beginner and advanced courses

01:03:54.600 --> 01:04:01.140
<v Michael Kennedy>on topics ranging from complete beginners to async code, Flask, Django, HTMX, and even LLMs.

01:04:01.440 --> 01:04:03.640
<v Michael Kennedy>Best of all, there's no subscription in sight.

01:04:04.300 --> 01:04:05.960
<v Michael Kennedy>Browse the catalog at talkpython.fm.

01:04:06.660 --> 01:04:10.040
<v Michael Kennedy>And if you're not already subscribed to the show on your favorite podcast player,

01:04:10.680 --> 01:04:11.320
<v Michael Kennedy>what are you waiting for?

01:04:12.000 --> 01:04:13.760
<v Michael Kennedy>Just search for Python in your podcast player.

01:04:13.860 --> 01:04:14.740
<v Michael Kennedy>We should be right at the top.

01:04:15.160 --> 01:04:17.920
<v Michael Kennedy>If you enjoy that geeky rap song, you can download the full track.

01:04:18.140 --> 01:04:20.040
<v Michael Kennedy>The link is actually in your podcast blur show notes.

01:04:20.840 --> 01:04:22.200
<v Michael Kennedy>This is your host, Michael Kennedy.

01:04:22.600 --> 01:04:23.660
<v Michael Kennedy>Thank you so much for listening.

01:04:23.940 --> 01:04:24.660
<v Michael Kennedy>I really appreciate it.

01:04:25.100 --> 01:04:25.800
<v Michael Kennedy>I'll see you next time.

01:04:46.000 --> 01:04:49.880
We tapped into that modern vibe over Kingy Storm.

01:04:50.760 --> 01:04:52.020
Talk Python To Me.

01:04:52.300 --> 01:04:53.400
I-Sync is the norm.

01:05:22.600 --> 01:05:22.620
.