#301: Deploying and running Django web apps in 2021 Transcript

00:00 We've been learning Django and now you want to get your site online, you're not sure about the best way to host it or the trade offs between the various options. Maybe you want to make sure your Django site is secure. On this episode, I'm joined by two Django experts. Will Vincent and Carlton Gibson talk about deploying and running Django in production along with recent updates in Django3.2 and beyond. This is talk python to me, Episode 301, recorded January 19 2021.

00:40 Welcome to talk Python to me, a weekly podcast on Python, the language, the libraries, the ecosystem, and the personalities. This is your host, Michael Kennedy. Follow me on Twitter, where I'm @mkennedy, and keep up with the show and listen to past episodes at talk python.fm and follow the show on Twitter via @talkpython. This episode is sponsored by Square and Linode. Please check out what they're offering during their segments. It really helps support the show. Before we get to our discussion, just one quick announcement. We started live streaming the recordings of talk Python to me episodes on YouTube. If you're part of the live stream, you'll have a chance to ask questions and your comments might get featured on the air. Just visit 'talkpython.fm/youtube' to subscribe to the channel and see upcoming and past live streams. Now on to the show. Bolton, Will welcome to talk Python to me again. Welcome back, guys.

01:28 Thank you, you came on our show and I was on your show three years ago. So Carl, first time on your show toolpaths a newbie.

01:35 It was great to have you on the show before we'll and we talked about Django then. And I suspect it won't surprise people with you and Carlton here that we may touch on Django.

01:45 I think we've got to talk about Django a little bit a little a little bit.

01:47 I often tell

01:48 people who you are. I'm Django fellow. So what that means is I helped to maintain the framework, a framework the size of all you've got me on twitter there, Gibson on Twitter, and on GitHub, numerous other places, probably, but a framework the size of Django, it just simply doesn't get maintained on purely volunteer effort. So there's a body called the Django Software Foundation, which Will's on the board for he can tell you a bit about that in a second. But they are the charitable foundation, which will collect money from donations from GitHub sponsors, and you can donate on Django project a calm, and then they The main thing they do is they contract the fellows, which is myself and Maurice felicio, who's my colleague now. And we do that with either What do we do we triage the tickets, we do pull request review, we do handle security issues, we handle the releases, not least the alpha releases today. So today, literally today, we released Django 3.2 Alpha one fantastic,

02:39 but is that the very first three to release?

02:41 That's the first one. So that's like, if you've got a Django project, and you've got CI, you download it now and you run it on your CI and you tell us all the things we broke better before we released the final, you know, it's much better during the the alpha pre release period, then, you know, just after 3.2 final you go, Hey, everything's broken.

02:57 Yeah, you really need people to test it. Now, before it's too late. Right?

03:01 Yeah. But yes, it's quite exciting, because the last couple of weeks have been building towards the feature free, so the alpha marks the feature free, so the beat over the pre release period, now we'll we'll merge bug fixes in the new features. So anything that people find them and report will fix and get out before the final, but there's no new features going into 3.2. So on Thursday, just gone, I branched the stable branch for 3.2, which will be that's Django 3.2. And it will be that for the next three years, because the long term release gets quite the long term support release, which 3.2 is gets three years of support. That's really great. I will definitely have to come back to that. But also, welcome, Will tell everyone about yourself real real briefly. You

03:40 know, you were on the show a while ago, and we talked, I believe it was learning Django we spoke about right

03:44 and left in Django at the time. So I'd written I think a couple books. So I have three books, Django for professionals and API's. And the last two years I've been a member of the Django Software Foundation Board about trees who were voted in who were play a prominent role in the community one way or another. And then the board is seven people annually voted on who managed Django itself, which is a nonprofit, what that later So yeah, I and I have now a website, 'learndjango.com', which is an online version of educational content. But in addition to having a podcast with Carlton, Django chat, since he's a fellow which is contracted by the board, we have another touch point, as if we didn't need

04:22 more. Exactly, yeah. So I was on your show a while ago, and that was super fun. I really enjoyed our conversation there. And you know, maybe just tell people quickly about your podcast, what kind of stuff you cover there. Obviously Django but you know, where do they find it? Where they cover? Okay, so

04:36 we get people on from the community. It's kind of an interview type basis, we get someone on it's fortnightly. Now we were doing it every week. But you know, we not like you Michael can't keep that up for revenue. So we've gone fortnightly, which has taken a bit of pressure off with COVID and you know, the kids off school and all of that. But we get you know, people who've contributed to Django or people who are using Django in their business and talk about it. So it's it's Django chat. It's chatting about

05:00 chat about Django. Yay,

05:01 yay, yay, yay, chat django.com

05:04 I think recreating a little bit of what you would have at a Django con event. These are annual events. So we started after I went to my first Django con at Carleton and was like, why can't I talk to people about Django more often? And at the time, there wasn't a Django podcast. So yeah, it's we've gotten to interview really my dream list of guests. I mean, we we had actually, I was just talking to someone else who works at stripe, which is still running on Sinatra, which is a Ruby thing. And I was like, Yeah, I've talked to DHH because he's came on our shows, we talked about rails versus Django. We had Carl Meyer on his Instagram was formerly Django core, all the basically everyone, almost everyone who'd want to talk to who's involved in Django is willing to come on and share their story. So it's, I think it's a really nice connector and educational for us I do as

05:45 well. And you're just sort of a sidebar for our show here. Yeah, being all of us, fellow podcasters. I think it's really interesting, the role that podcasts play in keeping connections to the broader tech community, when we can't go anywhere. I mean, even if you weren't typically going to conferences or meetups, you could still go to work and see other people, right. It's just the first one when the COVID hit. I was like, I've just I'm not traveling anywhere. I'm not taking my kid to school or anything like that. So there's no like natural place here. I'm stuck for 45 minutes, I'm going to just listen. But as it's drawn on, I started listening to shows, especially with multiple people trying to bring sort of a normalcy. And I get to kind of hang out with these people, even though that they don't respond to me, I still get to hang out with them. And I think that's a really interesting societal thing that's happening right now.

06:33 Yeah, I think it's beneficial across the board. I mean, I probably like like you, I listen to podcasts outside of tech as well. And it's sort of like people, I'd want to sit in on their conversations anyways. So for me, it's probably my primary media consumption, aside from books. Yeah, I mean, I'm not reading ancient Greek, like Carlton Carlton has a PhD and all that. We're gonna talk about specifically deployments on Django, and we can go on and on about, obviously, all the intricacies of Django, but I think suffice to say I'm an educator on the on the board, and Carlton is a fellow. So he's, he makes the releases happen, including 3.2 alpha, which is dropped today. And I guess you mentioned LTS, so that's confusing to non Django people. So since having ellos, like Carlton Django has a pretty rapid release cycle, where it's every nine months or so. So there's 3.0,3.1,3.2. This December, I think, Carlton is 4.0, every one of those is a long term service release. So that'll last two and a half years. So that is a three years. Yeah, it's it's on, there's a link on the Django project site. So that's a way that so Django doesn't really have. It's rare to have breaking changes these days. But the LTS is designed to help people who can't keep up with that cycle. Stay up to date, though, we have a lot of podcasts and opinions about why you should always stay up to date. And it's worth it. Yeah, because that's one of the most fair things you can do. Because as Carlton mentioned, there's bug fixes constantly. So there'll be, there's 3.1.1,3.1.2, there'll be, you know, 3.2 the 3.2.1 a month later. So

07:56 I'm a big advocate of you know, if you possibly can get on the latest major release. So you like hanging out, historically, the long term release was really the LTS release was really important. Because there were breaking changes, right, in each major version of Django, there were new things, and it was difficult to keep. But that's not the case anymore. It's really easy to update. So I'm a big advocate of that now. And then when I you know, I talk to fellow people in the Django community, they're like, Well, you know, I work in a real world, and you can't keep up on the latest major version. So for those folks, then, you know, the LTS is a really good option, because it's once every three years, you know, it's coming. Yeah, you get a six month window of overlap of support. So the old LTS gets six months of security release after the release of the new LTS, and that's your window to update?

08:41 Well, I think that running and maintaining software built upon frameworks like Django, it falls into two categories. For me, it falls into here's something that we have a team or at least somebody dedicated to owning this project, and we care about its ongoing life. And then we have the ones that are the Oh, please don't touch it. And the Oh, please don't touch it is we've long since stopped developing that maybe the person who developed it left, but it's still important, and we don't want to break it. It's working right now. But if you touch it, and you break it, you've now adopted it. You know what I mean? Like, it's that thing that just it runs like

09:17 if you break the build,

09:18 yes, but but worse, because it's a legacy build. So to me, I feel like that please don't touch it side. The LTS makes perfect sense for them.

09:26 Yeah, entirely, entirely. I, the way I like to think of it is, is this a thing that you feasibly going to add new features to, and if you're feasibly going to add new features to then you should be on the latest major version. And because it's only once every eight months, you need to allocate a day or two to keep up to, you know, fix up, download the new version, run the test suite, see the deprecation warnings, fix the deprecation warnings, you know, maybe you have to wait a couple of weeks for a dependency to update and, and then you can push forward. And that's once every eight months that that process and if you're adding new features, if it's a live project, ideally you'd be there. Yeah, there's something You just need to keep running for the long run. You know, you could do that much less frequently,

10:04 right? But it allows people who have that please don't touch it. But oh, there's a security problem. If there's no LTS, and there's a security problem, then not only do they have to figure out how to redeploy a fix, they've got to say, well, we didn't touch it for three years, and it doesn't quite work the same. So then you get into the discussion of Well, what's the risk? Will they really, it's just Java swing? I mean, come on, what's the problem? Right? how bad could that go?

10:28 You know, like the entire world's credit reports be now that we're in Python 3 world, Django was part of that. It's really beyond the security thing. It's also all the ecosystem around Django, the third party packages, like a lot of times, if you look at their project that says, I can't update, why can't you update, they did two things. They're using a third page, which falls into that, you know, touch it, you broke it situation, or they did something custom. At one point, they they went off the guardrails, and the bill comes due. I mean, it's so tempting, actually, and I actually want to speak and clear up the guardrails, quick note that, Django just passed flask stars, which is a really poor metric of popularity. But nonetheless, we'll take it, congratulations, that's awesome, very easy to spin up a couple API endpoints. And boom, you're using flask, that's a very different thing. That's like as DHH would say that's a Lego versus the Lego truck. That is a framework like Django. And anecdotally, a lot of places use flask. But in terms of a big site, that's all flask, that's much less common than all Django.

11:31 Yeah. Well, I was thinking about this this morning. And it you know, the flask and Django are pretty neck and neck in terms of popularity. They're not on GitHub. Yeah. Yes, sir. There's these metrics and popularity, they there's like all over the map. So one of the thoughts I had though, is the people that use flask, this is not a knock against flask, I like it. But one of the my impressions is, if I just need to, like you said, just create a couple API's and just, we're just going to get something going real small and simple. And just roll like those people are not in as invested in the ecosystem. And the framework, as I feel like the Django folks are, I feel like the Django folks, it's a more encompassing part of their development experiences or development lifecycle like Django feels more part of the project when people adopt it and love it.

12:17 I don't know, what do you guys think about that? You need a dozen apps to use flask. And I hope that we get David Lorde on to talk about flask because he's a Django con. So I mean, it's not a compact, I don't mean to say there's a competition between the two but serve different purposes. I would say Carlton, right.

12:30 Yeah, the different styles as well, like, you know, if you want to put together something, why not flask, but I mean, I've been using Django for so long now that even if I just need to spin up two endpoints, it's much quicker for me to do that in Django than it is to go and get supposedly microframework and work out how am I supposed to use this, you know,

12:48 ever, we can put the link in the notes I have, I made up a repo with the code, because you didn't provide code, you can have a single file, Django project, same way you can fly, because a lot of times that Hello, world comparison will lead people to assume that flask is much less complicated than Django, and it's a little bit less, it's more around to how it's structured, which is the point of Katherine's talk. Yeah, so

13:08 the talk was called dumb how to you using Jenkins and Michael framework or something like that. And it was about the the base HTTP handlers, that kind of that really real core of the framework. And I was right. You know, I put up a few examples from different frameworks, like flask and node example and Starlet. You know, async, microframework, and Tom Christie, and then I, you know, showed how you might put that together in Django.

13:30 Yeah. For people who don't necessarily live and breathe the web stuff like the three of us do.

13:34 Yeah,

13:34 give them a quick definition of microframework. Before you go on,

13:37 in the talk, I get I was like, you know, what is it? microframework?

13:40 Yeah, I'm asking you, because I don't want to put my foot out there.

13:43 The answer I came up with was, it's got a great HelloWorld. Right.

13:46 Yeah, okay. Sure.

13:47 I think the idea is that it's easier to contrast it with like rails or Django or rails and Django, they come with the batteries included with everything I need, you know, you've got ORM or active regularly,

13:59 you've got an O RM, you've got database, you've got migrations,

14:03 all this stuff was you microframework, you perhaps get the call HTTP handling. And that's about it. And then you have to go and find a forms like, okay, I found a forms library, and then you have to pull in an ORM, or a database of that. Okay, I got I'll use that. And you know, so there are there are node full batteries included frameworks, like happy, it's very good. It's got everything you need. But like, the classic node example is, oh, you know, I get Express, and then I pull it in this, this thing that has seen URLs and that thing, you know, so there's a continuum. But Micro frameworks fit more towards that, you know, you put the pieces together yourself, whereas a batteries included framework like Django, you get not everything but a lot in the branch.

14:40 Right. But your your experience is more Django pieces, Django building blocks, then a little flask, SQLAlchemy a little this little that definitely. And I've been doing it so long now that I struggle to break out of that. I

14:52 just, you know, I throw in a bit of starlet or a bit of FastAPI, see what's going on with the new frameworks, every new framework that comes out but always give it a run. And then my question is okay, well, what can we learn from that?

15:04 This portion of talk Python to me is brought to you by 'Square'. Do you run or want to build a web application that sells products or services. Building a successful online business is the stuff that dreams are made of. But accepting payments and handling credit cards with all the various regulations is a common stumbling block. But not with squares payment API, or Square your Python web app can easily take payments, you'll seamlessly be able to accept debit and credit cards as well as square gift cards. Let your users pay with their digital wallets Square works with Apple Pay, Google pay and masterpass. All in one go. Your API includes PCI compliance, end to end encryption, dispute management and fraud detection, build your online payment form in three steps with square payment SDKs. Step one, create a new square API application. Step two, add squares payment form to your checkout flow. Step three, use squares API to charge the card to get started building your business on square just visit 'talkpython.fm/square' or click the link in your podcast player shownotes. That's talk python.fm/square.

16:07 really large site, if you ask them, like what language and frameworks you use, it's like everything, just because of the needs of a massive site, it ends up being it's hard to say that it's truly one thing I mean, Instagram is still has pieces of Django in it. But you know, it's at that scale. It's its own thing. And I think you know, the micro frameworks are really good for doing that this might be a pessimistic take. But I've heard people make that micro framework allows you to switch the complexity from individual developers who may be cheering you just as much up to your upper level, because they only can touch a tiny part of the monolith. So they can in a way do less damage. That's actually an organizational argument for microframework. Because it only this software architects or the senior people

16:45 fit it all together, weave the pieces they put they build the puzzle or whatever. Yeah, that I guess,

16:49 especially if you're testing suite isn't as good as it could be.

16:52 Yeah, I think there's an interesting conversation about like monolith type software styles versus not, I'm actually a big fan of the monolith style, like all my attempts to break it into little pieces, at least in the micro services, which is not the same as micro framework. But I always feel like I've quadrupled my deployment complexity to half my software complexity, and I'd rather manage half as much software complexity is twice as much deployed.

17:16 Right? Like if you were part of a five person team in a large organization assigned to fix something and you're just like, there's no way we can get done within this model it but like, Oh, we could spend something up and solve the problem and move on quite a lot. I think you know, oranges are an individual project, you get fancy and you customize Django or you use a third party package that is out of date. Or maybe you shouldn't use any of the same issue.

17:38 Yeah, I don't know. Like, okay, with sweet was segwaying slightly back into the deployment topic, which extensively we're going to talk about. But they the I think the thing with microservices, right. So if you've got a massive Corporation, you can't have all the things going on in one team. So you need to segment it somehow. And then you can define service boundaries, and it starts to make sense, then all the tech blogs become about microservices and how this is the way and then learners come along. And they're like, well, I've got to build my thing using microservices. It's like no, that's, that's not a good move. But there's that wherever it is, the blog posts, not for you know, and now the pendulum swings back. And that you know, there's there are lots of posts now about how micro monoliths are making return. And now they're the way to go. And so hopefully the beginners come and see those posts. But when you're learning, and everyone's saying, Oh, you must use a microservice architecture, it's like, hang on.

18:28 Yeah. Because the people who are knowledgeable are writing about their day job. They're not writing about spinning up a Django app in a weekend, which is possible, and you can deploy it to which we could. Should we get into that deployment? I

18:39 do want to just point out, I think this might be the article that I read before, like the Yeah, you you're not Google, you're not LinkedIn, you're not Facebook, you're not Netflix, there are people who actually that's an in accurate statement about, there's plenty of people that work at those companies. But like you said, Carlton, the people who come along, they see these companies who they respect and say they must be doing it, right. And so often with these deployment stories with these design patterns, you know, should you have like separate caching servers that run like something like Redis? Yes, you should? No, you shouldn't? And that's the right answer at the same time, but with the context that you need, right? Are you trying to run 10,000 servers and let 500 people work on this project? Or are you two people trying to do a startup like those are not the same trade offs and balances you want to make? Right?

19:25 Yeah, and it depends as well on your model. So one, one example that I really like is Stack Overflow. Now they're built entirely on the Microsoft stack. They've got SQL Server, and they're using dotnet. And but the basic point is they've got a really big database one, and then you know, a few worker processes in front of those. And that's it. It's a classic monolith, and then one of the biggest sites on the internet and yet they're incredibly fast. Yeah, exactly as fast as you could ever want. Okay, they do it in this kind of monolith old school style, you know, vertical scale, make it bigger at night, don't spin out. parallels, just make the thing bigger by a bigger Is database service. So that's you're not going to be bigger than Stack Overflow, right? And then another example we have when you are you will reevaluate it anyway, what

20:07 are we gonna solve that problem? Yeah,

20:10 but you're not going to be bigger than Stack Overflow. And then we had another example, which was doctrine demand. They came on Matt Lehmann, from those from doctrine man came on Jango chat recently, they've got a more containerized approach to the to it, but the thing is, they're doing peer to peer communication between, you know, many simultaneous peer to peer video communications, because doctor on demand is like a tele telemedicine right, you

20:31 Right, right. Like, I need to book a video appointment with my doctor.

20:34 Yeah, exactly. So, but they've got they've got so many. So many of those processes needed to be spent spun up that they need something which scales out and then back down again very easily. So they've got a kind of more like modern what a color technology. They're not as big as Stack Overflow, probably in terms of raw traffic, but they need that flexibility of architecture.

20:54 Right, we say it depends. That's it?

20:57 Yeah. Exactly. That's your subtitle. I do think the context, right. When people talk about design patterns, deployment, architecture, there's those are always within a silent, not mentioned most of the time context in which that makes sense. And you got to decide, are you in that context? Are you not? Yeah, no,

21:15 I didn't say it. What problems are you trying to solve? Like, you can't innovate everywhere. Like there's so many pieces to a website? Do you want to innovate on deployment? Not until you're, you know, Instagram? I don't think

21:24 No, I think that's a great point. And it's when you're small, the most important thing is adding features. And going quick, right? When you get bigger, the most important thing is to not go down, right? If random little project that is like an ordering service for a restaurant in Milwaukee, if it goes down for half an hour, that's not ideal, but it's not the end of the world. If Google goes down for half hour, people really notice, you know, it's like front page news tight. So it's, it's just a totally different thing. Now before we move on to a more deployment stuff, and we're kind of in that I do before we get away too far. Maybe we can touch real quickly on some of the new features that are coming in three, two, and like the Django three stuff that you guys are excited about.

22:03 Yeah, maybe Can you start with just three in general Carlton, because I think that makes three to four contacts. Well, okay. So

22:09 but three introduced it three began the process for making Django async. Right, so we added. So historically, Python has this whiskey, the web service gateway interface. So Django is a whiskey framework. flask is a whiskey framework, it was this standard so that application servers could talk to protocol servers, which could talk to the internet without, you know, each framework having to have its own protocol server. So ganache right is a whiskey server, and it can speak to flask, and it can speak to Django, and it can speak to any other whiskey server,

22:39 right. You can use that and I can use micro whiskey. And yeah, it's we don't have to coordinate or do anything. It just happens that yes, of this WSGR whiskey thing, right. Yeah. So

22:49 that's the standard. And so in order to make things async, there's this thing called ASCII, which is the asynchronous web gateway interface. Oh,

22:58 yeah. I don't know. I don't really yeah. Anyway, it's not whiskey, it's ASCII. So

23:01 first of all, Django 3.0, brought in an ASCII handler, so it wasn't async at all, but you could run it under an ASCII server. And then 3.1 brought in async, actual async views. And you can define an async def view. And you can use I don't know HTTP x, which is like an asynchronous HTTP client with a request. Yeah, it's like requests, but async.

23:23 Right, or, what's that? ORM story. Does that support async? await yet,

23:27 right? No. So that's not there yet. So this is something that will develop over probably over the course of the for Django 4 style. So there's plans and there's thoughts. And we need to get to the point where you with the ORM, where you can down to your kind of filter call. And so things like filter, they can be totally synchronous, because they don't actually make do any IO, they don't actually hit the database. But when you then go, like, I've got my query set, and I'm going to iterate it, and I'm going to fetch the objects, database, we need that bit, even if the actual connection is running a thread or whatever, we need that bit to be fully async. And then Django will will feel async as it is, at the moment, if you write an async view in Django, you kind of have to say, Well, I'm not going to touch the DB, I'm not going to you can wrap the ORM in a in async to async, wrap up, but you kind of lose the point of that.

24:17 Yeah, but that's coming. Right. Yeah.

24:19 So Andrew, yeah, so Andrew Gordon, who's driving this effort. He's, he gave a talk at Django con Europe, about the structuring of this, and what its gonna look like. And it's like, Yeah, that makes sense. This is going to work and we don't have to get what's nice is we don't have to get async all the way down very to the very bottom, because we can run the actual we can sort of hand off the actual database connection in the thread executer, but from the outside of the ORM, it will look async and then you can you know, you can stream out your content records of maximum throughput as you want to that kind of

24:50 Yeah, well the fact that you can already do external API's. And if you're going down the micro service path, you can do your micro service calls with HTTP x and await those that's already Yes, yes. Yeah,

25:00 so without and this works out what's amazing about the way Andrew is just absolute hero, but he put it together in such a way that you can run this with a whiskey service, you're running Django 3.1. With Garner cotton like you always have, and you think yourself, I just need to make a couple of API calls, but they're quite slow. And I want to make them in parallel, you can do that just by writing an async def call, and then, you know, making the parallel calls using their HTTP xa. And it just works. And Django does all the rest for you and adapt it and you didn't have to change your application server, you didn't have to do you know it just yet, right.

25:34 But eating just that one of the challenges, if you're talking to the database is usually Okay, find the bottleneck was we're waiting at the web server level, at the Django level, and then we're gonna push that down to the database, we're gonna just make all that async, all of a sudden, all the pressures now on the database, which can be a problem. But if you're talking to external API's, you're now pushing the pressure onto the internet, which is a way scalable.

25:57 Yeah. And it's always going to scale more than you, right? You're not going to need that many client requests, probably.

26:03 Yeah. Also, David Smith out there says 3.2 will be a great release, you guys have done a great job getting many patches. Oh, yeah, for sure.

26:09 Thank you, Dave. Super, we have we're working really hard it was we had too long a list and we got you know, one, we had to bump for Django 4.0. But the rest we got in so we were very Okay, so 3.0 async and await This is a big deal. Yeah. And so those are the big features for 3.0. And then what's coming in 3.2, it's got various other bits, bits and bobs, you can customize the primary key. So traditionally, they've just been auto filled, which is in 32, well, eventually, you know, you get 22 billion of those or something you can run out. So you can now customize that for big and over the next couple of releases, you know, major releases, we will make the default begin, because that's probably what it should be in 64, because then you're never going to run out of primary keys. But that's something that the big sites run into that, especially if you start creating, I don't know, event, an event log, yeah, you know, a site can generate a lot of events, and they can add up quickly, you know, so, again, functional index is in the in the ORM. So you can create an index on an expression like that these were greater than or that the sum of this was and then you can query on those fullspeed. Because their index, that's a really big feature,

27:16 oh, really. So you can do a query like the sum of the orders of this, the customer is greater than $100.

27:22 And that's a relationship. Yeah. And then you can create an index on that value. And you've been able to do that in your SQL for you know, any amount of time. But that's now exposed at the ORM level. And that's, that's awesome.

27:32 can just go that for people who are doing reporting or or that kind of thing. Yeah, that's massive. This portion of talk python to me sponsored by Linode. Simplify your infrastructure and cut your cloud bills in half with Linode's Linux virtual machines, develop, deploy and scale your modern applications faster and easier. Whether you're developing a personal project or managing large workloads, you deserve simple, affordable and accessible cloud computing solutions. As listeners of talk Python to me, you'll get a $100 free credit, you can find all the details at 'talkpython.fm/linode'. Linode has data centers around the world with the same simple and consistent pricing regardless of location, just choose the data center that's nearest to your users, you also receive 24/7 365 human support with no tears or handoffs, regardless of your plan size. You can choose shared and dedicated compute instances. Or you can use your $100 in credit on s3, compatible object storage, managed Kubernetes clusters. And more. If it runs on Linux, it runs on the node, visit 'talkpython.fm/linode' or click the link in your show notes and click that create free account button to get started.

28:45 And then we've got I don't know a new memcached back end for the cache. We've got you know updating the the API for using the admin for nice decorator API for creating various admin customizations. We've got themes in the admin going to ship a dark theme is a big one for

29:00 People see right away and is to because yet the Django admin could use a refresh and there's been all sorts of third party ways to customize it but now be built in

29:08 there's all sorts in the in the blog post for the arthritis. I described it as a mess Berg new features in that's exactly what it is it but what what's nice is it's not there aren't apart from async, which is coming. There aren't any new major features in Django, right? It's 15 years old. It's rich and mature and features largely feature complete. But each major release each eight months isn't it always amazes me when we're drawing to get up the final release notes, how much depth and extra like substance there is in the features that have managed to add and that have been contributed over that eight month been as fantastic.

29:41 So I feel like you're capturing much of the modern Python awesomeness. One other area. That's, I think, interesting. I have no idea what your plans are. But maybe you could just give us your thoughts is type hint type annotations. Yeah.

29:56 That's difficult for us,

29:57 Carl about that a little bit.

29:58 Yeah. I mean, that's difficult for Because Django I mean, if you take your it's super dynamic, and there's an amazing project, .Django stubs where they've got stub files, and they've got your nice on a super job. And it really does work well. So if you load up VS Code or PyCharm these days, you get very good autocomplete around things like Django model field definitions. And that's all powered by Django. Yeah. Which probably is powered by type hints underneath, right? Yeah, yeah, it totally is. a year or so ago, there was some discussion about whether we would make those inline hints in Django itself. And the technical board at the time said, No, we need the typing technology to evolve a little bit further in, in Python itself. Before Django can jump on a jump on and say, Look, we endorsed this particular technology, there is how there's various ways of doing it, and various type hinting type checkers and various and so so we favored my PI, which is obviously the endorsed one. But what about the others? We can't support them all. And they might change in pay rate and all the different various initiatives. Yeah, we can't take it out once it's in the thing we have. The reason why people love Django is because it's super stable. You know, you write you write a site five years later, you don't really have to do very much to keep it going. And so the technical board at the time said, No, we're not going to bring those type hints into Django, but that will be reviewed. You know, we'll look at it again in a little while time. One thing I will say about typing in all the peps, it says, type into the remain optional. They're not meant to be compulsory, even by convention. I feel maintaining Django a certain pressure to have them I do worry that that optionality of type hints is perhaps undermined a little bit in the day to day development. I'm not sure. I also code in Swift, right, which is only statically. typed

31:38 Swift is pretty strong in its syntax, like you can't even know out yet. Even if they could be right. If there's not allowed to be optional, explicit. There's a lot of interesting stuff happening the swiftype

31:48 Yeah, I mean, Swift is a fantastic language, but there's no option. It's type hints, or its type annotations all the way. And then when I come back to Python, after a few hours of that, I'm like, this is so fresh, you know, the thing that I

32:01 really the comparison that I make a lot is between TypeScript, which is type annotations for JavaScript and Python type hints. And I always walk away from TypeScript feeling like, Ah, that was way harder than it should have been. Why was it so frustrating and fighting me, whereas in Python, you might get some squigglies? Because you didn't quite define something just right. Or it wasn't defined at all, but then it just, it carries on, right, where's things that are a completely it must work within this context, they can become challenging, especially the reason I bring up TypeScript is the stuff that it brings in is not necessarily typed just like Python, whereas Swift is end to end. It's all from scratch under that same system. So it's coherent, but you know, bringing us a jQuery into TypeScript. Like there's no TypeScript, jQuery. I know, there's like stubs you can put on and whatnot. But like, that kind of stuff is more Python, like it's I think it's pretty interesting.

32:51 Yeah. And Django is in that very much in that boat where it was never written with type hints in mind. Yeah, of course they didn't. And if you look at Django stubs, it's awesome. But a lot of things are like string or any. And it's like, that's horrible, right? You don't want to write that every single time. That's an issue. And then another thing I see, which I'm not sure about, sometimes they make a type like a model, the Django model admin, they make it a generic that takes takes the model type, which tells you what it's the admin of, and I remember, and I look at that, and I went a little bit like back to C++ templates or something, right? Yeah, yeah. And I see where they're going with that. It's amazing what they've been able to do that. But I remember coding Objective C, UI TableView. sec. And what you would always do is you get object, your row class in from the UI TableView delegate, where you getting a list of sales, right? So like a list of email messages, and you'd immediately cast it to the type you wanted, so that you knew what you were dealing with. Now, in a way, when I get something back from a model admin,

33:46 I'm quite happy to write colon model name. Yeah, to tell the the editor what it's meant to be to tell the type checker what it's meant to be. And then for the rest of the method, I get the autocomplete, I get the type checking, and you probably don't put any more type hints because it's just now float. And it's Yeah, I agree. It's so good at General, like allowing you to generate code incredibly fast if the tool supports it. One quick comment or thought that before we move on this having typed around the ORM is really interesting, because all the big ORM's seem to do the same basic flow, you know, I'm thinking, you know, Django ORM, SQLAlchemy, the even no SQL ones, like Mongo engine, which I use, they all have a class, which has the columns, defined as descriptors. So at definition time they create the tables are the collections, and then at runtime, they become the scalar versions of the thing. They say they are right integer column is now actually an integer. So what I've done when I define the model class is say like, name, colon, stir equals string column. And so the Python thinks it's all the types are actually what I said, the primary types, like in the model, and then the underlying ORM can do what it needs to do. But you know, the rest of my code is like, Oh, that's a blue column right there or flowfield That's been really helpful. I don't know. Yeah, I haven't run into any problems doing that. But it's been pretty helpful.

35:05 Yeah, I mean, it's the thing that that brings money to it. Well, there's data classes, right, which they bought in 3.7 or 3.8. Yeah. And then Pydantic is exciting thing, which is Pydantic. Fantastic. Yeah, you define a model, call it a model. With exactly this. It's like field name, type, with a annotation, string, or int or whatever. And that works really well. And I can imagine us generating Django ORM models from that kind of thing. But then you end up needing options, like, Is it nullable? Or is it Yeah, you know, is it required? Is it what validators does this field have? And then it starts when you start to add validation, it starts to look almost like what we've got now. I mean, Pydantic got one good advantage in raw serialization speed, it's very fast. So that's something we can learn. But the short answer is, yeah, I don't know. But it's exciting times, isn't it? Very exciting times.

35:55 All right. Let's talk really quickly about maintaining your your content or things that you generate, like you guys all write, articles, books, do online stuff, even podcasts. So 3.2 is out what broke like real quickly? Well, maybe you want to just talk about what you've been doing to, you know,

36:13 yeah, there's a really up to date Django books, because the release cycle of every nine months doesn't overlap with the traditional publisher cycle. So for me, I mean, I'm on version, I think, five for most of the books. So yeah, I was on 110, or 111. So I've, it's sort of like writing the book again, every time I basically go through from scratch. And because all the code is linked to the text, because I'm self published, I can do all that I have my flow down. So really, it's I have a kind of a list of new features, I know that are in there. So I'll play around with them to make sure that things don't break. And then I'm constantly emailing with readers. So I have feedback on kind of what works and what doesn't. So half the new features are fixing things. So it's smooth, and then half just making the text. So at this point, I feel pretty good about the flow of all three. And people say what about this, and maybe I can explain a little bit better. I added, I mean, so for three one, I added in the beginners book, I really wanted to have proper deployments, but not go as deep as I do in the profession. So I did introduce environment variables, and showed, I think, a pretty elegant way environment variables and to have some lockdown for deployed site. Whereas for beginners, it was just sort of like the local version, which is pretty thin, was what you deployed, because it was more about getting something up. But I was able to introduce environment variables, we could talk about that more, there's a third party packages. Great. So so that's always the tension for me is showing and telling, right, like I will, I will explain everything, but I don't want to overwhelm people. And so that's part of the thing I just think about for 3.1,

37:43 I think also for Django being so stable, at least you can say, all this stuff is still the same, there's just new features we haven't mentioned or a new way, maybe it's better, but as opposed to this does not work anymore.

37:54 Well, it's the difference between what trips, the professional programmer, and someone who's new. So for example, in 3.1 the path was added when you create a new Django project, the settings.py file, which is the default settings, the way the routes are done is a little bit different. A minute StackOverflow thing for someone who's used to using Django or breaking stuff, but that can completely derail a beginner, there's usually a couple things like that, you know, so in that case, like I have a dedicated blog post, because I knew that was coming. And just this morning, I got more questions around it. So it's that sort of the the thing that funny is, how do I, you know, be compassionate to the true beginners, but

38:31 we're more

38:33 further along. It's a lot of work to do. I don't know how you could do it.

38:36 I don't either. I was just dealing with some GitHub issues on some folks who are having trouble because a mismatch of pip upgrades and other packages and it's just like, just how it goes real quick, john, from the live stream, really excited about the admin themes. So that's really cool that that it's gonna come along.

38:53 Thanks, Shawn. Did they add up to greatness? I think that's, I love that.

38:57 Yeah, that's a great, great say, All right. We talked a lot about deployment already, which is why I've been dancing around a bit. But let's go ahead and talk about deployment. So when I create a web application with Django, we're really honestly, with flask or anything. I've got a cool app. But it's, you know, web apps are not generally meant for the individual. They are meant to go on the internet and be amazing and connect things and allow other people to do stuff. And that's a huge gap. Right? So maybe I've just learned how to do Django, you just talked about your beginners book will. And I finally got that one page that wouldn't show in the database stopped, you know, crashing when I did that one query, because I checked for none now and it's all good. Wait after learning Linux, and I have to become an admin. What the heck just happened? Right? So how do you guys approach that? Like, what do you tell that person?

39:42 Yeah, first of all, I hand on shoulder, I feel you because it's just there's this massive deployment gap. Like we've got the Django tutorial, we've got the Django girls tutorial, we've got the Django rest framework tutorial. People have done those. They've got their app together. They come With some ideas, I've learned a teeny bit of JavaScript to make it a bit better. And now put it online. And exactly this. I've got to learn Linux. Oh, my word.

40:08 Yeah, among other things. I mean, two things, right. I think I think like when I think about others, there's two ways to think about one is making a movement friendly, which we can talk about. And then there's the specifics of do I, you know, do I do it on a VPS? Do I do it on a pass? So I there's two separate things. The first one, I think we can generalize, and Django has some good notes on that. The second one is sort of a deep end of opinion,

40:32 I guess I'm in favor of like I use, I'm a big fan of platforms. If somebody had something like AWS for Django, I would use that. I mean, so I'm just going to throw it in here. Now I'm working on an app for this exact problem. It's called button is 'btn.dev'. And it's the idea is to be able to spin up a small environment for this exact I just want to get my hat on nine use case without, you know, you're having to become a sysadmin. Expert. Yeah.

41:00 So the spectrum looks like to me pass right platform as a service, which you said you like Heroku. And there's other options as well. Yeah. Heroku digitalocean, then I think maybe button is even before that. I'm not entirely sure. Like in terms of complexity,

41:15 let me jump in. You've got the platform to serve. You got Hiroko, which is there at one end. And that's kind of like they'll run your app. But then yeah, that's it right? Or the you know, you can run another worker instance. But then you say to yourself, I want to I just want to put a file online. It's like, well, how on earth do I do that? I can't, or I want to put up a static website. So I've got to bring in I know netlify, because I another service. And

41:36 yeah, 'Netlify' is really nice for just pure static sites, actually. Yeah,

41:40 it is. It's awesome. But then all of a sudden, you're running two services, and then you want to do some log analysis. So you've got to hang on, I've got three services now running, and it's, it gets a bit out of hand. So then on the other hand, you've got AWS, and you go to AWS. And it's like, oh, my word. You know, it's the Paradox of Choice. It's just like that in Azure.

41:59 I mean, they're both incredibly powerful. But it's just like, oh, my goodness, look at this admin dashboard. Like I there's huge thing with a 50 choices, I go into one, and it expands into 20. That I need to figure out the right one of those 20. It's just it's mind boggling.

42:12 Yeah, exactly. And so battle where button fits in that is it on AWS, but it provides a very simplified invoke this, because it's part tool is a tool to do, but also a guide that says, Look, this is how you should deploy. So the app puts together the environment for how you should deploy your Django app or you know, anything that you can do any app. But you know, I begin with Django, because that's where, obviously what I know. And then the difference between Well, why wouldn't I? Why would I use that instead of a platform as a service like Heroku? Well, because if you grow out of it, it's just AWS. And so you can go on and on to other things. And you can export to when you do go closer to

42:49 now you are kind of Google or whatever, you've got a growth path, obviously, not not far down that road, but your little ways you can go, Oh, we actually need to do this other thing.

42:57 Yeah. And so that's what I'm aiming for. And that that's launching very soon to be early access by the middle of February, you know, beater by end of the quarter. And then what I call version, what ends the beach, I don't know, end of the year, I would imagine. Yeah,

43:10 yeah, that sounds really cool. We'll

43:12 definitely link to it in the show notes. When I was pausing there. I said, Okay, well, on one end, we have PaaS like Heroku and other platform as a service options. What is the other end is the other end Linux virtual machines, or the other end Kubernetes clusters? It's Kubernetes clusters, it is well beyond virtual machines, right? So like to spin up a virtual machine is not a problem. The problem is that you get a Barrow s, and then you've got to do all the apt get install, turn it into something you can deploy on, right? And then you do that once. And then the second problem is that six months later, you need to upgrade it, and you've got to replace that VM. So that's very difficult. And I think that's what leads people into this kind of world of containers. And then into containers. It's like a wall I've got I need an orchestration platform to and that's there are people who make a career doing that. Yeah. You can't expect to do that sensibly.

44:06 I agree with Carlton. It depends slightly and that like so in my professionals book, we I show how to use Docker. And Docker is for a smaller site, I think, actually, you can put your containers online. So you can a lot of different flow is actually quite nice. But I agree with Carlton's broader point about, you know, patties and all the rest, it's

44:25 I feel like is the goal, I'm setting up a Kubernetes cluster and I'm running the data layer and I'm running the caching layer and I'm if you're managing that whole thing in a cluster, you're at the far right end of the most complicated, but if I'm going to deploy to a hosted Kubernetes container and consume a Database as a Service in a hashing tier as a service, then it starts to fall back to Well, I just get my container run and plug in these connection strings and we're good to go. I think there's it can kind of live on both sides. Honestly,

44:52 there's different approaches and different sweet spots for each individual. The difference for me is you can containerize your application and run it up but when Do you contain at that time? Where do you run that container? Is that on a set of VMs? Which you provisioned yourself and put a container management system on like Kubernetes? Or is that on somebody else? Like, is it basically service? The cloud run cloud run takes your container container and runs it and you don't know anything about where it's running. So that's more like a platform as a service again,

45:20 yeah, it is it falls back in that. So will said he's a fan of Heroku. Carlton, let me ask you this question. If you were writing an application in Django that was expected to get 5 million data driven, like proper requests, not for CSS and whatnot, but proper requests a month? Yeah, how would you deploy it right now,

45:36 I would deploy it on a pretty big VM with as many process workers as I need. So I'd put stick nginx, as my front was my front end, perhaps. And I'd probably use something like application load balancer in front of it, one of these things that AWS will provide or as your got the equivalent load balancer, then I'd have my nginx, which is serving my static files behind that I'd probably have two three. It's also doing SSL, for example, right? Your, your Let's Encrypt bits, yeah, yeah. And they're handing all certificate renewals and all that kind of things, which is probably worth the money because it only takes you a few minutes, but you forget, and then your site goes down, and you know, just handle that stuff off. So I'd put that on a nice size VM, I'd have, you know, 345 ganache home workers serving the application instances, load balanced, behind nginx. And then I'd have Postgres on a decent size RDS, which is a managed database service behind it as a first preference, I would scale up the VM, and I would scale up the RDS rather than way before you scale out. Yeah, way before scale out. Yeah,

46:37 I agree. If you can, as soon as you have them scaled out, it gets tricky, right? You because You it takes you up a notch in that DevOps story.

46:44 At that point? You you're hiring, you know, an extra hand just to manage the ops an extra hand to manage your site? Well, what was the revenue of the application that justified that the salary and the social security of that extra hand?

46:55 Yeah, maybe you're like, well, I don't really want to pay for that larger VM, that's $80 a month instead of $10 a month. But if the alternative is now we spend human hours on making, you know, 310 dollar VMs run, it's so easy for businesses to become Pennywise pound foolish around these kinds of things, right? It's just like, well, we can't get you a new monitor, and he cubieboard like, really, you pay me like $80,000 a year, you won't give me a new keyboard to be more productive? Or I feel like that's the deployment equivalent of that. Yeah,

47:24 you know, it's it's Penny pinching for no reason. I mean, the other great example it's century right. So century is your kind of, look, you log all your errors to its Andrew exceptions to it, it gives you a great stack traces, it links it back to GitHub, it's awesome. Why would you not have this and then you're, you're on the free account, you know, give you 5000 events a month or something? And it's like, oh, we need 30 bucks to get on to the like, the million events a month, the man is like, No, you can't have paid century like, What?

47:48 What are you talking about? Stop logging you like, well, we can't log that event, because of century and then you you're blind, and then your app goes down. And you spend four hours trying to work out what it is, while you're losing business. Like that was 30 bucks a month, the century just pay it. But I think

48:03 if your manager has felt that pain, individual contributor themselves, right, because if they haven't, then yeah, stop complaining developer, you know, yeah, you should, I always thought I mean, and I Quizlet, which is now like, half of all high school students, we had giant, which I always tell around their hosting company, but we're having scaling. They came in and we had the whole team CTO, which was great. And, you know, at the end of the day, they were just like, you need bigger hardware. You know, if you can get more, you know, as a business person, if you can throw money at a scaling problem, like, yeah, that's the easiest money you've spent. Yeah,

48:37 I think you know, so what I'm doing right now, Carlton is I'm running nginx, on Ubuntu but I'm using micro whiskey yoke. And I came across this, although, as I'm starting to think about the as gi stuff, doing acquiring g unicorn, and like UV corn worker processes is starting to look like a really awesome thing. But what I wanted to point out is there's this great article by the Bloomberg tech folks talking about configuring micro whiskey for production. And man, do they have a bunch of good little tips of like, Well, here's all these things. But how did you know if you turn on like enabled threads versus not or if you set single interpreter mode to true, you automatically get better performance because it was configured to potentially run like Python3.7 and 3.8 at the same time. And you're never going to do that in one process. Most likely, there's just all these really fantastic settings and like vacuum and whatnot to clean up sockets and whatnot. So people if they're running micro whiskey game production, they should absolutely check that out. That's a really, I wrote a bunch of the way I was doing things after reading that

49:37 just scroll to the image at the top if you would, please, Michael. Yeah, like yeah, so that looks almost exactly how I deploy nginx in front of a worker process. You know, that's running your Python applications and you know, talking to your database.

49:51 Yeah, exactly. It's,

49:52 it's pretty standard. Yeah, they've got memcached in there. So I'd have Redis instead of mem cache, but same deal.

49:57 Yeah, there's a comment about does EC 2 sit between Have a VPS and Kuber. Yeah, Kubernetes. Since your AWS expert, okay, well,

50:06 I don't have my expert, but so but on AWS like you, this is why I haven't found in AWS, I would love to love their lightsail product, but it just seems totally uncared where they kind of manage the cluster all for you. But on the I can't even remember what the cluster services called. But the you have to provision the EC2 instances. And you can do it quite easily. It's not hard to do, but you have to spin those up. And they're kind of your responsibility. And then it will run the cluster service on top of it, the Kubernetes on top of it, and then you deploy your containers in there. This is where something like cloud run is a bit easier, because you don't have that layer of I'm provisioning the underlying instances myself, and you have to choose the size of them. And you can scale them up easily enough. But you know, it's Yes, it's non trivial to set up a cluster on AWS, that's how I spend my phyllanthus. I also feel like you're sort of committed to their hosted services. And if for some reason, you don't want to use them, the step is not a tiny bit more, it's a bigger, much bigger step to like, try to rework that. Yeah, the example is running, you can run your own Postgres on EC2 instances, and you know, you can provision the disks and you can handle that, rather than RDS. And for me, unlike Okay, if you've got a very specialist use case, then yeah, do that. But RDS is great. Just use RDS. Just use the hosted service, because, again, you're saving money. And yet it's a bit more expensive hour but not as expensive as you your time, your life force. Yeah.

51:29 Well, I think also, it depends, right? Are you one single person who is building up an idea and you have zero revenue, and you're just trying to do this for two hours a week to see if you can get a little tiny bit of traction, maybe it is worth an extra five hours, right? versus I have actual money coming in the door. And I'm not building this feature, or that feature? Like those are really very different contexts. Also, though, you

51:51 get yourself a Raspberry Pi, and you can install nginx on it, and Postgres on it, and no, but you can run although, yeah, you can do it locally. And you can bet you can run a patchy venture going against it, and you it will serve plenty, so you can get not the nano one. But yeah, sure, but you can get you can run Postgres is my point, you don't need RDS to run a small Postgres. But yeah, if you're running a slightly bigger one, then yeah, use RDS. And then if you're a specialist, and you really need to customize your Postgres somehow, then Okay, yeah, you know, spin up the easy to install Postgres on it and customize it to your heart's content. But that's well beyond my skill level. And yeah,

52:30 this is why they have hundreds or 1000s of employees, right. Like when people say, why did the big companies do that? It's like, Well, why does Instagram have 1000? Whatever, 1000 engineers, when Carlton, you and I could spin up a, you know, prototype? relatively quickly. It's like, because we're not dealing with the traffic they are. And so it just gets exponentially harder, which is why you need a lot of people for all these issues.

52:52 Absolutely. Will you mentioned something really interesting there that I would like to just touch on for a moment, then we're getting short of time. You said you could run SQL lite?

53:01 Oh, well, so that Carlton has a good story about SQL Lite. I mean, about how it's So generally, people say we can't use it, you shouldn't use it at all. But there are some cases where that's not the case. I mean, if it's a read heavy application, you can get away with it. Simon Wilson, a lot with data set.

53:16 Yeah, I just had him on the show. His episode just came out this week, actually, yeah, he's great. And that is such a cool project. I feel like that is actually an undersold story for the beginners, right? Because deployment of web application often means deployment of web application plus database server plus backups of database server plus, like all that, like all of a sudden it goes from, you know, 20% to 90%, hard or whatever, right? Or you go to hosted, we're like, Okay, well, I

53:44 still got a backup that thing, potentially. And, you know, there's just you've, you've got potential migrations, I know, there's a lot of stuff going on. And if you can say, well, until you actually get much traffic, you can just say, here's the DB file and the SQLite connection on the one machine that I have, and you backup that file every now and then that might be a good story. If the alternative is it's too much for me to get my app out. Yeah, no, definitely, especially if it's a blog or a content site, like a shell catalog where it's all read only. And you know, maybe you're using the admin to manage the content on it. If it's only one person using the admin, you're not going to have concurrent rights, which is the thing about SQL Lite. And so it's never going to be an issue and it is fast. It's a read read only workload. It's fast. It's fast enough.

54:27 It's Yeah, it's very fast. Yeah, you can use and there's no server for people who don't understand are not totally aware. Like it comes with Python. And it runs in process. There's no other server to set up and connect to. Yeah,

54:38 so Carl did not at one point because I was saying Oh, yeah, NO SQL lite and he was so I have to give credit to Carlton for there are some instances

54:46 where he's the secret, SQLite. It's got a writer headlock by a wow mode, which means that actually you've got a good chance of being able to do kind of concurrent writes to Django's ORM has got it retry value, which if you set that, you know, you know, a little bit higher, if it gets database locked, it will try again in a second. And then, you know, you can go quite a long way enabling these things. And then when you finally in production, actually get a database was locked error. And you could think, you know what, I think it's time we moved to post. Yeah,

55:18 in my world, I'm running MongoDB as the database and it, it doesn't make sense to consider running that on SQL Lite. But I can remember back when I first started deployment, like okay, well, I got to learn Linux, I got to learn nginx I gotta learn micro whiskey like, Okay, what other deployment like, how do I learn running my database on these things, there's just, there was a lot. And I can definitely see if you could say, here's an intermediate step to get it out and get it going. And then you just change the connection string, at some point over to a big, separate server. I think that's a really good path.

55:47 It's a little bit like, you know, Roth's equal versus Django models. It's a timeline because I've had a thread emailing with a reader who doesn't quite it was has something in mind. And he's like, I modeled it all. The scheme all out and SQL, and SQL is easy to learn the basics, and really hard to scale. And while Jane Lynch SQL, you really, really, really should resist doing that, unless you're way better programmer than I am. Yeah. So whereas before, you would have to learn tons of code. Same thing with deployment, you can get a lot of the way there by trusting someone who says them until you need it. Don't bother the same thing with database stuff. Okay, you know, do some basic SQL, understand a little bit of relations, but that's the power of the Django ORM is that it will handle so much of this for you. And if you think you need to do custom, unless, you know if you have any doubts, you shouldn't do it.

56:34 Yeah. In the migration side, that's a huge part of a challenge if you were maintaining it yourself. Yeah.

56:40 Yeah. I mean, I was, you know, Andrew, Andrew was responsible for them. doing most of that a lot of the async stuff now.

56:46 Nice. Alright, guys, we're getting short on time. But I do want to just really quickly cover one more thing, servers up and running, feels like it works. I watched it all day, it was fine. I want to walk away from my desk, I want to know that it's probably still fine. And to me, like Carlton, you already talked about one thing. There's some really great monitoring services that you integrate at the API level, like you plug in as a library to your code century being one of them. They're really great. They sponsored the show for a while. So thanks for that as well. But I use century on Yeah, they're great. And I, I've got century Reddit on my apps, and every now and then I'll get an email and a lot at this point. When I first integrated that type of service. I got a lot of, Oh, I didn't know that was happening. Oh, crap.

57:26 I'm gonna have to fix that. That's why there's a level of logging and stuff, right?

57:29 Yeah, yeah, absolutely. And once I kind of dialed in now, it's more like, well, somebody tried to send like some binary hacky thing that broke the URL parsing, but it's not really a problem. It's just great. But having that thing there is really important, I think, yes, absolutely. Absolutely.

57:45 Century I, I just can't see a reason not to use it, even if they've got the free plan, why we have that

57:51 as a free tier, like, once you run out, you can decide whether or not you want to pay for it. But yeah, at least can get us some kind of insight into the errors that are happening.

57:59 Yeah, the other thing that I found is promethium takes a teeny bit of setting up. But again, you can get yourself a Raspberry Pi. And you can play with that locally. And it's not that hard, you could put it on your Mac, or you put it on your Windows machine, it will run but get a Raspberry Pi set up monitor the device, Prometheus is good fun. There's a dashboard that goes with it called refiner. And then they do a log thing called locky, which you know, you have to configure it yourself, but it teaches you about your application, you have to say, Okay, I want to monitor this. And then you can put a little couple of tags in, and then Okay, you get metrics down for that bit. So you know, perhaps you've got a slow view. So one thing you can do is you can put on your nginx logs, you can log the upstream response time, so you've handed off to garner corn or to micro whiskey, and nginx will then log how long those responses took. And then you can say, Well, actually, that those responses to that particular request is taking a long time. And then on that request, you can go into that view. And you can add a little, you know, a little bit of instrumentation, and then you can start getting metrics for what that view is doing. And yeah, yeah, you know,

58:59 he does ever all of you guys. Yeah, I'm also a fan of DataDog. I think those are pretty similar.

59:03 Okay.

59:04 I think they're Boston based, actually. But yeah, they do. Yeah.

59:06 They call it APM Application Process Monitor. And so there's quite a lot of commercial solutions. And then Prometheus is a sort of DIY,

59:14 nice equivalence. Okay, cool. So that's the errors and performance at the other side is just the website is down, right. So maybe there's so many different layers and moving parts, for example. And we have very little downtime with our train courses and stuff. But one thing that did happen is we have this thing that will take the main IP address and switch it to a failover server that we can do reboots and patches, and then it'll switch it back. And that's all done through an API through like a git push, and then all just sort of goes down the chain. Well, at one point, there was a bug where that switch of the IP like disassociated it with either server to the server thought it was fine, just like well, things are chill, right? Because he

59:53 law gave me so Yeah, no problem

59:55 about the site was just timing out or so I don't remember what happened, but it wasn't good. I think there's some really nice uptime monitoring things that that are free that people should just plug in and go hit these three pages just to see I can talk to the site, it talks to the database. And answer comes back.

01:00:09 Yeah, for sure. I mean, I've got those configured. I was I know, we're almost out of time. I did want to mention on deployment, I think one thing that trips people up, especially around Django is static files, which is something Yeah. But if you're doing it, you know, VPS. Yeah, you know, I think the key thing is that your server doesn't want to run your static files, like it can ways to do it. But you know, so both within Django, Django out of the box defaults to local settings, and the Django has a local server, they'll just run your static files. To put that into point and deployment, you would run collect static, which is a command that makes a single static file, and you probably want to use something like white noise, but there's a couple steps there. But something that really sort of intermediate level Django, people, it really trips them up. I have a blog post on that, because I forget the commands, there's like four things to tweak. You just have to do every time. But yeah, it'll work locally, why doesn't work live? That's like, partly cuz people don't understand, you know, how their servers working, because they don't have to. But you know, that's, that's, I just want him.

01:01:05 Yeah, awesome. Yeah,

01:01:06 one of the big four, right. So get your code in place, however you do that tarball, git pull, you know, whatever, you get it there, configure virtual environment, you know, so in pip install all your requirements, configure database, and make sure you connect to that and get your static files in place.

01:01:19 Now, the other one that big, you know, well, you know, this might go like beginners, they're like, well, I update it locally, and I pushed my code, why isn't my production database different? Right? And, like, I still occasionally make that mistake, you know, just but you know, your code is not your two separate databases.

01:01:36 Yeah, absolutely. That's what you want as well. You want to be able to write your blog post in locally, and then it appear on your server. You don't want to have to go Oh, no, now I have to paste it into the admin on my remote. Yeah, it depends

01:01:47 on a lot of ways you want to go I mean, for for simple, like somebody learn Django site, which is basically one admin, largely a content site. I mean, in terms of easy peasy, it's just Django and I've got CloudFlare in front for it's done. It's super fast. I get, you know, decent amount of traffic. I don't have to think about it. I've got I think I've got Pager duty one of those, you know, and

01:02:08 yeah, okay. Yeah, that's the one that tells you if it's, it's completely unreachable, right.

01:02:12 Yeah. But it's an external one that you know, exactly. To your point, because you can always trust yourself. Yeah,

01:02:18 yeah. There's some interesting ones. I use status cake. And it's neat, but there's some one that I heard that Dan Bader was telling me about that, which sounds a lot nicer, but I'm like, but this one's already set up. You know, it's not that important. I get I get notifications if it goes down. I'm

01:02:31 all good. It's plumbing. Right. Like, I'm happy to well,

01:02:33 we have the new silver pipes, like yeah, my bronze pipes. Fine. I'm just gonna leave it.

01:02:37 Well, that's Dr. Carlton's, you know, so.

01:02:40 loads.

01:02:42 Alright, guys. No all the time all the time. Let's just wrap it up really quick. There's a Django newsletter. I learned about that. That's interesting. You guys want to just mention that that exists?

01:02:50 Yeah. Yeah. So I there's something I run with Jeff Triplett, who's partner at rev says, he's a Python board member, very involved with Django. So Django-news.com. There wasn't a regular chicken noodle just wasn't a podcast. So he and I have been doing that for a little over a year and has projects articles. I sort of wish someone else did it. So I didn't have to do it. But yeah, we got a lot of people listening using it. So that's a good resource for the community. Yeah. Fantastic.

01:03:13 All right. So before I let you out of here, final two questions really quick, we'll go the first notable PyPI package out there that you are like, Oh, my gosh, I ran across this the other day? I can't believe it. Ah,

01:03:24 that's a good question. I'll go back to an old school on which I think is Bleach. Because I've been thinking about a course on forms. And bleach is a Python one, not gender specific. But you pretty much always want to have that added to this validates like user input to make sure there's no like cross site scripting type stuff. Yeah. So that's okay. Top. I'm not a new one. But I think, you know, along with white noise, you just got to use it. Yeah.

01:03:48 Awesome, Carlton. Yeah, no, well, the one that's just really captured my imagination recently is Rich, which is the heritage score, the library for creating basically nice terminal output console output, but it does everything even has like tables and also stuff. Yeah, it's really neat. But like, it's got this inspect functionality where you can get in here, you're sitting there and you're in the shell, and you're like, I want to see it and you print it, and you get it and you you look at the dict and it's all it's not and then you go inspect poor and rich formats, this thing where it's like, oh, yeah, I can look here, and I can see exactly what's going on. And it's just a map, which is 20,000.

01:04:25 That's what I love about coding, right? I'd never heard of this before. And it's clearly a very established thing.

01:04:31 Well, I think last year,

01:04:33 it's taken off quite Yeah, it's about a year old. It's taken off quite steeply in its adoption, one that I ran across really recently. Just throw it out there because it's a Django semi related one is disk cache. Have you guys heard of that? No. So it's a really interesting a caching plugin that will instead of using memory for caching, it will store it on to local disk because usually have way more harddrive space than you have memory in the cloud. And it plugs into Django to like a standard for the cache there.

01:04:59 Okay, point I mean, I'm a big fan of file cache. Yeah, in general, because like memory is expensive, right? So memory caches is the best way. And Redis and memcached are good ways of doing that all in memory. But then you think I just want to generate some HTML once and then just pipe it off the hard disk. Exactly. To me, this is a kind of the equivalent of SQL lite versus a real database. It's like you don't need Redis or memcached, or something like completely set up just like until it gets beyond whatever.

01:05:25 It looks pretty interesting.

01:05:26 Packages things, by the way. Like we include packages every week and Django news. And there's also like an awesome Django repo that Jeff and I do, you know, because the problem is you want some degree of curation? Right? Like something new. That's right. like legit enough to be worth looking at.

01:05:39 Yeah, absolutely. Thanks. Yeah. That's very cool. All right. And then final question. If you're gonna write some Python code, what editor Do you use

01:05:45 VS Code? Same for both of us? Yeah.

01:05:49 Well, I have to, yes, VS Code. Very nice. I use it a lot. Also, though, I still like BB edit.

01:05:55 Yeah, right. All right, keeping it old school every now and then just for a little nostalgia.

01:05:59 I tell you what, right. So I'm using VS Code. And I'm like, but I need to do some transformations. Or I need to, you know, do a multi file search from the diff here and vs codes, all that stuff. But it's a bit you know, this is built in JavaScript was BB it's a native Mac App. It's got a logo.

01:06:13 Yeah, absolutely. I use PyCharm. Yeah, I'm a big fan. local company here in Portland panic came out with so called Nova. I think it's Nova. It's pretty interesting as well. Yeah. It looks just beautiful. transmit. I love transmit. Yeah, I use it all the time. All the time. For s3 stuff. We're talking about fantastical. AI. Gentlemen, thank you so much for being on the show. Thank you for having us. Yeah, it's been great. And thanks for all the work on Django and I'll catch up with y'all soon.

01:06:42 All right. Thanks for having us.

01:06:44 This has been another episode of talk Python to me. Our guests on this episode we're will Vincent and Carlton Gibson. And it's even brought to you by Square and Linode. With square your web app can easily take payments seamlessly accept debit and credit cards as well as digital wallet payments. Get started building your own online payment form in three steps with Squares Python SDK at 'talkpython.fm/square'. Simplify your infrastructure and cut your cost bills in half with Linode. Linux virtual machines develop, deploy and scale your modern applications faster and easier. Visit 'talkpython.fm/linode' and click the Create free account button to get started. level up your Python we have one of the largest catalogs of Python video courses over at talk Python. Our content ranges from true beginners to deeply advanced topics like memory and async. And best of all, there's not a subscription in sight. Check it out for yourself at training.talk python.fm Be sure to subscribe to the show, open your favorite podcast app and search for Python. We should be right at the top. You can also find the iTunes feed at /itunes, the Google Play feed at /play and the direct RSS feed at /rss on talkpython.fm. We're live streaming most of our recordings these days. If you want to be part of the show and have your comments featured on the air, be sure to subscribe to our YouTube channel at talk python.fm/YouTube. This is your host Michael Kennedy. Thanks so much for listening. I really appreciate it. Now get out there and write some Python code