#224: 12 lessons from 100 days of web Transcript
00:00 Michael Kennedy: Back in May of 2018, Bob Belderbos, Julian Sequira, and I started what would turn out to be a nine-month project. We wanted to create a dedicated 100 Days of Code course specifically for Python web developers. Much of what we created for that course, we did have prior experience with, but with that many different topics, much of it was also new to us. On this episode, we teamed up to distill the lessons, tips and tools we found interesting on that journey into a quick list of cool tips and techniques. We hope you find something new and useful. This is Talk Python To Me, Episode 224, recorded July 29th, 2019. Welcome to Talk Python To Me, a weekly podcast on Python, the language, the libraries, the ecosystem, and the personalities. This is your host, Michael Kennedy. Follow me on Twitter, where I'm @mkennedy. Keep up with the show and listen to past episodes at talkpython.fm, and follow the show on Twitter via @talkpython. This episode is brought to you by Ting and Linode. Please check out what they're offering during their segments, it really helps support the show. Bob, Julian, welcome back to Talk Python To Me.
01:16 Panelists: Thank you. Good to be here, Mike. Nice to be here.
01:18 Michael Kennedy: Yeah, it's good to have you both here. We kicked off our acquaintanceship way back, I can't remember which episode it was. It was quite a while ago when we talked about this 100 Days of Code project, and journey that you all had been on and it actually, it's turned out to be quite the journey from there as well, right?
01:36 Panelists: Oh yeah, just nonstop. I think it was Episode 140, as well.
01:40 Michael Kennedy: Yes, it was 140, thank you.
01:42 Panelists: 100 Days survivors. It was quite a hardship.
01:46 Michael Kennedy: Yeah, so back then, you guys had been going through 100 Days of Code and documenting it and blogging about it, and then, you know, you came on the show to talk about it, and after that show, we were sitting around talking, like, you know, this is really cool. There's not a whole lot to support people specifically in Python to do it, so you guys and I, we all teamed up to write the 100 Days of Code course that we released last year, and that was really fun and people seemed to like it, so we wrote the 100 Days of Web in Python course, and that was quite the journey as well, and recently released that. So this show is going to be based on stuff that we take from those experiences, and I want to be clear, I don't want this to be just like an infomercial about the course. I want us really just to focus on the technology, but when we created these courses, this is like, it took us nine months to write that 100 Days of Web course and some of that was just the effort of recording, but a lot of it was research and polishing ideas and like, understanding things better, right?
02:45 Panelists: Oh, without a doubt, the amount of tech that, and skill that I had to develop in order to teach it, was insane. There were so many topics that I just, I knew a tiny bit about, but not enough to actually build something usable, and something I could teach, so yeah. A lot of that time, I would say 80% of that time was actually learning and trying to figure stuff out.
03:07 Michael Kennedy: Yeah, Bob?
04:41 Michael Kennedy: That's right, there's a ton of fun to be had, and so when we went through this course, we all learned a bunch of stuff, and basically, it's made up of roughly 24 four-day little projects and each one of them is generally a separate technology and generally standalone. And I said I don't want this to be an infomercial about the course, so I want to really point out that this is all, every bit of this is, we're going to talk about, is in a GitHub repository, and each section has its own little bit with its own readme, so if you want to go play with that, the GitHub repo is open and free, and we'll talk about like where each one of these topics lives in the GitHub repo, and I'll be sure to put that in the show notes as well. So with that in mind, Bob, do you want to kick us off with the first topic that you got out of the course, that you think is interesting for Python web devs?
05:29 Panelists: Yeah, sure, and I start right off the bat with APIs, and API Star. It's by the same author of Django REST, so I chose that framework as it's relatively new and knowing the author, I expect it to have sensible defaults, and it had. It was, yeah, a fun little project that makes it very easy to build your APIs. I got pretty quick results. They also needed some test data, for which I used Mockaroo, which is a service, or website, that makes it very easy to get fake data, and combining the two, yeah, it became an interesting project where I think I used some car data and built an API around it. One of the challenges, though, that I saw when we deployed the app, is that there were some breaking changes so yeah, one warning. When you use top-notch, open-source, off-the-shelf products, that they might change, and they might change rapidly. So I think in the end we had to pin the version in the requirements, so to make sure that people weren't hitting those issues.
06:37 Michael Kennedy: Absolutely, and API Star is really cool. It's by Tom Christie, and it has changed a lot, actually, even since we wrote the course and that's just the way it goes. That's what I was sort of hinting at, not specifically thinking of API Star, but there's a ton of these projects that are like that and you know, I think it's good to do, like, a survey of all of them out there, but it's also challenging to figure out which one is going to gain the popularity and the energy, 'cause they ebb and flow, but yeah, it's really interesting. I love this, the way that it works. One of the things that you introduced me to here, that I found super interesting, is a great service for fake data, because you know, as a course developer and trainer, having fake data is awesome because you don't want to use real data, but you want to have realistic data, but you know, there's not that many trainers or course developers but almost everyone has to have a prototype app, and you often have to build the app before you have the data, 'cause the app's going to collect the data, right? And so in order to build out the UI and the API and all that stuff, it's really good to have realistic data, so tell us about this place where you got the data from on that.
07:46 Panelists: Yeah, it's mockaroo.com and it has 143 data types as we speak, and it's broken down in location, commerce, advanced, personal nature, and yeah. I chose cars because that's kind of relatable. That's, yeah, a topic easy to resonate with. In this case, I chose car make, model, model year, and the VIN number, and yeah, you just pop them in the field names and then you can download it in CSV, JSON, tab-delimited, SQL, Cassandra, Firebase, Excel, XML, like all these useful data formats, and it generates a file, and then yeah, you can just load it in and start using it. It's really cool.
08:28 Michael Kennedy: Yeah, you named off some data types, and one of them was like, car VINs, Vehicle Identification Number, and that's like this big long specific thing, but it literally generates reasonably valid-looking VIN numbers, not just random values there, but like, actually what you would expect...
08:42 Panelists: Realistic.
08:42 Michael Kennedy: In terms of number of character, yes, exactly, and it has that stuff for medical bits, and like, it's really quite interesting all the data sources that it'd pull from.
08:52 Panelists: Yeah, indeed, and if I look at the API class, then that literally translate to the data, so I have a car class with id, manufacturer, model year and VIN, so it made it relatively easy to build an API on top of it.
09:04 Michael Kennedy: Yeah, that's cool, so if you're out there building something and you need data for your API or web app or whatever, Mockaroo, that's pretty cool.
09:12 Panelists: And one thing I also wanted to highlight in this section is Postman. It's a GUI to test your API, so that's another thing. I used it before, but I guess I became better at it because I had to use it again, and yeah, it's very convenient to test your API and GET and POST requests from a GUI.
09:34 Michael Kennedy: Yeah, and you can do it from your dev tools, but in Postman, it's so nice. It lets you set the headers just right, it lets you do all the HTTP verbs, you know, POST, DELETE, and things like that. For Talk Python, I have a whole suite of APIs I have to maintain, for most of them are in the courses, and I have a whole, like, segment of saved, like, development and production stuff in Postman for testing that and make sure it all works, and yeah, I definitely can recommend Postman as well. Yeah, awesome. Alright, Julian, you want to talk about this next one? Like, I'm really excited about that, I just can't believe how much we get out of this next one here.
10:10 Panelists: Okay, so I'm really excited about this one. This is Netlify. Sounds a bit like Netflix, but it's Netlify. It was a static site-generating deployment website. It's just like a jack of all trades. I absolutely loved it, and the funny thing is, as we touched on before, I knew nothing about this. I didn't even know it existed before this course, as we were planning the course, you showed me this, Mike. You said, "Hey, if you're doing something on static sites, check this out."
10:37 Michael Kennedy: Yeah, originally, your goal was to just cover static sites and like, what do you use, Pelican or something like this?
10:42 Panelists: Yeah, Pelican.
10:42 Michael Kennedy: Yeah, that's right, and I'm like, oh, if you're doing Pelican, you've got to check out Netlify, 'cause that looks super interesting.
10:47 Panelists: And I thought, oh, what is this? What's this thing Mike's pointing me to? And when I went and checked it out and actually had a go, it was fantastic, so to give anyone who's listening the general overview, is, it's this amazing tool where you write your static site in GitHub as you normally would, or whatever, and once you have it built in there using Pelican or Jekyll or Hugo or whatever else you want to use, you then point Netlify at your GitHub repo, so there is a little hook in there that links into your repo, and it just monitors your repo. It's this continuous delivery, or continuous deployment-type tool where once it detects a change in your GitHub repo, it'll build your site for you and deploy it on the net for you, so just completely hands-off. You don't have to do a thing, and that was probably one of the most satisfying parts of the course, was when I was demonstrating that, or recording it. There was that moment of fear where I thought, oh, is it going to work, please work! And then it just worked, and so as I'm recording, I generated the site, I pushed my GitHub code to GitHub and then it actually created the site within a minute, and yeah, it's just, it's live up on the net. They give you a free URL, so it's obviously some garbled rubbish, that's your URL, and you can obviously then go and purchase your own domain and add all that in, that's all extra, but from a free-based standpoint, you can run your static website on the net using Netlify, and it was really cool, I really enjoyed it.
12:25 Michael Kennedy: Yeah, it seems like just such a nice way to do things. To me, it feels kind of like what Heroku is to Python apps, like you can just git push heroku and it takes your app and figures out how to run it. Like, Netlify is that for purely static sites.
12:42 Panelists: Yeah, exactly right, so I think one of the most surprising things for me was going through the documentation on their website and just seeing how I guess fully documented everything was. It actually tempted me, 'cause we use Pelican for PyBytes, that's our static site framework of choice for PyBytes. So I was familiar with it, but once I got on their documentation and started going through it, I'm like, ooh, I want to try Jekyll, 'cause I know Bob's mentioned that. He uses it for his blog, and then I've heard good things about Hugo, and I just, there's this massive list that you can go through, and so if there's anyone listening to this that has a static site, I totally challenge you to give this a go, because it was one of the most satisfying things.
13:26 Michael Kennedy: Yeah, it's super cool, and I can't believe what they offer for free, right? I mean, just to be clear, we have no affiliation with Netlify. They didn't pay us anything for this. Just like, this is really cool, I can't believe that it's actually available, like, so, custom domain, HTTPS certificates, continuous deployment through GitHub, all of that for free, it's crazy.
13:47 Panelists: Yeah, it's a beautiful tool.
13:47 Michael Kennedy: Awesome. Yeah, so if you got anything static, that's definitely the way to do it, that's cool. Alright, so I guess I'm up next for the next one here. One of the things that I wanted to focus on was just foundational HTML, because I want to hear your two opinions on this, because to me, I for a long time didn't want to get into web development because I knew I was a good developer, but I didn't feel like I was artistic in the graphical visual sense, you know? Like I was always, like, bad at art in school and whatever. So I felt like I'm going to have a hard time designing stuff, so HTML and CSS and all that, like, was, until I took the time to sit down and properly learn it, it was actually not a huge roadblock, but so I would always try to find, like, what is the way I could do this without really writing HTML? Like, what little widget will generate the thing that I want to work, like the calendar, 'cause I don't want to write, you know what I mean? Like, that was me for a long time. But then once I learned HTML and CSS super well, like, it became this thing that I was happy to do and I felt like it was made, like, getting to the stuff much easier and took away all that hesitation, so I thought it was great, so I wanted to talk about, with the history of HTML, and there's just some stuff that surprised me there, that I thought I would cover here. So in the GitHub repo on days 5 to 8, in there, there's a section where we go and recreate yahoo.com and google.com from scratch, as they were in 1996.
15:16 Panelists: That was cool.
15:16 Michael Kennedy: It's cool, but oh my gosh, can you guys believe what those sites used to look like?
15:23 Panelists: Trash, I don't want to know.
15:24 Michael Kennedy: It's so weird, I look back to 1996 websites and I'm like, wow, it was really easy to actually create a web startup because, like, Google and Yahoo, we recreated the sites in 15 minutes or something, you know? And it's just like, wow! Really, 15 minutes, okay. And now Google is a highly interesting technical foundation, but Yahoo's like a directory listing, like, you could've finished it up in another hour. I mean, it was really interesting, but that's all it was, and so some of the things I found interesting as well is, back in 1996, you couldn't use spreadsheets. Spreadsheets, sorry, not spread, style sheets. Style sheets were not introduced until 1999.
16:05 Panelists: Wow, I didn't know that! Wow.
16:06 Michael Kennedy: Yeah, all the HTML attributes had to just be set in the DOM, like over and over and over. Yes, exactly, like you would for like a newsletter, 'cause they don't allow style sheets. So that was crazy, and then the other thing was, as we look at the history of HTML, like, there was a lot of rapid growth from 1990 'til 2000, there was a bunch of versions, you know, every couple years or something like that. In 1999 was HTML 4.0. In 2000 was XHTML, which is an attempt to make all HTML documents formal XML, like a superset of XML, so every element had to be closed, every attribute had to have a value with quotes for the value, and things like that right? So that was 2000. The next change to HTML was 14 years later, so HTML didn't change for 14 years, even though this was like, basically on the heels of the dotcom boom and everything. Like, what happened there, right? So in 2014, HTML 5.0 came out. Now, I know there's probably like super minor little changes but like, no important changes happen between 2000 and 2014, literally in the specification.
17:16 Panelists: That's one of the things I actually, kind of off-topic, but what I love about this is that you did all this research, and discovered all this stuff, and then I got to go and learn it through you, so this is all cool stuff, because, and I hate to say it, Mike, but you know, back in those dates you were talking about, I was still pretty, I was still in school, I'm sorry.
17:36 Michael Kennedy: Well, you could've been one of those kids that started a startup and dropped out of middle school. I'm just kidding.
17:43 Panelists: Should've.
17:45 Michael Kennedy: Could have, should have, would have, right? What are you doing, I'm going to go to college next year. Yeah, I'm going to take my yacht on a tour because my business is doing pretty well.
17:54 Panelists: One day, I'll push that on my kids. No, I'm kidding.
18:00 Michael Kennedy: This portion of Talk Python To Me is brought to you by Ting. Let me tell you about Ting, a new mobile service available in the US that's targeted developers and other technically-savvy folks. First of all, their average customer only pays $23 a month, but they're no discount provider. Their service runs over T-Mobile's and Sprint's fast, nationwide network. If you don't use that much data because you're usually on wifi, like many of you are, then Ting'll save you a ton of cash, but don't worry. You can still use as much data as you like for just $10 per gig. One mobile feature I use all the time is tethering, and with Ting, you get unlimited tethering at the same data rate with your account. $6 a month for a phone line, $10 a gig, $3 a month for text if you usually chat over iMessage or WhatsApp. Think about it, no contracts and super clear and fair billing. Visit python.ting.com, that's python dot T-I-N-G dot com, and check out their savings calculator. Enter your usage and see exactly what you'd pay. Use that link and you'll get a $25 credit to try them as well. That's python.ting.com, or just click the link in the show notes. Alright, Bob, you're up next.
21:28 Michael Kennedy: That's cool. What was your thought of React, coming from a Python background? Like, did you like it? What's the general programming model feel like?
21:39 Panelists: It was a bit confusing at first, but then I did start to appreciate it for its design philosophy, especially how it treats mutability. So I did end up liking it, yes, but it took a little bit to get into. There's a lot of moving pieces.
22:24 Panelists: Yeah, JSX, right? The new syntax, yeah. That's a little weird at start, I agree.
22:30 Michael Kennedy: Yeah, I guess if the tooling supports it, right, if the tooling treats that little HTML block as HTML, then it's not so bad, but it's definitely a mind shift, yeah?
22:37 Panelists: And there are tools to bootstrap your app. For example, I used create-react-app, which I think is from Facebook, who invented the whole framework, and that did help, because you get all your defaults in place and you can just npm start and you have a basic structure in place, so that really helped.
24:09 Michael Kennedy: Cool, cool, Alright. Julian, you focused a lot on Flask, so you would cover one of the plugins for Flask called Flask-Login, right?
24:16 Panelists: Yeah, so this one was more like a bucket list item for me. For ages, I've wanted to deal with how to log into a website using Flask, and I got to tell you, Mike, this one, I wanted to rip my hair out by the end of it. It took, I think it took me a solid month to wrap my head around it. For anyone who's not familiar with Flask-Login, the actual Flask-Login part is the easy part. Everything you need to learn before that is SQLAlchemy to build yourself a database, to build yourself a model for your users to be able to log in, and I'd never dealt with any of that before, so this was one of the major learning points for me in this whole challenge, this 100 Days. But it was the most satisfying thing to figure out. When you look at my page, as you said, Mike, it's HTML from back in 1996. I think my page looked even worse than that, and it was, but to see a logged-out and a logged-in sort of view, just, I think I had a few beers that night to celebrate.
25:22 Michael Kennedy: Nice.
25:22 Panelists: Yeah, it was one of the most satisfying things because it's actually really simple, once you get your head around it. And there is a whole lot of SQLAlchemy you have to learn or I had to learn, and I teach this, before you get into the Flask-Login, and that was the hardest part for me, as I've just said, but if you can get through that, and if you know that stuff, then the actual Flask-Login's pretty easy. I think one of the greatest parts of it is, as with Flask, there's a lot of stuff with decorators, and there's this amazing little decorator you can throw on any of your functions for the webpages that you're going to be running on your site, called, I think it's just login, or is it, no no, it's login_required, login_required. You throw that on any of your pages, any of your routes, and if someone tries to browse to that page, it'll check to see if they're logged in or not, and it was as simple as that. You sort of had to do nothing else to your site to be able to do that, to get the login page to work. That was probably the most satisfying part of using Flask-Login, so that was a really good one, and if you haven't given it a try, just give yourself a few hours and give it a chop, because it actually is quite simple to use, and once you implement a logout page to create users, so that people can actually register an account, tie all that in with an SQLAlchemy database backend and this thing just is beautiful after that.
26:51 Michael Kennedy: That's cool, yeah, it sounds like a really nice way to do it, and these plugins for Flask are definitely cool. It also, if you're using Flask anyway, there's a high probability if you're using SQLAlchemy also, right, like Flask is a microframework and doesn't come with some ORM, so as long as you're using a relational database, the chances are very high you're using SQLAlchemy. I mean, you could be using something else, like there's Peewee ORM, which is nice. You might be using something like MongoEngine from MongoDB, but you know, most likely, SQLAlchemy, so you're already kind of in that space, right? So here it's just the user aspect being handled for you. That's cool.
27:26 Panelists: Well, the tough part as well is, if you're a brand new person to Flask, that getting from that single-page Flask app to running a fully fledged Flask app with your directory structure and views, and also having a database, there's a massive leap there that you have to make, so get the SQLAlchemy down, that's the first part, then work on your plugins.
27:48 Michael Kennedy: Yeah, absolutely, I feel like I'm not sure what the right way to put it, but I feel like Flask sells itself as being so super simple, right? Like I create the one file, I create app.py, I can call the Flask thing to create the app, I use that to decorate a function, hello world, app.run and we're good, right? But in reality, there's all these other things to it, right? There's like the database access and the structure and the factoring and the testing, and there's more to it than initially meets the eye when you create these things.
28:14 Panelists: It's almost like a false sense of security.
28:16 Michael Kennedy: Yes, yeah, that's why I kind of said I wasn't sure how to put it, 'cause it doesn't, I don't necessarily put that forward as a positive, right, like Flask tells itself, it's like, oh, it's just this one file, but like, if you really create an app that's just one file, like a whole website, like you're probably doing it wrong. You know what I mean, like, its selling point is kind of like encouraging you not to do the best thing, so anyway, still it's a great framework, I like it. Alright, I guess that puts me as next here, so the next thing I want to talk about is also Flask, and this one is called Quart, so this puts together a couple of ideas that I find really interesting in Python. You know, a lot of times you hear people tell you that, like, asynchronous programming in Python is super complicated, and it's basically broken, and like, you can't really use it anyway because of the GIL and stuff like this, and I did a lot of thinking about this and a lot of research on this, and actually, I think as of Python 3.5 and beyond, like, that's not really true. It's not really, I don't think it was totally true before and it's definitely not true these days. I think asynchronous programming in Python is really interesting, so I'll talk about that in a second, but also, things like Flask and Django and Pyramid, none of those support async programming, which is odd. I mean, it strikes me as odd. Does it strike you guys as odd?
29:28 Panelists: Yes, a little bit. Yeah, they expect so these days, but it is weird.
29:32 Michael Kennedy: Exactly, right? Like, what web apps basically do is they talk to other systems and, like, coordinate it, and then they wait while that happens, right? Like you make a request, it talks to the database, it waits, maybe it has some other questions for the database after it figures out what user it is. It's going to ask, well, what's in this category that you posted over to me? And it waits for that, and maybe it's going to call an API and wait for that. Maybe it'll use a cache and it'll call Redis and wait for that, and it's going to put those answers together in a template and give it back to you, right? Like it's all about coordinating and waiting, basically is the way I see the web. And yet, that's the perfect scenario for asynchronous programming, like, let you do other stuff while you're waiting on the database, 'cause you're actually not doing any work, you're just waiting. But because of the history of the way these frameworks are built up, they're all based on WSGI. WSGI is like basically single-thread request. There's not a great way to do it, so if you do want to do that, you have to pick one of these many plethora of these brand new frameworks, or you could pick this thing called Quart, which is cool, and I've covered this on Talk Python before, but Quart is basically a API and plugin-level compatible thing with Flask, but it supports asynchronous methods and async and await and all that kind of stuff. So that's pretty cool, so you just basically replace the word flask with quart. If it's a lowercase flask, it's a lowercase quart. If it's an uppercase F in the Flask, it's uppercase Q on the Quart, and like, that's pretty much it, and things like Flask-Login and all that stuff, still work with it, and yet it allows you to write async def home or async def login or async, whatever view you want, you can do async stuff, and then within there, you can await your database call. You can await your Redis key lookup and things like that.
31:15 Panelists: That's nice! Oh, that is cool.
31:16 Michael Kennedy: Yeah, so it's really nice. It's interesting if Flask, sorry, if Quart is going to be its own thing in the long term. I know there was some conversation with David Lord and the maintainer of Quart about maybe, like, somehow working together if they're so similar. Like, could we just make Flask async? And Flask does have async on its roadmap. I know there's a conversation about Django's async roadmap, and things like that, so you know, these other main frameworks are working towards that, but they're already existing. I don't know, it's a challenge, but there's a bunch of really interesting things there, and the other thing I want to throw out while we're on this topic, that was just one of my favorite things, is, there's all these different ways to do asynchronous programming in Python, like we've got threads, we've got processes, we've got asyncio, which uses the async and await keywords but for some reason, threads, you can't use async, you can't await a thread, which is weird, because you should be able to. Things like that, but there's this thing called Unsync, and Unsync lets you, yeah, Unsync lets you put a decorator on all of your asynchronous methods, and it will figure out which type of asynchronous programming or execution is appropriate for it, so sometimes it'll use thread, sometimes it'll use asyncio, sometimes it'll actually create subprocesses, but your program model is all the same. It's just async and await, straight across the board. It's a beautiful library, I just love that thing, and I don't think we talk about it in the course here, on this stuff, but it's definitely worth checking out if you're doing much with async and await. It's definitely cool, so anyway, if you want to play with this conversion from Flask to Quart for async bits, that's days 21 to 24 in the GitHub repo.
32:52 Panelists: Great.
32:52 Michael Kennedy: Yeah, that's a fun one. Speaking of Django and its async roadmap, Bob?
32:57 Panelists: You know, Django, I'm about to ask you, is the async on the roadmap? Anyways, if it's not, it's still my favorite, so a big chunk...
33:06 Michael Kennedy: Yeah, the Django guys are talking about it, they're planning it. It's just not there yet.
33:10 Panelists: And I'm sure they are. Yeah, I'm a big fan, and so a big chunk of the research and the coverage I had in the course was the Django, and Django login registration and Django REST, so also Django is a very big topic, and people find themselves with a steep learning curve. So the best way to go is to build a very simple CRUD app, so create, read, update, delete, and I built a quote app with one model, very simple, but a great opportunity to look into Django's ORM. So if you're not familiar with these frameworks, one of the things they do is get rid of you writing raw SQL, so you can use the Object Relational Mapper or ORM and it makes it very easy to talk to a database, and Django's ORM I really like, I mean, it's very intuitive and the syntax is, makes a lot of sense.
34:11 Michael Kennedy: Yeah, and Django's different in the sense that, like, Julian and I were talking about SQLAlchemy, which we both like, but SQLAlchemy is its own separate thing and you choose to bring it into Flask or Pyramid or something like that, but with Django, it comes with its own, like, more integrated data access layer, right?
34:26 Panelists: Right, that's a great addition, yeah, because yeah, Flask, you have to set it up separately, and usually you go with SQLAlchemy, but Django has all the stuff in the box. It's a complete solution. Also, the templating, I think for Flask you would use Jinja, and Django has its own templating engine, so Django's great if you want to have, yeah, a complete solution out of the box, but then it is opinionated, so some people like it, some people don't, and I do like the decisions they made. What do you like, Django or Flask? What do you think? The never-ending battle between you and me, huh? You're a Django man, fine. Yeah, so another thing I covered was, I mean always, usually if you make a website, you need some sort of registration and login, so I covered a Django-registration plugin, which recently was upgraded to 3.0. Great plugin, if you need to capture emails and do two-way authentication, so you need to send an email, confirmation email, for a user to confirm and register. The only thing confusing was that the templates were missing, so I ended up providing them in a .zip file. Even so, I would really recommend looking into that plugin, which is called Django-registration. And lastly in this context, also covered Django REST. As we mentioned before, same author as API Star, and yeah, it's another great framework that makes it easy, relatively easy to build APIs, but do it on top of your Django models, so I used the same quote app.
36:00 Michael Kennedy: Right, if you've already got your Django models made for your web app, like if you want to expose that as a RESTful endpoint, like, this is a supernatural choice because you only have to put the API, like a thin API layer on top of what you already got, right?
36:12 Panelists: Yeah, you basically make a serialization module, and that's almost all you have to do, so it makes it really easy, and it has also support for permissions, so I think that's done via some middleware, and yeah, and you also plugged in a swagger for documentation, and yeah, that's all pretty easy, relatively few lines of code, so I really liked the Django REST framework. I can highly recommend it.
36:39 Michael Kennedy: That's cool. Did you use Django REST framework in your code challenge platform?
36:43 Panelists: Yeah, good question, I actually not.
36:45 Michael Kennedy: Just you used Django, but not Django REST, right?
36:48 Panelists: Yeah, just Django made that pretty easy to do. We just manually made some API endpoints, yeah.
36:54 Michael Kennedy: Yeah, I end up doing that a lot as well.
36:55 Panelists: We were thinking though, if at some point we need a mobile app or something and we need to make an API for the whole thing, then we'd definitely want to use it.
37:05 Michael Kennedy: Yeah, for sure, I've been down that road.
37:07 Panelists: Yeah, another I just want to highlight was SendGrid for emails, because with the registration plugin, you have to send a verification link, and SendGrid makes it very easy with their API to send emails, so I used that as well in this context.
37:25 Michael Kennedy: Yeah, the days are long gone that you'd just set up your own SMTP server and talk to it locally. Using something like SendGrid makes a lot of sense.
37:32 Panelists: Yeah, and I think I also at some point tried to use Gmail but then it doesn't really like being used as an API or so, a security concern, so it's not, that was not a success.
37:46 Michael Kennedy: The lessons you learn. Why is my email not getting delivered? This portion of Talk Python To Me is brought to you by Linode. Are you looking for hosting that's fast, simple, and incredibly affordable? Well, look past that bookstore and check out Linode at talkpython.fm/linode. That's L-I-N-O-D-E. Plans start at just $5 a month for a dedicated server with a gig of RAM. They have 10 data centers across the globe, so no matter where you are or where your users are, there's a data center for you. Whether you want to run a Python web app, host a private Git server, or just a file server, you'll get native SSDs on all the machines, a newly upgraded 200-gigabit network, 24/7 friendly support, even on holidays, and a seven-day money-back guarantee. Need a little help with your infrastructure? They even offer professional services to help you with architecture, migrations, and more. Do you want a dedicated server for free for the next four months? Just visit talkpython.fm/linode. Julian, what do you got for the next one?
38:44 Panelists: So the next one's actually kind of simple, but again, it was very satisfying, a lot of fun to do. So I did some Heroku deployment on, what was it, day 65 to 68, something like that, and it was really satisfying, so I know with Netlify I was able to deploy a static site and have it all hosted, but this time I wanted to get my Flask app and one of the best parts was the Flask app that I was creating throughout all of the course, it's all running local, you know? It's all local on your desktop, so this was the chance to get it out on the web, to get it running in a dyno on Heroku, and just deployed out there so I could spam it to all my friends and family and say, "Look what I made." And they make it really simple, and this is what I would think, we used Heroku as well for quite a few things on the side, but they just make it too easy, and the best part is, with your Flask app, they fully support Flask, so you can just deploy it. You link the repo that you've created locally to Heroku, and again, when you push the code up to Heroku, it processes and it actually deploys your site onto your dyno, and it's live, so I think the only thing you really need to throw into your code is a little Procfile and a couple of little tags here and there, and that's it, Heroku takes care of the rest, so it's actually really simple. That was part of what I covered in that section of the course, but I also wanted to highlight some of the cool features Heroku has, and again, I'm going to say I know this sounds like a massive plug for Heroku, but it's not, I just really like what they do. They have a lot of plugins that you can use. I think they call them plugins, yeah, whatever.
40:30 Michael Kennedy: We're going to call them plugins.
40:31 Panelists: We're going to call them plugins today. So there were two that I wanted to cover. Bob just mentioned one, SendGrid, 'cause we used SendGrid a lot. So again, you can hear that a lot of the stuff we're discussing here, we use actively, so that's why we wanted to cover it in the course, but this SendGrid plugin, it links the add-on, it links with your SendGrid account and it's the most minimal amount of code to throw into your codebase to then shoot off an email, automatically, and it just works really well, so I really enjoyed that. Teaching that, in fact, I think the hardest part was making that section take a day, because it was so easy to teach. It was, I was struggling for things to say. I probably should've just sang a song or something. But then the other one I wanted to cover off was Heroku Scheduler, however you want to pronounce that word, scheduler, but it's just essentially cron, and it makes your life so much easier. You don't need to go into any backends and worry about, it has a nice web interface, if you're that way inclined, like me. And it's just super helpful, you can automate so many tasks. You can, little parts of your Flask app, whatever you want, with this little scheduler, which really put in the code that you want it to run every time, and it's just, it's really easy, really pleasant to use, really pleasant experience. It's not something you get to say very often.
41:55 Michael Kennedy: Yeah, that's a great way to describe deployment, right? Like normally deployment is something that people got to learn a lot about, it's really tricky and you have stuff that goes down, and you're like, where are the NGINX logs? It says it doesn't work, but I don't know why and I can't figure out where the logs are. You know, like, so this is really nice. I know there's a lot of people like you guys who love platform-as-a-service stuff like Heroku, but there's also a good case to be made if you want a bunch of VMs depending on, like, how you want to configure things and whatnot, like on Linode or something, so yeah. It definitely, if you're inclined towards the platform-as-a-service side, that sounds like a good option for sure.
42:30 Panelists: Yeah, definitely.
42:30 Michael Kennedy: Alright, so the next one I wanted to talk about kind of has to do with, like, deployment and evolving your app over time as well, and that's database migrations. So if you're using SQLAlchemy, Mike Bayer created this thing, who also created SQLAlchemy, created this thing called Alembic, and if you're using Django ORM, there's migrations built into that as well, and the idea is that, regardless of which one you're using, but I'll talk about SQLAlchemy, just to have something concrete, if your SQLAlchemy class structure does not match what your database structure looks like, your app is coming to an abrupt, screeching halt , right?
43:07 Panelists: Oh yeah.
43:07 Michael Kennedy: You're going to get a, you know that, right? You get a 500 error, it's going to say, "Operational error: database schema." like this column doesn't exist, or something terrible like that, and you'll be like, whoa, that sounds kind of bad, but like, if you make a change to your code, like add a column or something and you deploy it, boom, your app is taken offline, right? Like it will no longer talk to the database, it's a problem. So how do you manage that, right? Like some people probably, like, evolve the database manually. That sounds a little scary to me, just 'cause it's, you got to write the SQL transformations to do it right, or like open up the tool and edit it, so what's great with Alembic is, you can just point it at your SQLAlchemy models, point it at your database. There's a little bit of configuration to set it up, but then you can just run a command, auto generate the migration. It'll look at your classes and it'll look at the database and go, well, this column was dropped, this one was added, and this index was updated in this way, and then it just lets you run a command line operation that will automatically transform the database, like that. So one developer makes a change, the other developer checks it out. Obviously that change needs to accompany a migration, so then they can just run the migrations as they get the new version of the app. You push it to QA, or staging, you run the migration, and then as part of your deployment, you can just have it always run a migration and like, if there's no operations, it just goes, you know, if it's up-to-date, it just goes, eh, nothing happens, so we'll just keep going. So it's a really nice way to make sure that your database constantly stays in sync with your ORM models.
44:37 Panelists: Oh, that is really cool.
44:39 Michael Kennedy: Yeah, and when I first looked at him, like, whoa, this looks complicated. Why is this so hard to set up? Maybe it is, I don't know, there's a few little hitches that, like, caused some challenges about, like, how do you import the models correctly from the context of executing within Alembic? And stuff like that, but once you get it all figured out, it's not too bad. We covered that in days 61 to 64 so you can go check out the little sample code there. Another alternative is to just not use a relational database like, you know, all my stuff, like, in production runs on MongoDB, and MongoDB doesn't have this problem, but it also has a, basically an ORM, something called an ODM for them. MongoEngine's what I use, and it's basically like SQLAlchemy. It's more like Django ORM, actually, but same idea. But MongoDB doesn't have this problem in general, so you don't have migrations, really, there. You can sidestep it that way, but a lot of people want to use that relational database, and if you do migrations, it's a great way to solve the problem.
45:32 Panelists: Ah, that's really cool, and I can see it, so I'm just on the documentation, and it's actually alembic.sqlalchemy.org, so ah, very cool.
45:39 Michael Kennedy: Yeah, yeah, same organization, same people behind it. Bob, do you guys use migrations, the Django migrations for what you're doing?
45:46 Panelists: Yeah, very simple, make migrations, that makes a file, you commit to version control, and then just migrate and then it syncs it to the database. Works great.
45:55 Michael Kennedy: Yeah, sounds real similar to the auto generate feature there, it's cool. Alright, Bob, what's your last one?
46:00 Panelists: Slack bot. That was one of the most fun chapters I worked on, and well, we all...
46:07 Michael Kennedy: Slack is used for all sorts of things, right? Like people chat with it, but it's also got so many integrations, right? Like, for example, if I have an error on my website, I have a Slack channel that Rollbar will detect the error on my website and then post a message to me on Slack that there was an error on the website, right? And that's like a Slack bot sort of, sort of, I guess. But yeah, there's all sorts of uses for it.
46:31 Panelists: Yeah, when people do a pull request or a code challenge repo, we get a notification in our pull request channel. Also, if you think about, like, the general Slack bot, it's super useful, like you can set reminders and stuff. So, and yeah, bots overall are becoming more popular, so it's a nice skill to have, I think. And so yeah, for this chapter, I looked into Slack API, and we made a random book slash command, so if you type /book in our slack, it reaches out to some books API we have for another app we run, and it posts back some random title. You can also do /book and then user, and you get the last book for the user, so that's fun.
47:15 Michael Kennedy: That's cool.
47:15 Panelists: And what's nice about these apps is that there's a lot of moving pieces, so it's not only the Slack API, but you have to call another API to get your data, so in this case, that was Google Books API, which I think has been around forever. And we had to make a little Flask app too, for the slash command to actually grab data, so it was, and retrieving data from an API and making an API, and then before rolling it out, you want to test your local server on the internet, and for that, I used Ngrok, right? Which is a pretty important tool if you want to, how would I say that? Link your localhost to your internet address, or make your localhost available on the internet, so to say, so you get this kind of temporary IP and anybody connecting to that IP is actually talking to your local web server, which is cool. Useful.
48:10 Michael Kennedy: I'm so glad you covered Ngrok here, you threw this in here, because Ngrok is incredible. So like, if I type Ngrok and I give it a port, it'll say, I could map like port 5000 if I was doing Flask locally over to, like, 88532FF5.ngrok.io and it actually has an HTTP and HTTPS version. So at first I thought, "Oh, this is kind of interesting." But I've been using this for some incredible stuff, so for example, when I was developing my mobile apps, I wanted my phone, where I had the debugger attached to my phone, to be able to talk to the web services that I was building locally, with the debugger in PyCharm attached to the web service, right? And of course I could drop my firewall and, like, set this up, and like all. But when you just run an Ngrok command, you don't need to change firewall settings on either end. It just, it automatically goes through the internet, through HTTPS and then right back in it, and you can step through one and step through the other. Another one was, I was doing some integration with Gumroad for the purchasing of the PyCharm book that Matt Harrison and I wrote, and we had to do some, there's one option you can buy where you also get a course, and so it had to, like, create an account and, like, put you in the course, and like, it wasn't working, so I put the URL, it was supposed to work, and it wasn't working. Like, why is this not working? And like, well, why don't I just put the Ngrok URL in there and I was literally stepping through in the debugger, Gumroad's callback into, like, my local machine, into PyCharm, into the APIs I was creating. It is so cool, this Ngrok thing. It's definitely underestimated.
49:44 Panelists: That's really cool, if you cannot debug something on localhost, you're lost, right?
49:49 Michael Kennedy: Right, I mean, here you want to have Slack, in your example, you want to have Slack send a message over, and it goes to your HTTP endpoint to do something and then it gets a response, like, you can't debug Slack, right? That's their infrastructure out on the internet, but you can actually, like, deeply run and iterate on, like, this, your local version using Ngrok. It's awesome. Yeah, okay, I'm excited by that, I've got to stop, but yeah, it definitely, you found it valuable as well, huh?
50:15 Panelists: Now that is cool. So the whole Ngrok thing is amazing. I didn't even know you used, that, Bob, in that part of the course. Did you create any other Slack bots, other than just the books one? There's one we actually use quite a bit which is called Karma bot, on our Slack, and if you want to give people credit for contributing to the community, helping each other out, you just use @handle++, and Karma bot keeps a local database of the user's karma, so the user then gets extra points, and it's funny how people sometimes react to that gamification. Definitely boost community participation, and yeah, it's all run by a slack bot. It's kind of funny, it's a bot, but people kind of treat it as a human at some point, right? It's, that's really, it was one of the, yeah, most satisfying projects. It's not in the course, but it's on GitHub. You can, we will link to it. It's another example of a bit more elaborated Slack API project.
51:18 Michael Kennedy: For sure. Julian, you want to wrap up your section with Chuck Norris?
51:24 Panelists: Oh yeah I do. So while we're having a bit of fun, this was the most fun I had in the course from a actual fun standpoint. So what I wanted to do was, I guess I'll give you the quick background. It was Flask, but what I wanted to do was give the users a sort of real life use case, so when you make a Flask app, especially when you're learning Flask from the start, one of the easiest things to do is to try and talk to an external API, so not make a Flask API, but just go out to an API, pull some data, and then present it on your Flask app webpage, so that involves, you know, setting some GETs, getting some POSTs potentially, and then presenting that data, so doing something with it to present it on your page. And it's all little simple bits, but combining it all together is, I remember when I first learnt this, was a bit complex. So I wanted to demonstrate that in a few days, and probably the fun part here was choosing some interesting APIs to play with, and the first one I chose was the Pokemon API. Actually, no, the first one I chose was Chuck Norris, so there were two, Pokemon and Chuck Norris. And the Pokemon one was really nice because what I was getting people to do was enter in a color on the webpage that you create, and that color then shoots off to the Pokemon API and returns, you get returned a list of all Pokemon with that color, and then the course shows you how to list that data on your page or print it onto your page, so really simple stuff, but really actionable and you get that instant feedback of hey, look what I just made, this is cool! And because you get to see the Pokemon API, you get to see all the cool little things that you can pull, so I've seen people on Twitter who then expanded on this, like spread into another day or two and got all sorts of other information from the API, and it was just really cool to see, but what was actually quite hilarious was the Chuck Norris API, so I couldn't believe it existed, first and foremost, which just killed me for a day. But the API just returns a Chuck Norris joke, and the reason this is so funny is 'cause obviously, the jokes are hilarious and lame and terrible, whatever, but I didn't realize they were, some could kind of be inappropriate, and I didn't realize until after I'd recorded everything, and put all this time into the course, and then I'd got to the end, I'm like, "Oh, that joke wasn't fantastic." So I think there's a disclaimer in the course now saying just please ignore any jokes. Just, this is more for the API, not so much for the joke!
54:05 Michael Kennedy: Yeah, it's funny, though.
54:06 Panelists: This was a great chapter to do, as it's really actionable and a lot of fun.
54:10 Michael Kennedy: Yeah, and consuming APIs, especially from Python where we have requests and other things like that, AIOHTTP client, for example, it's so accessible. I think before people try it, if they haven't, it sounds like, oh, this integration might be hard, like, usually it's not.
54:23 Panelists: Yeah. It's actually quite simple, and probably the best part is, when you're dealing with that sort of thing, generally you're, a lot of use cases will have you use forms, and doing the very simplistic Flask forms that are built in. They're nice, and then it's a nice gateway into the WTForms that Flask has as well, so yeah, it's good, a lot of good stuff in that one.
57:13 Michael Kennedy: Absolutely. And then I want to throw in two other quick things. There's something called the Quasar framework, which builds on top of Vue.js, and it has a whole, just a ton of these, like, widgets and controls that you can just, in Vue.js, just include, and you get all sorts of great, like, higher level UI elements that are just prebuilt and plug right into Vue, which is pretty cool. And then the other thing is something called Vue.py, Vue.py, and so this is a Python layer, so you can write Python code in the front end of your browser, that actually is Vue.js bindings.
57:56 Panelists: Wow, that's really cool.
59:05 Panelists: Oh yeah. A lot of technology here. There's like six months worth of content.
59:08 Michael Kennedy: Yeah, I would say, and you know, this is only 12 topics that we covered. Maybe 13 or 14 if you guys kind of doubled up. And in the course, we actually covered 28 different technology topics in these different segments, which is pretty insane. So let me just ask you both this. What is, like, after going through all these topics and researching them and studying them, what's your kind of takeaway from this whole exploring the web, broadly, experience?
01:00:31 Michael Kennedy: And vice versa.
01:00:33 Panelists: Yeah, it's stuff that wasn't there when we finished, exactly, there was, what's that library that Kenneth Reitz makes? The, not Request, the other one that he just came up with?
01:00:44 Michael Kennedy: Responder?
01:00:44 Panelists: Responder, like that came out while we were making this course, I'm pretty sure, so that sort of thing is just going to keep happening, so this is why I'm Flask and Bob's Django. Bob, what are your thoughts on this?
01:00:57 Michael Kennedy: Bob, what are your thoughts on this?
01:02:48 Panelists: Yeah, I think that's a great point, especially when people in the community ask, like, "I want to make a blog and I need to learn Django." That's fine, but you can actually use static sites and GitHub, right? And so you don't actually have to learn Django per se. I would definitely really look at what you're building and then try to choose the right tool, 'cause it's easy to overdo it.
01:03:10 Michael Kennedy: Right, maybe it's Netlify and Pelican or something, yeah.
01:03:12 Panelists: Yeah, exactly. Yeah, exactly.
01:03:13 Michael Kennedy: Yeah, cool, Alright, well, that was a lot of fun to talk about those. Let me ask you both the two questions before we get out of here, though. If you're going to write some Python code, Bob, what editor do you use?
01:03:23 Panelists: If you ask, Vim.
01:03:26 Michael Kennedy: Vim?
01:03:26 Panelists: It's still Vim, yeah. It's just habit.
01:03:29 Michael Kennedy: Yeah, yeah, cool, Julian?
01:03:32 Panelists: Well, I have, I was going to say Vim, because that's just what I use, but look, if I got to choose some other one, I'd actually stick with Notepad++. Actually really enjoy that.
01:03:41 Michael Kennedy: Yeah, that's cool. And I guess I'll throw mine in there as well, PyCharm. Not a huge surprise to people. Now, notable PyPI package, I mean, like, we explored probably over 100 different PyPI packages like, for the various parts, you know, of this, and the dependencies and whatnot, of all the stuff that we interacted with. So there's got to be some that you're like, "I didn't even know about this and it's really cool." Bob, you want to go first?
01:04:06 Panelists: I guess the question for this episode is notable PyPI packages that's not mentioned before. So I cannot do Django.
01:04:14 Michael Kennedy: I mean, it could be one that you mentioned before, if you want to put it that way.
01:04:16 Panelists: Yeah, Django would be not a surprise. Now, for example, for web scraping, right? Two weeks ago, we had a live code challenge and we were scraping some PyCon Spain website data and Beautiful Soup, right? It's a great package that makes it easy to traverse the DOM, and yeah, I'm still happy with that package. It's very easy to use and gets the job done.
01:04:41 Michael Kennedy: Yeah, and if you want to keep it short, you could pip install bs4.
01:04:43 Panelists: Yeah, even shorter.
01:04:47 Michael Kennedy: Exactly, Julian?
01:04:47 Panelists: It's actually got nothing to do with web stuff because Flask would be a no-brainer for me here. But one of the ones that I've used time and time again, especially at work, has actually been Pexpect. Have you guys heard of that?
01:04:59 Michael Kennedy: I haven't heard of that, no.
01:05:00 Panelists: It's a little automation library or module. It's, pretty much the main function is to automate interactive processes that provide predictable output, so think Telnet, SSH, Git, and everything like that, so you can automate what's going to happen because you know what your response is going to be when you SSH to a server. You know that the password prompt's going to come up, so you can code in, okay, when you get this exact prompt, now what do you do? And it's really cool. There's a whole, obviously, bunch of documentation on it, but that's the library module I really like.
01:05:35 Michael Kennedy: Oh yeah, that's really cool. It does SSH, FTP, Telnet, all sorts of cool stuff. I love it, see, this is why I ask this question, because I learn something every time. The one I'm going to throw out there is Unsync. So I talked about Unsync before. It basically unifies all the different ways of doing asynchronous programming in Python and puts the async and await interface on it, and it's beautiful because it's only 126 lines in a single Python file, and yet, it like, basically fixes most of Python's async APIs. Like unbelievable that happens in such a small amount of effort, so really love that one, and if you're doing async programming, check that out. Yep, Alright, guys, thank you for being here, Julian, Bob.
01:06:20 Panelists: Thank you. It's good to be back, Mike. Thank you for having us again, man.
01:06:23 Michael Kennedy: Yeah yeah, thanks for writing this course with me. We all learned a lot, and it was fun to share it with everyone a little bit.
01:06:28 Panelists: Cheers, man, we'll see you next time. Cheers.
01:06:30 Michael Kennedy: Yep, bye. This has been another episode of Talk Python To Me. Our guests on this episode were Bob Belderbos and Julian Sequira, and it's been brought to you by Ting and Linode. Ting is the fast mobile network custom-built for technical folks. Use their savings calculator to see exactly what you'd pay. Visit python.ting.com to get a $25 credit and get started without a contract. Linode is your go-to hosting for whatever you're building with Python. Get four months free at talkpython.fm/linode. That's L-I-N-O-D-E. Want to level up your Python? If you're just getting started, try my Python Jumpstart By Building 10 Apps course, or if you're looking for something more advanced, check out our new Async course that digs into all the different types of async programming you can do in Python. And of course, if you're interested in more than one of these, be sure to check out our Everything Bundle. It's like a subscription that never expires. Be sure to subscribe to the show. Open your favorite pod catcher and search for Python. We should be right at the top. You can also find the iTunes feed at /itunes, the Google Play feed at /play, and the direct RSS feed at /rss on talkpython.fm. This is your host, Michael Kennedy. Thanks so much for listening, I really appreciate it. Now get out there and write some Python code.