#215: The software powering Talk Python courses and podcast Transcript

00:00 Michael Kennedy: Have you ever wondered about the software stack powering Talk Python, the training websites, the mobile apps, the video and audio delivery, and more? While at first glance, it might seem pretty simple, there's quite a bit going on actually. We have our own custom search engines. We deliver about 15 to 20 terabytes of traffic per month. Our course video streams from eight locations throughout the world. Our database server is sending about 12 megabits of traffic sustained all day, all night, with no media in the mix. And all of this stuff is powered with Python. So this week it's a bit of a role swap. Dan Bader from Real Python is here to interview me about the Talk Python tech stack. But we do get a chance to compare my tech stack with Real Python's. A site which is becoming quite an important one for developers and the Python world. This is Talk Python to Me. Episode 215, recorded on scene at PyCon 2019 in Cleveland on May 6, 2019. Welcome to Talk Python To Me. A weekly podcast on Python. The language, the libraries, the ecosystem, and the personalities. This is your host Michael Kennedy. Follow me on Twitter where I'm @mkennedy. Keep up with the show and listen to past episodes at talkpython.fm and follow the show on Twitter via @talkpython. This episode is sponsored by Linode and Backlog. Please check out what they're offering during their segments. It really helps support the show. All right, hey folks. We got a little bit of a different one this week. So I'm excited to be here with Dan Bader from realpython.com.

01:37 Dan Bader: Hey, how's it going everybody?

01:39 Michael Kennedy: It's good to see you, Dan.

01:40 Dan Bader: Likewise, this is so unreal because the last couple of times when I was on your show we were not sitting right next to each other at a hotel in Cleveland. We were doing this over Skype and so it's, it's kinda cool actually.

01:52 Michael Kennedy: It's super cool, yeah. And greetings everyone from the tail end of PyCon. Not quite from the conference floor anymore since they cleaned that up. So now we're at the hotel floor.

02:00 Dan Bader: Yeah, and it was a really sweet conference. I mean, I don't know, I'm sure you had a great fantastic time, too. I know you were always swarmed and surrounded by people and fans that want to take photos and chat with you and really cool.

02:10 Michael Kennedy: It's one of my favorite times of the year, honestly. It's exhausting but it's great.

02:14 Dan Bader: Yeah, no, it was an awesome conference. I mean, thanks to everybody who organized it, put it on. It was really, really amazing. I feel like it's just getting better and better every year and you know, there's more interest in Python and it's just so cool to hang out with the community.

02:27 Michael Kennedy: Yeah, absolutely. You meet all these people that you collaborate with on the Internet or other things and you would never run in to them because they live in Australia or Vancouver or wherever. But here you run in to all of them. It's amazing.

02:41 Dan Bader: It's really, really cool. And one of the things I like the most is when you've been working with somebody or you know them online but you never actually met them or been on a video call and then you run into them and you're like, oh my God, I had no idea. Oh this is you, like this is so cool. And it's, yeah, just been a ton of fun.

02:55 Michael Kennedy: Yeah, another one of takeaways I guess is everyone's just a person, right? Like all these people working on these amazing projects, they're really open to meet you and share ideas and like, if you just go and say hi, chances are that you'll get to learn a lot about whatever you're interested in.

03:11 Dan Bader: Totally yeah, I think everybody's is really, really welcoming and I mean that's the whole point. I'm sure it's the same thing for you. Like that's why I go to these conferences, to hang out and meet new people. I'm not going there to hide behind my laptop or just being like business mode but it's just fun to walk around and chat and grab a cup of coffee or beer with somebody.

03:29 Michael Kennedy: Yeah, absolutely. I consider this my geek holiday. So I did say, the opening that we have something a little bit different. So we're gonna sort of reverse the roles and Dan is going to interview me somewhat. But we're also, I'm also gonna have a lot of questions for you, Dan.

00:00 Dan Bader: Sounds good. I'm looking forward to this.

00:00 Michael Kennedy: So a lot of folks have asked various questions at different angles like, how have you built your Python stack? Or why are you serving your audio this way? Or how did you build your new mobile apps that you released? Just all these different questions about the tech involved in the podcast and the training and all the derivative stuff from that. So, I decided Dan you would be the best person to sort of represent the audience on that side of the conversation because one, you're super knowledgeable, we're good friends. But also, because you are also running a pretty high end website. Real Python is one of the biggest Python sites out there these days, isn't it?

00:00 Dan Bader: I mean, it's kinda hard to

00:00 Michael Kennedy: Define.

00:00 Dan Bader: Measure that in terms of, I don't know, like impact and everything but I can share some stats that we just hit 1.3 million unique users per month. So that's like several million page views a month just from realpython.com alone. Which to me is just an insane number so yeah, I think, first of all thanks and I think you were up there somewhere.

00:00 Michael Kennedy: It's awesome, yeah. It's definitely really, really growing since you've taken over stewardship of that resource in the last year, year or two and it's awesome to see. So we're gonna talk about realpython.com and what you're up to as well.

00:00 Dan Bader: Cool.

00:00 Michael Kennedy: All right. I guess one of the things I did wanna start, like set the stage with before we get into this conversation is we'll talk about some part and people might go, well why didn't you build that serverless? Or why isn't that a microservice? Or couldn't you have hosted this here in that way? And probably the answer is yes, yes, and yes. But I'm not doing any of those because there's just a lot of ways to make this work, right? There's a lot of choices out there and I think maybe that's even why people are interested, right? Like if it was you go to HP and you ask for a rack of servers and you set up this software on it and you put it on the Internet and that was the answer like, nobody would ask. Say how many servers you have? I have three, one use server. Okay, interesting, right? But because there are all these choices I think it's interesting but I also just wanna set the stage like, the choice that you might have made may be different than the choice I'm making or the choice that Dan is making but you know, just be aware. Like, they may all work. Maybe better, maybe not, I don't know. But yeah.

00:00 Dan Bader: We're not judging and so please give us the same, the same favor in return and I think there's also, you know, with all of this stuff there's a huge like hindsight effect here, you know, happening here where sure, we could have built this probably on a number of different platforms and hosting services and tech stacks. But a big reason, I mean for me personally, why I'm doing all this stuff and why I love being involved with the Python community and being able to teach in the Python community is that I get to play with these things and build software in the way that I like to and that I enjoy. So not everything is like a pure, what is like the most effective or efficient way to do this.

00:00 Michael Kennedy: That's so interesting.

00:00 Dan Bader: Yeah, this so personal.

00:00 Michael Kennedy: And you just made me realize that I did the same thing but unconsciously. So like, I knew nothing about deploying on Linux so I feel like I originally tried going down the platform as a service route and like, things that would hold my hand as much as possible and then I'm like, no I feel like I need to know this. So I'm gonna force myself to use a infrastructure as a service thing and learn Nginx and uWSGI and Linux and all these deployment things and how they work together so that if, I guess so I wasn't be a weak spot for me or something. And I do feel like I took that on as a learning challenge as much as a good way to deploy. I think it also works really well but also to learn right? Like, if you get a chance...

00:00 Dan Bader: Totally.

00:00 Michael Kennedy: It's worth pointing out that you and I both run small businesses. We are not at a huge enterprise with somebody that has made rules about what tech we can use. We can use anything. Long as like, we're willing to live with the consequences. And so, at least personally, I use that as an opportunity to say, I wanna use this tech so I can learn it as well as I think it might be a good idea.

00:00 Dan Bader: Yeah, I think that's a big part of it and then, I mean, also, wouldn't it be a shame if the biggest Python podcast was hosted on a bash script somewhere. Like, it's gotta be built on Python.

00:00 Michael Kennedy: It's a Java app.

00:00 Dan Bader: Yeah, I guess nobody's questioning that decision, right. To build the app in Python or similar text type.

57:34 Michael Kennedy: Exactly, I kind of think like, set the stage we're running on Python both of us. Let's start there actually. That's an interesting case because when I started the podcast I didn't want to start a Python website. I wanted to start a podcast and yet, I needed a website to serve the RSS feed and the mp3s so that I could have a podcast. Like it was, table stakes. And right now there are pretty decent options for hosting a podcast if you're willing to pay a little bit. Like fireside.fm is not bad. Brian Okken uses that for Test and Code, for example. When I started Talk Python there was a few options and they were bad options. Like they were not so good. So I looked at that and I said well, am I serious about this? Do I want a site that looks good and people will respond to or is going to look like the worse WordPress theme you've ever seen. But technically it will be a RSS feed and who cares. You don't go to the website, right?

00:00 Dan Bader: Yeah, yeah. I totally get that where, and a lot of this stuff has changed also, I think. You know, because, when did you start Talk Python training?

00:00 Michael Kennedy: I started...

00:00 Dan Bader: Or the podcast.

00:00 Michael Kennedy: The podcast started in April 2015. The training site started in 2016 in February 15th. I know 'cause that was my day I was free from full-time employment.

00:00 Dan Bader: And so, you know, with all of this stuff I think, especially in the podcasting space, you know, a lot has changed. I mean, now it's sort of a legit thing like hosting a radio show. My parents listen to podcasts and not my grandparents. But like people know about, that this is a thing. And when I think back, even like three, four years ago it wasn't at that level. I mean just looking at the apps. I really like the Overcast app for example. I don't think that was out back then. And it was all like, basically Apple iTunes or something and it's really changed and grown since then. So, now there's probably all kinds of different options but there's a lot of personal investment and all this custom software that you've built, I mean that we've both built for our businesses. And there's sort of also a barrier to move to something else. I feel like we sound super defensive talking about this stuff.

00:00 Michael Kennedy: Well, I think it's really interesting to consider like, should I have just said no, I'm not gonna build, say for example talkpython.fm and Python and build it myself and do the web design myself and all of that and figure out how a RSS feeds and, work and how am I gonna get the video, the audio delivered without like breaking the bank and bandwidth and all of that kind of stuff. Like I could have just put my credit card in at Podbean. Got a crappy site for, you know, $5. I don't know what the price is or whatever. And just gone with that. But, you know, I felt like there's going to be a point where I wanna do something that it doesn't do. And do I wanna be walled off over there? And also, it was a little bit of a learning experience like you had said. Like, this will be, should be a cool thing to run to understand how it all fits together. So I just understand it better. But, you know, I would say like three months in I never started the podcast thinking I would get sponsors. I thought that was a crazy idea. I was just be something that I could have a community of people I interact with 'cause I didn't have that. But about three months in, one company approached me and said, "Hey we would love to sponsor your show. It looks so nice and polished and professional compared to all the other stuff out there." And it's because all the other stuff was hosted in places that were like super cookie cutter and not so great. And I'm like, it's started to feel like it paid off at that point, like, oh yeah. And I can feature them in this way that wouldn't be possible if it were there. It slowed down the start but it seems like it really paid off in the long term.

00:00 Dan Bader: And that makes a lot of sense and I just wanted to add to that you know, also looking at my experience kind of building a custom text stack. There's a huge element of survivorship bias in all of this where if you can pull it off and you can, you're successful with a custom solution, it's gonna be awesome. It's gonna make you unique and you're gonna be able to build out all this unique features but for every like 20 people who are trying to start a podcast of their own, there's gonna be a lot of people who never make it to the actual recording because they're spending five month building...

00:00 Michael Kennedy: Absolutely.

00:00 Dan Bader: Their own infrastructure. And I've been there where you know, not with the podcast but with other stuff.

00:00 Michael Kennedy: That's a good thing to keep in mind for everything, right? Like, getting started is the most important thing in this journey for whatever you're doing. I guess just to put a bow on that like, when I decided I'm gonna try to build a site and see if I can make it work before I decided to host something else, host somewhere else, I started out on a Friday. I said, all right. I'm done with work. I really wanna create this thing. I'm gonna just go all in on it this weekend and by Monday morning, it was like, live and beautiful and ready. And so it was like, I'm willing to try a couple of days to see if doing it myself is crazy or not. And then I would have maybe gone and yeah, paid a couple bucks to Podbean or whatever. I wouldn't have waited three months before I even tried the podcasting inside of the thing. It was like, this is exciting, let me try it. And if it works, I'll go with it. Otherwise I'll pull the ripcord and do something else.

00:00 Dan Bader: Yeah, like do a little experiment and then see how far it can get.

00:00 Michael Kennedy: Yeah, exactly. So, maybe, I don't know. Maybe it's worth talking a little bit about the just quick flyover the tech stack. What do you think?

00:00 Dan Bader: Yeah, I think it'll be interesting. How do you wanna slice and dice this? Maybe we should talk about like, first of all what are the main services that you run for your podcast. You've got the training site and all these other things.

00:00 Michael Kennedy: Right, because they all have different goals and constraints and whatever. For example, the training site demands low latency video delivery where the podcast site it doesn't matter at all if that's delivered a little slower because it auto refreshes on your app and you just open it and it's there. Yeah, so. Let's see. I have talkpython.fm which is the website and let's just group the audio delivery in for that one. I have pythonbytes.fm that I do with Brian and it's auto delivery. And I've got the training site, the Talk Python site and the Python Bytes site have custom search engines that I have as rest endpoints. So search.talkpython.fm. for example. You can consume that and search that if you want. And internally, the site uses that. So you can search for stuff and whatnot. And same for Python Bytes. So there's kind of that side of things. And the audio delivery there. And then there's the training site. And the training site's actually a lot more complicated. I would say on the infrastructure side 'cause it's got to deliver the videos, it's gotta do E-commerce, it has a bunch of accounts. But also it has to have the video delivery, low latency anywhere in the world. It has the apps, right now it has the apps. The iOS and Android apps around it and so on. So, yeah, that's probably the landscape. Also...

00:00 Dan Bader: Your mobile apps.

00:00 Michael Kennedy: Yeah, inbound, outbound, email. We've got a bunch of services there. Like the video stuff. So if, when we record the videos we don't release what we record. We have to transform it in to certain formats that work well on like iOS and are minimized and they're video sized based relative to say the resolution. And there's like, all that processing pipeline is pretty interesting as well.

00:00 Dan Bader: Yeah, this is super cool. I mean, it's kinda mind blowing to go over this and sit down and really tease it apart. Like what are, even all the services that you run because it's so much stuff, right. People would...

00:00 Michael Kennedy: Isn't it just a podcast?

00:00 Dan Bader: I thought this myself. Yeah, you know, it's a podcast. So what do you need, you know, a laptop to record this. We're sitting here, it's kind of a nice and minimal set up but there's a lot of infrastructure that is powering all this stuff so that your listeners get to enjoy this on their commute and in their cars.

00:00 Michael Kennedy: Yeah, I mean just to give people a sense, like a while ago I sort of summed up all the data usage, bandwidth usage of my various servers and whatnot, between the database and then the audio, not even counting the videos which are their own special thing, and I think it was running at like 17 terabytes of traffic.

00:00 Dan Bader: Wow, that's a lot of Python.

00:00 Michael Kennedy: That's a lot of, that's a lot of Python so. Yeah, it's pretty cool. It was actually pretty awesome to see.

00:00 Dan Bader: Amazing okay, so this makes this even harder to kind of slice and dice this and dive into the most interesting bits. I know one of the things that you and I have had some conversations about is the whole like, monolith versus microservices and most recently, also the serverless approach. And I think it would be interesting to talk about that a little bit. Because I feel like now there's a return to the monolith in the industry or people who got burned by this other...

00:00 Michael Kennedy: Yeah, the pendulum has swung a little backwards, right? So I felt a little uncool, to be honest. A couple years ago.

00:00 Dan Bader: Same here.

00:00 Michael Kennedy: 'Cause people were like, "Dude how are your microservices working?" I'm like well, there's three services. There's say, talkpython.fm and then it also has like the search service and then it has a audio delivery bit that like a little bit separate. But that's not really microservices so much, right? Microservices are way more small grained certainly, and Talk Python Trainings like definitely a monolith. But I tried to have a really pragmatic view of these things. Like, I'm not Google. I'm not Facebook. I'm not LinkedIn as Brian likes to say, right? Why do I need to trade that off? And I guess in this context, I'd love to hear your thoughts on this but when I think of like the, should it be a microservice or should it be more monolith? It feels to me like what I'm doing is taking the complexity and moving it from code complexity in monolith to operational complexity in a bunch of services. I'm better suited naturally with my experience to handle code complexity than devops complexity. If I was maybe really good at devops complexity and not so good at code, maybe I would have chosen microservices but that wasn't where my capabilities were. So I'm like, I'm gonna play to my strengths here and just stick with this.

00:00 Dan Bader: Which I think that is the best solution if you're running a small business and in our case we're both building all of the software ourselves. And we're basically the CEO and CTO and main...

00:00 Michael Kennedy: Chief accountant.

00:00 Dan Bader: We have the whole thing.

00:00 Michael Kennedy: And all this stuff.

00:00 Dan Bader: Yeah, all this stuff in one person. And so, I think especially then to go down a route of adding a lot of complexity like operational complexity early on is not a good place to be in. What I found really inspiring is Basecamp as a company. It used to be 37Signals.

00:00 Michael Kennedy: I love their, I used to read like getting started, their book, religiously.

00:00 Dan Bader: Yeah, and it's really good books and I think also really, just a really, really inspiring company and really good, or like interesting from a tech stack perspective too because their totally going down the monolith route. I mean their CTO is basically the inventor or creator of the Ruby on Rails framework which is pretty much like Django in the Python world and they are purposefully building software using monolithic approach so that they can be effective with a small team. Which is ultimately, if you're running a one person or two person business, a super important, like I mean, you want that.

00:00 Michael Kennedy: That 20% efficiency or whatever is massive when you're small. When you're a huge company, huge companies are massively inefficient. Like from my experience so. That's pretty interesting. I'm definitely a fan of just having things working simply and knowing that you can change it, right? We could break it in to microservices if we need to but I feel like just serverless is just like that but plus more. Like it's more pieces I got to manage and so until I need it, I feel like I'm gonna stick with what I'm doing.

00:00 Dan Bader: To me it's interesting also that you know, your search service is a separate thing. So your, it's almost a little bit of a hybrid model there for you.

00:00 Michael Kennedy: Yeah a little bit.

00:00 Dan Bader: Where and I think the Python Bytes, that's a separate

00:00 Michael Kennedy: Separate app, yeah.

00:00 Dan Bader: Separate app right? And it's similar for me where realpython.com is like a monolithic web app but then we have this feedback form system that's separate or at the PyCoder's Weekly newsletter that I run is also a separate application. And I think, it makes sense to kind of slice things up this way because then you can treat those systems in isolation and I guess the question is just when, I think it's a spectrum, of like monolith to microservice to let's say serverless. Maybe it was like really, really tiny microservices basically. And so, I mean we're both not running pure monolith, right? In that sense where if it were a pure monolith like everything you have like the podcast and all the APIs, the search...

00:00 Michael Kennedy: It would be one giant thing. Yeah, I have eight different servers that are participating in like all this stuff we described at the beginning.

00:00 Dan Bader: So maybe I don't know.

00:00 Michael Kennedy: Plus more honestly.

00:00 Dan Bader: When it is a microservices architecture versus when is it a monolithic architecture? Because I mean, you basically have a search microservice in that sense, right? So if I've always been struggling with those terms a little bit.

00:00 Michael Kennedy: Yeah, I think there's two driving forces. One would be technical and the other would be sort of team personal based. Like is there a technical reason to have those two things separate? Do you want to consume them separately? Should they be versioned separately? Should they live on separate servers? Things like that, that's maybe a technical thing that'll put you into this mini microservice. Microservice lite. I don't know, whatever you call it, right?

00:00 Dan Bader: We're gonna term.

00:00 Michael Kennedy: And then the other one is do you have people who are not experienced with the whole thing but maybe they could work on the search service? If I had somebody who was like a junior developer who came on and I was gonna have them work on the search service for a month. Like it's not really worth it and it's not that big, but imagine it were, right? Like it might make sense to have that separate. They can deploy it and live with it and if they take down the search service it's not the end of the world. Do you know what I mean? But if they take down the main site, like all of a sudden it's a problem potentially. And so I feel like certainly microservices are easier for different teams to work on different bits or different people to work on simpler bits but like you said, we're mostly doing the software development ourself. And so, that's absolutely not a pressure that is applied to our stuff.

01:02:30 Dan Bader: And I think that's a good point where you want the ability for one of those services to go down, like if it's an experimental new thing that you built let's say like the search and you're playing with the, working on the search feature trying to make it better in some way, you wanna gracefully degrade the experience for everybody who's using the website. Maybe search doesn't work but everything else would still work. I did that, for example, when I built like an ad serving, like an ethical ads platform for realpython.com and it's now also used in PyCoder's Weekly and some other sites. And there was the same thing where it was like, you know, I don't really know if I'm gonna be running this thing in a couple of months. It's more of an experiment. And instead of kind of polluting my monolithic code base with that new thing, I'd rather have it separate and so I could also scale it independently. So I'm wondering if that also factors into your decisions like maybe search needs to be scaled differently or one of these sites is gonna grow faster than the other ones.

00:00 Michael Kennedy: Yeah, absolutely. Like if you need to scale one part a lot it's easy to just scale that tier or put a load balancer in front of that. I mean, also, it's running under different core domains and different SSL certificates and stuff so maybe that makes it a little easier to have it separate like search.talkpython.fm not talkpython.fm/search. Although that's kinda may be minor. I could change that around. But yeah, it's certainly a consideration. This portion of Talk Python to Me is brought to you by Linode. Are you looking for hosting that's fast, simple, and incredibly affordable? Well look past that bookstore and check out Linode at talkpython.fm/linode that's L-I-N-O-D-E. Plans start at just $5 a month for a dedicated server with a gig of RAM. They have 10 data centers across the globe so no matter where you are or where your users are there's a data center for you. Whether you want to run a Python web app, host a private Git server or just a file server, you'll get native SSDs on all the machines, a newly upgraded 200 gigabit network, 24/7 friendly support even on holidays and a seven day money back guarantee. Need a little help with your infrastructure? They even offer professional services to help you with architecture, migrations, and more. Do you want a dedicated server for free for the next four months? Just visit talkpython.fm/linode.

00:00 Dan Bader: One of the differences between both of our setups there that I'm personally really interested in is that for the hosting aspect of the infrastructure we're using different approaches so.

00:00 Michael Kennedy: Absolutely.

00:00 Dan Bader: The way I understand it like, you will actually provision an individual virtual server and then run an application on that, let's say Python Bytes has it's own virtual machine somewhere in the Cloud and more like the infrastructure as a service model whereas most of my stuff runs on platform as a service providers like Heroku. I don't provision my own like Linux server and keep it updated. I just kinda off load that. And so, I'm really interested in how the infrastructure as a service model is working for you or if you ever considered switching for example or if you see some benefits in it?

00:00 Michael Kennedy: It's working pretty well. I feel like I've definitely gained a lot of experience that I wouldn't have. Like give me 15 minutes, I could set up a new HTTPs service on Nginx backed by some uWSGI or Gunicorn or something. And having that experience, devop experience I think is nice. I think the way I kinda got there was a little bit of that learning and exploration stuff. I kinda wanted to have that as a skill set so like let me try it. And you know, when I started it was like $10, 2 $5 servers. One for the database. Maybe might have just been one, right? Like on the same server even or something. But it was like really just a quick little easy way to get started and it worked really well for me. I'm like this thing is just perfect. It works, it's up all the time. I love it. As things have grown, it's gotten a little more complicated and I feel like I'm still really happy the way it is. But it's sort of taken on it's own life a little bit to have...

00:00 Dan Bader: Because you're provisioning your own databases, right? You're not going like, Amazon I need a...

00:00 Michael Kennedy: So the way I have it set up is I have MongoDB as my server. I didn't start with MongoDB but I have it. And I have the different services and they have all different databases but they all are on the same database server, right? So all the main sites, training.talkpython.fm, Talk Python itself, Python Bytes. They all have their main server and a load balance failover server. So if I gotta reboot or do maintenance I can just like automatically through Python with an API as part of the deployment stack, what happen, flow, it'll switch it to this other super, super cheap Cloud server. Handle the traffic well enough while it's off and then switch it back. But all of those share a central MongoDB virtual machine that's like, instead of having a bunch of small ones I bought like one high, high performance one and they just share it. And it's fine. I mean if you go log into the MongoDB server and you pull up in load and you say what is the traffic on this server? The average bandwidth is about eight to nine megabits per second continuously. So it's getting a lot of work. But if you pull up the CPU usage it's one to 2%. I mean it's like, it could take way, way more of what it's getting in than it does. So why did I choose that? Why don't I go on Heroku or something like that? I feel like what I got is already working. I'm really happy with the way it works. I can make it do just what I want. Any change that I want to make I feel like the sky is the limit, right? Do I really wanna change Yeah. how Nginx works? Fine, I wanna make it serve HTTP2 too for this part but not for that. I can totally do that.

00:00 Dan Bader: Did you ever have some incident with the database let's say, where you had to rebuild it from scratch or restore...

00:00 Michael Kennedy: I've never had any, the database, it's been that configuration let's say for about two and a half years. I've never had it go down or get corrupted. What I have had is the network infrastructure inside the hosting, inside the host has had issues. So the things couldn't find the server but that wasn't the server process fault or the machine. It was like the glue was broken for a little. But that's still, you know like couple minutes if you sum it up over all those years. The thing that makes this challenging that you avoid which I wanna hear your side of this here in a second is I can manage all this stuff but I'm not running MongoDB in a replica cluster which would be the way for like true 100% availability. Because the changes that I need to make, incredibly small. But about once a month there's a 10 second downtime because the underlying server has security patches. And so I got to reboot the thing that is the center of that hub and so I've actually built stuff in to say, Talk, the training site for example, that I'll flip a switch and say the app is in off mode and every request will send you over to a page that says we're doing a major upgrade. Please hold on. And it has a meta tag that refreshes every five seconds and it notices when the database switch backs on. It takes them back to the page they tried to get to.

00:00 Dan Bader: Come on man, that's not good enough. We need eight nines.

00:00 Michael Kennedy: Now,

00:00 Dan Bader: For the Pythons...

00:00 Michael Kennedy: I realize like some people have to have 100% availability. But for my world 10 seconds down time per month is acceptable, right? I don't know that's the way I have it set up. Tell us about what you're doing at Real Python 'cause I get a lot of traffic. I maybe do more pure bandwidth 'cause I have audio in large quantities.

00:00 Dan Bader: I think you, yeah. Pretty sure you do.

00:00 Michael Kennedy: But you definitely do more 'cause I probably get under two or three million HTTP requests that are data driven Python requests not static files and whatever per month if I add them all up across all the things. But I'm sure that you get quite a bit more. So you're running a totally different set up. Tell us how that goes.

00:00 Dan Bader: The set up that I use for Real Python, all the other stuff like the ethical ads platform, PyCoder's, pretty much all runs on Heroku these days and the downside is it's a lot more expensive than provisioning your own box. But the upside is that you know recently I was going on my first vacation in like a really, really long time and my wife and I we were going on this hike and we're coming back to civilization...

00:00 Michael Kennedy: Very far away from a computer and from monitoring.

00:00 Dan Bader: Of course I brought the computer. I had like...

00:00 Michael Kennedy: Yeah, so did I. I never go without it. Just in case.

00:00 Dan Bader: We're coming back from this hike and I check my phone and there's like 50 Slack messages, not 50 but you know, there's like, Slack is on fire. Like the Real Python's Slack is on fire and I check my email and there's like all these exception tracebacks and I'm, oh my God, oh my God, oh my God. And it looks like the database has some issue with the database. That service has just disappeared. And then I keep reading further down and I get this email from my hosting provider from Heroku from where we have the database set up and the way it works is you just say okay, I want the Postgres database this version with this many rows of capacity. And you get that box provision and they do all the backups and they do, they provide a automatic failover and all of that. And so I scroll down the emails further and I'm like, okay, they detected that five minutes ago and already switched over to the failover automatically and then reprovisioned the master and kind of switched it back. And I didn't have to do anything and it totally saved me probably like at least an afternoon of work while the website would have been down.

00:00 Michael Kennedy: And not a regular afternoon of work.

00:00 Dan Bader: A vacation.

00:00 Michael Kennedy: Sorry honey, we were gonna be on the beach Because... but not I'm in the hotel frantically fighting this bad Internet afternoon of work.

00:00 Dan Bader: Exactly and so I was like, oh man this is great. I very much like that and so I was always a bit worried that something really bad would happen from a hosting perspective if I was gonna start provisioning all my own boxes and then updating them.

00:00 Michael Kennedy: This is a constant fear of mine. Of just like, what happens if it just goes down. It hasn't been a problem yet but that doesn't prove anything. It's just it hasn't happened yet, right?

00:00 Dan Bader: Yeah, totally.

00:00 Michael Kennedy: It's a real fear that I also have all the time. It's pretty interesting. But you're pretty happy with the platform as a service?

00:00 Dan Bader: I'm really happy with it just from a usability friendliness perspective as a developer because I don't have to worry about installing, you know when the whole like Spectre thing happened and all of these like interesting...

00:00 Michael Kennedy: Oh yeah, we had that 10 second downtime for a lot of reasons, so that's a...

00:00 Dan Bader: Yeah, you know I didn't really have to do anything about it because I just basically tell the platform like, here's the code for the app. Here's the services that I need. So I need like red as a cache. I need Postgres as the database and just give me an endpoint that I can talk to or like credentials for the database and they will keep it patched and updated and make sure there's backups and there's like, it's always like a failover, primary, secondary system. And I very much like the sense of security it give me but on the other hand, I mean, it's probably fairly unlikely that your server will just melt down and so what are you really getting? You know, maybe it's worth the trade off.

00:00 Michael Kennedy: Yeah, I mean, you're already protected of hardware failures.

00:00 Dan Bader: That's right.

00:00 Michael Kennedy: On the IAS systems 'cause they'll move your VM across, I've had several times the underlying machine has died and we move your VM over somewhere else and just kept rolling.

00:00 Dan Bader: Have you ever considered running your podcast on real with bare-metal in your living room with your, office room?

00:00 Michael Kennedy: No, I'll tell you a funny story really quickly. Slight diversion but I think it's worth it. So this is from 19, remember where I was, this is like '95, 1994, '95. Remember the first Mozilla browser came out in '93. So I had a friend who was a web designer friend. And this predates all the...

00:00 Dan Bader: Good friend to have.

00:00 Michael Kennedy: Yeah, I mean this is like couple years in the Internet. And he somehow, I have no idea how he did this, he had gotten the job to build the website of one of the major helicopter manufacturers in the world. And he was hosting all of his sites on his laptop in his house. But we had a party at his house and somebody walked by and tripped on the power cord and unplugged it but it didn't turn off right away 'cause it just went to battery. But a few hours later it ran out of battery and it turned off and he went to sleep. He woke up the next day and people were upset like, "Why is the website down?"

00:00 Dan Bader: Oh no.

00:00 Michael Kennedy: We had a server failure, is what he told them. But it was we had a party and someone pulled the cord on a laptop. Is that insane? Would that ever happen now?

00:00 Dan Bader: It's incredible. Yeah, it goes to show like how much this whole industry has changed. I mean maybe that's what they're doing at Heroku. I don't know. It's like in somebody's basement.

00:00 Michael Kennedy: Yeah, I would never consider on like bare-metal. Like there's just, having the failover at the network level and just the bandwidth and the flexibility, the geolocations. I think a lot about where stuff located for different purposes.

00:00 Dan Bader: That's gonna be interesting too to chat about that aspect because I know that for the video delivery for the courses, you're doing some really interesting stuff like basically built your own little like, not little but like a content delivery network to be like physically close to the person that's watching the video.

00:00 Michael Kennedy: The first thing I hear when I talk to people about this is you're not just using a CDN. And I'm sure I'm gonna hear some stuff about this again 'cause it's irresistible it seems to talk about it. But I'm not using CDN, I'm using something else that let's me create, have more control over the visability of the URLs and the content because I don't wanna have just public content even if it's sort of protected on the app side. It's like once you have URLs they're just public, right? So I have a way to create signed short term URLs. But yeah, the video delivery is really important. So that doesn't happen out of my servers. That happens out of other places throughout the world so I have like eight locations. Like there's a site in Mumbai and there's one in Sydney. There's one in Tokyo. There's one in Ohio where we are here and it's on. And so when you go to training.talkpython.fm and you play a video, not the little marketing videos but all the other real content, the first question the site asks is who are you? Are you logged in? Are you allowed to do this? And the answer is yes. Then the next one is, where are you? So, it'll figure out what of the servers that it has available to it, which one is closest to you and then it'll stream it to you from that location wherever you happen to be.

00:00 Dan Bader: Nice, yeah.

00:00 Michael Kennedy: And then to get it in those locations I've got one place I upload to and then this little system, a little like unit, system unit from systemd Python thing that runs continuously on one of the servers and any time it sees a change to the main location it'll synchronize it throughout the other seven places in the world. So you know, upload it in one quick place, walk away, seven, few minutes later, like it's in the different seven locations.

00:00 Dan Bader: So the crazy thing to me here is that that you're also doing your own video transcoding and all of these things to be able to deliver to the app and to different browsers and...

00:00 Michael Kennedy: Yeah, well I tried to just publish the Camtasia videos and they wouldn't play on iOS and then they were too big in other places and stuff. And you've got to have certain underlying, like it's not good enough with just an Mp4. It's gotta have this version or these settings or it can't be over 1.5 megabit in coding for certain things. It's super painful and so, actually shout to to Cecil Phillip. He pointed me towards this. So what I found and started using, I mean, it's been around is Amazon has something called Elastic Transcoder service. So, Elastic Transcoder service let's you come up with a very carefully structured set of transformations for your video and you give it a video and it'll generate that and give it a five minute video in 20 seconds. And you have an output that is perfect for streaming on the other devises. So I wrote a Python script that'll go to a set, like a course video folder, run it, it'll take all the videos, it'll upload them to a temporary location in S3. Then it'll use the API to talk to the transcoder service and have it drop in to another location in S3 when it's done and then when it's done it'll grab it, download it, put it in different spot on my stuff and here's the sort of streaming version of that course you just built. It's awesome.

00:00 Dan Bader: So I mean, its not... And it still seems crazy to me but I mean, I know that you're a really good engineer and the fact that you can pull this off I think is really, really amazing. When you told me about this the first time I just thought it was mind blowing because like, I'm just gonna use Vimeo.

00:00 Michael Kennedy: Well here's the thing. I started out with Vimeo. I started out using Vimeo and it was, it was really good until it wasn't good enough. Vimeo was good but it would periodically the embedded player stuff would stop working, you know, and it'd be down for like 15 minutes. You know, I'd start getting emails, "Michael, sites down." I'm like, yeah, I know. But it's 100% out of my control. Then I'd get questions like hey, can I play this at 1.5 speed. Like, no. Player doesn't do 1.5, it doesn't have speed, so no. I can't do that. Just on and on, can I have an offline version or whatever. Like there's all these things that you start to want and they eventually pile up and you're just like okay, well, if not that then what, right? 'Cause it was working mostly great but then it's like, all right well now it gets really complicated really quickly. But you know, it didn't start that way. I was selling a course, those things were going great and then like I needed more than I could get from that sort of pro set up I had, like the pro account or the business account I had set up with them. And I'm like, all right, that's, thank you for your service but it's time for me to move on and do something else. By the way, this is way more expensive, right, like. I'm paying for the bandwidth.

01:05:56 Dan Bader: Oh yeah. I bet it is.

00:00 Michael Kennedy: And the transcoder's pretty cheap. It's like $10 to transcode like hours of video and you do that once. But the bandwidth is more. This portion of Talk Python to Me is sponsored by Backlog from Nulab. Developers know the importance of organization and efficiency when it comes to collaborating on a team. And Backlog is the perfect collaborative task management software for your team. With Backlog, you can create tasks, track bugs, make changes, give feedback, and have team conversations right next to your code. You track progress with features like Gantt and burndown charts. You document your processes right along side your wikis. You can innovate with the tools you use everyday like Slack, Jira, and Google Sheets. You can automatically register issues from email or web form submissions. Take your work on the go using their top rated mobile apps available on Android and iOS. Try Backlog for your team for free for 30 days using our special URL at talkpython.fm/backlog. That's talkpython.fm/backlog.

00:00 Dan Bader: So what do you say the fact that this also makes your offering unique, your website is unique sort of like a bespoke design build it from scratch, like the search, all of that stuff is unique and makes you special and it makes, I think, as a visitor or reader, follower, consumer, it feels different to go on one of your websites than like a run of the mill like WordPress site. So is this something that sort of happened organically or is this something that that was your goal initially?

00:00 Michael Kennedy: I would say it somewhat happened organically but it was also somewhat of a goal. Especially around the training stuff. There were a lot of options. Like I could have used Teachable or Thinkrific or Udemy or I could have partnered up with Pluralsight or one of those sets of people or those types of platforms and the consideration there was this is like my bet on my future. And this is the most important thing I consider that I'm doing for the next couple years. I want it to be as nice as it can be. I know I can build a better site than Thinkful or whatever, Teachable, these different places. And I think it'll be more special if it's not just under some subscription place like Pluralsight or Safari Online or whatever, right? And so, I actually thought as hard as I could like, is there a way I can avoid doing this? 'Cause I really don't want to. It's getting in my way and it's gonna take me weeks. Like that actually took a long time. It was like almost a month to write the site and all that stuff. But in the end I decided, you know, if I just do it like a bandaid, just pull it off, do it now I'll be in a better place, you know? And like, keep in mind this is the decision I had to make the day I quit my job and I had no income and I could have no income until that was done. It wasn't just like, oh maybe I'll evolve it, whatever.

00:00 Dan Bader: That gets you motivated.

00:00 Michael Kennedy: Yeah, that'll get you going. Like you'll get it right quick.

00:00 Dan Bader: You know, when I think back, we've been chatting about this stuff for a long time.

00:00 Michael Kennedy: Yeah, absolutely.

00:00 Dan Bader: Right, kind of developing our stuff there in parallel. And you've sort of drawn me into that direction more to do more custom stuff.

00:00 Michael Kennedy: We had a real similar debate, right? You're like I could probably just set up some WordPress thing and make this happen for Real Python or whatever it was you were doing at the time.

00:00 Dan Bader: My first sort of Python education website, dbader.org which is still around. Sort of my personal thing there. That was a custom, sort of like a static site generator but it was just all serving it from memory. So it was like written in the Bottle framework. And you know, way back when I wrote it in 2012 I didn't really know what I was doing to be honest. So, it's kind of limping along and when I had the chance to do things properly with Real Python I was sort of torn between oh, I would love to do something custom but I also know that there's downsides to it in the long run.

00:00 Michael Kennedy: It's like a constant friction that is on you. Yeah, like I would say I spend at least 10% of my time just keeping things running and smooth and adapted. It doesn't, I don't know why but it seems like there's always just a little bit of friction. And you wouldn't have that necessarily with the other stuff, right?

00:00 Dan Bader: Yeah, well or so you would think, right?

00:00 Michael Kennedy: Yeah, I guess. So you would think.

00:00 Dan Bader: I was talking to a lot of people just in the web dev space and you know I used to work as a backend engineer for a company in Vancouver and shout out to Mobify, really cool company. And my friends were pushing towards like, just use WordPress you know, you can get all the plugins. If you want to hire somebody there's a lot of people who know about WordPress and PHP.

00:00 Michael Kennedy: Right, buy a theme.

00:00 Dan Bader: You can buy a theme. You know, there's like this whole ecosystem for this stuff. And back then I think we had a similar conversation about this and you're like yeah, you should really consider doing a custom implementation because then you can tailor it exactly the way you want to. And I'm so glad I changed my mind or partially you convinced me and that I rewrote realpython.com as a Django app because now it is sort of a super power being able to have really tailored and unique workflow for all the author's contributing to the site. You know, for our editing team and just to be able to run something like a little linter on an article. And I don't have to A, write that in PHP and then B, I think the other thing is as soon as you get so for example my wife's website, she has a graphic design portfolio, we built that on WordPress and that's cool and all. You're so effective and fast in the beginning but then you start running into all these issues with these plugins playing together.

00:00 Michael Kennedy: And maybe you don't even fully understand the underlying tag like, you can jump in to Python and fix it. Like can you actually resolve that problem with PHP?

00:00 Dan Bader: Yeah, no.

00:00 Michael Kennedy: No, me either. I, some people yes, but me, no.

00:00 Dan Bader: Yeah that's a big pain now. Where it's like well, this plugin is getting updated and then there's like a new big new release or some security patch and these things are not all updating in sync and they kind of all doing their own thing and it blows up a lot. Whereas if you run your own application, you have a lot of control over the deployment process also. Like in retrospect, now I'm really glad that it's a custom Python app.

00:00 Michael Kennedy: Yeah, that's awesome.

00:00 Dan Bader: I guess the question's how are we all going to feel about this in five years from now where it's like oh God, we've got to maintain this thing. Actually this brings me to another question that we haven't really touched on yet. Like choice of Python web, frameworks. So you're using Pyramid, right?

00:00 Michael Kennedy: Yeah, I'm using Pyramid. And you're using Django.

00:00 Dan Bader: For the most part, yeah. Django.

00:00 Michael Kennedy: For the most part. I'm also using a little bit of Flask for some APIs that people don't see but they're there.

00:00 Dan Bader: Got some bottlepy in there still for dbader.org but...

00:00 Michael Kennedy: Yeah, exactly.

00:00 Dan Bader: Everything else is Flask.

00:00 Michael Kennedy: Yeah so, that's an interesting one. You know, there's two things that I really, really liked about Pyramid and keep in mind this is 2015 not 2019 when I decided these things, right? So, at least for one of them. One's unchanged, one's true. For when I went like, my personal web style is like a minimalist bring the little pieces together. I'm kind of a micro framework type person. So I thought Django's nice but I feels like kind of bigger, I need to understand more of it at once. And I felt like with the micro frameworks I could learn it piece by piece, you know? Like I'm gonna understand this little part here and I don't know what that does. Well, I don't need to know. I mean, that's actually on the Pyramid framework site. Like one of their principles is you can be effective with a partial understanding of Pyramid. so anyway. So I was considering Pyramid. I was considering Flask and a few other things. But at the time, super underscore that, at the time I went to the Flask website and said this is for Python 2. It may, may work on Python 3. Your mileage may vary. We have no guarantees for you there. And I'm like whoa. I don't know if I want to get into that. I don't want to start a brand new thing on Python 2 if like that's it's world. I did a quick check of the performance stuff and Pyramid was faster than the other two on Python 3. I think Flask was faster on Python 2 but that didn't make me happy. I wanted to be on Python 3. So you know, having no legacy code I'm like all right let me try this Pyramid thing. The other reason I really like Pyramid is the Chameleon templates. Like I very much dislike the style of the Jinja template and it's not just Jinja. You know, C# has the Razor space that's very similar. Django's templates are similar. What I don't like is template languages that are very much not HTML. Like you can't take a Jinja template and load it in a standard HTML editor and have it think it's okay, right? Because you've got that angle bracket, percent, for, something, something. Like there, angle bracket, percent, and for all over the place, right? Where as Chameleon is more angular or view like, like Vue.js like where it's attribute driven. So you might have an attribute that is not understood. But it's valid HTML other than tal, colon, condition has a string. So I gotta know what attribute means but it's valid HTML. And to me...

00:00 Dan Bader: But does it actually like parse the template as XML or whatever and then they can make sure that it's valid?

00:00 Michael Kennedy: Yes, it does. 'Cause if you have an unclosed tag it freaks.

00:00 Dan Bader: That is cool. I had no idea.

00:00 Michael Kennedy: It's like true HTML that's like extended by Python and also it's more you can't write arbitrary Python code there. You have to have a little better architectural separation. Those two things when I got started I'm like, okay I like both of these about Pyramid. And I didn't consider Django as much 'cause the micro frameworks bent that I have, I guess. And that's a personal thing, right?

00:00 Dan Bader: Yeah, no for sure, yeah. And I think it also I guess depends on the nature of the site and your work flow. I think you do a lot of terminal driven stuff, right? When you you just showed me your deploy process or when you, like what does it look like for you, I'm really interested in the work flows because for a new podcast episode to go out. Like what's gonna happen once we're done recoding this and you wanna publish it on the website?

00:00 Michael Kennedy: Oh that's a good question. So when we're done recording this right now we're recording it, I'm gonna convert it to multitrack .wav files. And I'm gonna hit it with some audio filters and quick editing, quick leveling so even if our mic settings are different or distance we're holding the mics are different or normally it's recorded at different locations, it all sounds like we're in the same room mostly. I'm gonna send that off to Marco, shout out to you. I'm sure you're editing it now. And Marco is an awesome guy that's been supporting me for a long time and he edits it into its final form. I take that back. I actually have a bunch of other little bits like the ad bits and the intro bits and the music and I do all that in Garageband. Because Garageband is good for multi track stuff. I don't actually do the audio editing there but I do like the composition of the tracks there. Put that into an mp3. I upload that to S3.

00:00 Dan Bader: Like manually with the...

00:00 Michael Kennedy: I use Transmit.

00:00 Dan Bader: Cyberduck or yeah, okay.

00:00 Michael Kennedy: Yeah, exactly. So I'll upload that. And then, then it goes in to the Python world. So then I log into a backend on Talk Python and I say here's a new episode. I type in the details and I say here's the URL on S3 and you could just serve it off S3 but the amount of bandwidth that's going through there. Like let me just do a quick test here. So we have, I want to have the right numbers and I don't wanna making up stuff here. So if I serve it off S3 it would be like $600 or $700 a month. Which is not the end of the world but it's, it's not ideal I guess. Especially when I was getting started. And so, I actually have another server, another virtual server that has tons of available paid bandwidth. And with that I just put it on there and just serve it out of Nginx as a static file. So when I add the S3 URL it will grab it, it's sort of a cache server, it'll ask from the place it actually comes from and if it's not there it'll get it from S3 but if it is there it'll server it off disk. And the reason is, that I'm even doing that is if for some reason I need to stop doing that or it goes down then we're, I can just flip a switch and it'll fail back to S3. So S3's always there. It's like a reliable backend for the media content.

00:00 Dan Bader: As far as like updating the content on the website goes?

00:00 Michael Kennedy: There's like a little editor backend I built.

00:00 Dan Bader: Okay, like a little web form or something where you could.

00:00 Michael Kennedy: Yeah, exactly. Like four or five pages. Then I'll edit episodes

00:00 Dan Bader: Interesting.

00:00 Michael Kennedy: that a guest, things like that.

00:00 Dan Bader: Yeah it's interesting like, I was just thinking maybe a big part of the different approaches there is for your work flow because it's heavily media based. Like the deliverable is an mp3 file and so...

00:00 Michael Kennedy: There's a lot of steps I can't avoid no matter what, right?

00:00 Dan Bader: Yeah, and it's like a different you know when I compare this for example to Real Python, Real Python is more like a multi-user online magazine really, right? Like on the from the backend prospective. And Django is great for that because it was built for that purpose. You know, at a news organization and so, at a newspaper journal. So it was like a perfect natural fit for that.

00:00 Michael Kennedy: Right, of course.

00:00 Dan Bader: Because it gives you all this amazing admin interface and all that stuff. But when you're doing a podcast like a lot of the technical aspects, you gotta build a custom work flow for that anyway because a lot of it is audio processing, distribution and the content delivery. Whereas with more like a magazine based site it's more, you're doing a website.

00:00 Michael Kennedy: You need multiple people interacting with the site side of things. Where my main work is on the media locally and it just happens to be delivered there. Yeah, interesting. Here's the thing. It's like could I change things to do something better? Maybe, maybe not. But it's like...

00:00 Dan Bader: Total relative.

00:00 Michael Kennedy: It's just working so well for me. It's totally spot on. And I'm happy so, you know, it's all good.

00:00 Dan Bader: Yeah, yeah. I think that's the biggest thing to keep in mind there that it's all as long as it works. There's not necessarily like, there's not a huge benefit to improving things much beyond that point. I mean, I wanna rewrite things all the time. But realistically, it's a diminishing returns thing.

00:00 Michael Kennedy: Right, absolutely.

00:00 Dan Bader: For example, you launched your mobile apps now.

00:00 Michael Kennedy: Right.

00:00 Dan Bader: That is a huge thing. Instead you could have spent the last year rewriting all of your existing infrastructure, right?

00:00 Michael Kennedy: Exactly, so it's in the hottest new framework

00:00 Dan Bader: The apps are way cooler.

00:00 Michael Kennedy: I could be all Vue.js or whatever. Like on Docker but whatever, I got my mobile out and that adds true value, right? When you're a small group of people working on something you got to focus where it counts. So maybe that's worth talking about a little bit. People ask did I use Python for my mobile apps. So people who don't know, released a few apps where you can take the courses in a much better way on iOS or Android phones or tablets, whatever. There's a lot, especially on iOS, there's a lot of limitations to trying to do that through even a nice web app. It's just not the same so having something truly native there was super important to me and I wanted to do them for a long time and I knew it was gonna set me back in terms of productivity and getting stuff done but I finally broke down and did it. So, I guess the first question's what's it built in?

00:00 Dan Bader: Yeah, is it Python?

00:00 Michael Kennedy: Man I wish it was Python. I tried to make it Python. I even looked at Ionic for a while. Ionic's pretty cool. But I wanted something native for iOS. Something native for Android. I didn't want to do Kotlin and Swift. I have a decent amount of experience with C# and the Javascript. So the two choices were do something with Cordova, PhoneGap type of thing. May be on Ionic and some framework there. Or do something native with C# and Xamarin. So Xamarin takes your code and compiles it down to native instructions on iOS and on Android and it's like a true native app. So that's what I ended up doing.

00:00 Dan Bader: And as far as the native apps go like the mobile apps in this case. I think you've done some experimentation in the past too with like Electron and stuff like that. That's more for the desktop side but in terms of your custom.

00:00 Michael Kennedy: Yeah, I had originally like prototyped it out with Electron JS and I was gonna like move that to Ionic 'cause they have the widgets that look native on the devices. And it just like, it was a constant Whac-A-Mole like, I fixed this, boom that broke. I fixed this, boom that broke. And I'm just like, all right this sucks. I don't wanna do this. Let me try something else. You know, and Xamarin is 100% free. So that's cool.

00:00 Dan Bader: Plus you were a C# developer in another life.

00:00 Michael Kennedy: Yeah, exactly. So I had enough experience that that was not a horrible choice to me. Would I prefer to write in Python? Yes, but can I go write, in fact what I did was, I wrote the prototype in C# and WPF on Windows 'cause I knew how to do that. I didn't know how to do it on iOS or Android. And then I hired a developer to help write that for mobile stuff and just worked with them on that. And that was a really good experience.

00:00 Dan Bader: What was it like publishing these mobile apps? Because I mean, in a way I feel like we're really jaded creating content for web or programming stuff for the web.

00:00 Michael Kennedy: I guess what I could say is I didn't realize how good we had it. I'm serious. I mean, holy moly. Like the stuff would break all the time. I would get billed from the guy I was working with and it would be like, oh, this doesn't work anymore. Oh, did you do a build, clean, rebuild all on it? No I forgot. Oh, now it works. you know, and you're like what the heck is going on with this stuff? You know then you hit the app store and the gatekeepers and oh my God. It's so super scary. There was one person I was working with at Google. And one person, presumably, I'm working within Apple. And both places rejected the app, over and over, and it was like, well if I can't make this person happy the last four months of work go down the drain and my hope of having a mobile app are gone. And there's this person who is massively incompetent. Like let me just give you, I'm not gonna go too long but I just wanna say I have a lot of sympathy for mobile developers now. So, I submit the app on Android and the title is Talk Python training player or Talk Python training courses, something like that. That's the title. The subtitle was take Talk Python courses on, on your device. But in the description it said Talk Python training for Android is the best way to take our courses. You can learn Python. You can learn to be a web developer and so on. Boom. Rejected. Why was it rejected? It was rejected because there's an app called Learn Python and I'm trying to impersonate it. Like, where do you get that? And eventually I see where it says you can learn Python here and oh, they must think, but it's not the title, not the subtitle. There's no graphic that are about it. But way down there's the word you can learn Python.

00:00 Dan Bader: No more verbs man. It's all trademarks...

00:00 Michael Kennedy: So I had to have an argument for four days 'cause there's like a 12 hour turn around period of it can say you can learn Python and it doesn't mean it is associated with an app who's title's Learning Python. Eventually, the thing that broke that lose was let's try a different word. Let's say it was physics or music. You want to, you can learn music with our app. Does that mean we are the app called learn music? No, but like, I had to have this conversation for days with this person and if they just disagreed with me they decided, then I'm out of the app store.

00:00 Dan Bader: You've made that investment to build the apps and somebody's worked on it for weeks and you know, you had to coordinate everything. Like, I paid for it, like pay for the development.

00:00 Michael Kennedy: And you can't find out until you're done. You can't pre-submit it

00:00 Dan Bader: That's really, yeah.

00:00 Michael Kennedy: and see if they like it, right? You can't, this what we're thinking. Yeah, you got it, it's cool, keep coming. You do it all and you try to launch it and they say no and it could be reasonable or it could be unreasonable but anyway, I'm super happy they're out. I guess to round out that conversation, there's a bunch of APIs it works with and of course those APIs are all on Python on the server but the thing you have on your phone is a Xamarin app. Which I think is a pretty cool thing that we have 95% code reuse, visual designers, and debuggers for it. 95% code reuse across the various platforms and so.

00:00 Dan Bader: And I think that's gonna help a lot in just keeping things manageable. Because again, it's coming back to like you're not 100 people working on infrastructure or you got...

00:00 Michael Kennedy: Exactly, I can't have a Kotlin team. Like I don't know all this stuff.

00:00 Dan Bader: Yeah, and you've got to be able and worse case scenario, you've got to be able to go in and jump in and maybe do a bug fix or something.

00:00 Michael Kennedy: Yeah, something quick or whatever, right? Like if it were massive I'd hire somebody but other than that, you wanna be able to at least take it over if you have to somewhat. So that worked pretty well.

00:00 Dan Bader: It's a really cool app. I mean, you showed me later and I installed it on my phone and it's really it's just really smooth and fast.

00:00 Michael Kennedy: Thanks.

00:00 Dan Bader: For a first release like it's really, really cool.

00:00 Michael Kennedy: I'm really happy how it came out. I'm glad we did it but boy it was a shaky process to get there. Not technically but like approval and stuff. I guess one more thing that I'd like to sort of exchange, what I'm doing, what you're doing. Just so people can know is like so we've the self hosted, like you talked about Heroku. I got my servers and so on. But I also use AWS for a bunch of stuff even though I'm not on EC2. I'm not a huge fan of EC2. I think it's really expensive and really complicated. I think it's built for like running Netflix and other massively awesome things. But if I just want a server, you know it's like, well my security groups are, like I just want a little firewall and I just wanna, anyway, let's not make that a rant but even though I'm not using AWS I'm still using Elastic Transcoder. I'm using Simple Email Service. I'm using S3, Stripe, got Stripe, Mailchimp, there's a bunch of other services that get involved here. How about you? Like what else are you using?

00:00 Dan Bader: It's so fun, not fun but it's hard to think about this and remember all the services.

00:00 Michael Kennedy: I know, I'm sure

00:00 Dan Bader: I don't have a big list

00:00 Michael Kennedy: I'm missing some.

00:00 Dan Bader: in front of me. Like for me, in terms of software that I use, Redis plays a big role. It's basically our caching system in between the database and the, not in between any sort of physical direct sense but like the Django app will talk to Redis and put some stuff there as a cache and the sessions live there. And I was wondering are you using anything like Memcached or Redis or any app like that?

00:00 Michael Kennedy: I've considered using Redis for some of the search, like build up a search index and put it there. But in the end, I'm just using MongoDB.

00:00 Dan Bader: I'm using Postgres for a lot of things. Like the full text search. I was like, let's just see how far we can get with Postgres full text search. And it's actually amazing how much modern databases can just do for you.

00:00 Michael Kennedy: Yeah, that's awesome.

00:00 Dan Bader: Where you, yeah you don't have to invent this stuff from scratch or build your own index. As far as infrastructure goes, I've got, actually pythonistacafe.com is hosted on a DigitalOcean box. That's been really helpful.

00:00 Michael Kennedy: Yeah, awesome.

00:00 Dan Bader: Yeah, I've been super happy with their service and just rock solid hosting I felt like.

00:00 Michael Kennedy: Let me throw one thing out there for people listening by the way. If you're thinking about hosting and you wanna be on Amazon check out Amazon Lightsail. You know about Lightsail?

00:00 Dan Bader: It's like really small EPSs.

00:00 Michael Kennedy: So it has the same pricing as DigitalOcean and Linode. In bandwidth you get a free terabyte with a $5 server which is $100 worth of bandwidth on S3 equivalent.

00:00 Dan Bader: Nice.

00:00 Michael Kennedy: And so if you just wanna be on Amazon but you don't want all that stuff like and you know, Lightsail as well. Anyway, keep going.

00:00 Dan Bader: Yeah, also use S3 and then Cloudflare as the CDN and for everything.

00:00 Michael Kennedy: Yeah, you do a lot with Cloudflare, right?

00:00 Dan Bader: Yeah, it's been incredibly useful. They have this great image transcoding service that is totally transparent to you where they will recompress your images and do like a browser detection. So if you're on Chrome or like modern Firefox they will transcode your images to WebP and then serve them out and that could reduce your image sizes to, to basically half them. And I don't have to go and on the backend and write the logic for like, oh does this specific version of Chrome support this format or Safari.

00:00 Michael Kennedy: Could it be WebP? No, it can't be WebP. It's gotta be whatever.

00:00 Dan Bader: They make that decision for you and they can update those rules for everybody.

00:00 Michael Kennedy: That's pretty awesome actually.

00:00 Dan Bader: So, that's been incredibly helpful.

00:00 Michael Kennedy: I guess we're basically out of time, Dan. This has been so much fun.

00:00 Dan Bader: But we've been going for, and there's still so much more we could talk about. Email I think would be super interesting but maybe you'll do another one of these if there's interest.

00:00 Michael Kennedy: Yeah, I guess we can come back. I guess one other thing that people ask a lot, I just want to throw out there. It's like why MongoDB NoSQL versus relational. And to me like, it's just the modeling stuff in the NoSQL database is real easy but the one thing I really love is I have not done a database migration in two years. But the data models has changed a lot. Because the document databases or schema can just adapt most the time to the new classes especially if it's additive and not subtractive.

00:00 Dan Bader: You've never been burnt by that? Like I kind of like having a schema I can trust.

00:00 Michael Kennedy: Yes, so here's the reason I don't think I've been burned by it. Is I have, to me I know the schema safety structure is in the app code. So I have a special tier who's job it is to just talk to the database and it always enforces the schema. And I never ever talk to it any way other than through that part of code. And so...

00:00 Dan Bader: The schema's in the...

00:00 Michael Kennedy: There is still a schema.

00:00 Dan Bader: Business code.

00:00 Michael Kennedy: There is still a schema 'cause I, otherwise the code won't run. But if you just ad hoc it or if you have five apps to talk to it and they don't share that bit then you very well could get burnt. So I'm just super careful 'cause I've done it before.

00:00 Dan Bader: And I think if you're the sole developer working on it I think that's a great model too, where you can take these short cuts and they're basically fine but as soon as you have like a, I don't know if you have of like 50 people working on an application then that stuff just starts breaking like every single day. Where it's like, oh who put a bunch of strings in the same picture field?

00:00 Michael Kennedy: Exactly it's, yeah but for example, I use MongoEngine which is like SQLAlchemy or Django ORM and it will not let you put strings where integers go, for example. So the safety is in the data access layer. But it's not that it's not there. You know what I mean?

00:00 Dan Bader: Right, it's just shifted like lifts the application versus the database.

00:00 Michael Kennedy: Exactly. It's kind of like the monolith versus microservice conversation as well, right? I know.

00:00 Dan Bader: Is there anything that we both really wanted to talk about but we haven't yet? I feel like we covered a lot of stuff. We haven't really talked about Python Bytes I think. The infrastructure for that.

00:00 Michael Kennedy: Well, Python Bytes is pretty much like a fork of Talk Python. So it's real, real similar. It's got its own instance of an app and database and stuff but it's basically the same. We got the training stuff. I think that's pretty...

00:00 Dan Bader: If you were to do this all over again like right now and you had a month to rebuild it.

00:00 Michael Kennedy: I had to start over.

00:00 Dan Bader: Infinite time to rebuild it. How, would you do anything different?

00:00 Michael Kennedy: I don't know that I would do too much different. Like I'm really happy where things are like it doesn't go down. It's easy to maintain. It's cost effective. The way I got here was like a circuitous path of like oh, I'm gonna try this platform with a service thing. Oh, it's kind of not really reliable. Oh, I'm, use SQLite so I don't have to deal with this other stuff but oh, now things are outgrowing that and just I would have gone more towards either where I am or maybe towards Heroku or something like that, where you are, if I felt like a platform as a service would have made me happier. I don't think I would change too much but I would have gotten to where I was a lot better but like that's, more quickly, like that's the curse of knowledge, right, like you kind of. How about you?

00:00 Dan Bader: I feel the same. I mean, I'm pretty happy with how like all of the hosting aspects and the technical side of that I think, I mean it's much more about creating, I have to continuously remind me of that. Where the goal of this business is to provide really, really good training material to people and to enable people to learn better and become better Python programmers and...

00:00 Michael Kennedy: Neither of us are selling tech. We're selling...

00:00 Dan Bader: Yes, exactly.

00:00 Michael Kennedy: Content and education and experiences. This is just the facilitation layer.

00:00 Dan Bader: Yeah, and so from that perspective I'm super happy with the tech and I think, also considering that, what we just said, it's really important that it's fun to work with the tech and that we get to work with all these interesting tools and learn more about them so we can go out and teach others about them and share that. So from that perspective, yeah, I'm super happy. I mean, I wouldn't want to rewrite it in PHP or anything like that.

00:00 Michael Kennedy: No definitely not. The themes, they're just not worth it. Don't do it. All right well, Dan thanks for doing this like sort of, interview me and sort of 50/50 hosting of the show today. I really appreciate it.

00:00 Dan Bader: Been a ton of fun. This is so cool to do this in person. Like sitting next to each other. Like initially I was like, oh, when we're talking to each other there's no delay. Like usually there's like a 200 millisecond, or I don't know how long the delay is.

00:00 Michael Kennedy: We talk over each other all the time 'cause the latency and stuff. Yeah, it's great. The bandwidth is awesome player.

00:00 Dan Bader: All right well.

00:00 Michael Kennedy: It's 3D too. It's all cool.

00:00 Dan Bader: Thank you, Dan, for doing this. And I can certainly recommend folks come out to PyCon next year. Hopefully... Oh yeah.

01:05:56 Michael Kennedy: You can see some of this life as well.

01:05:56 Dan Bader: Oh yeah. No it was great. It was just a ton of fun. We're gonna be PyCon's gonna be in Pittsburgh. That's, I've never been to Pittsburgh.

01:06:05 Michael Kennedy: Me either, I'm looking forward to that. All right, well cheers. Thanks for being here.

01:06:08 Dan Bader: All right, thanks so much for inviting me again.

01:06:11 Michael Kennedy: Bye. This has been another episode of Talk Python to Me. Our guest, or should I say, guest host, was Dan Bader and this episode's been brought to you by Linode and Backlog. Linode is your go to hosting for whatever you're building with Python. Get four months free at talkpython.fm/linode. That's L-I-N-O-D-E. With Backlog you can create tasks, track bugs, make changes, give feedback, and have team conversations right next to your code. Try Backlog for your team for free for 30 days using the special URL talkpython.fm/backlog. Want to level up your Python? If you're just getting started try my Python Jumpstart by Building 10 Apps course or if you're looking for something more advanced check out our new Async course that digs in to all the different types of Async programming you can do in Python. And of course, if you're interested in more than one of these be sure to check out our everything bundle. It's like a subscription that never expires. Be sure to subscribe to the show. Open your favorite pod catcher and search for Python. We should be right at the top. You can also find the iTunes feed at /itunes. The Google play feed at /play and the direct RSS feed at /rss on talkpython.fm. This is your host, Michael Kennedy. Thanks so much for listening. I really appreciate it. Now get out there and write some Python code.