Learn Python with Talk Python's 270 hours of courses

#310: AMA (Ask Me Anything) with Michael Transcript

Recorded on Wednesday, Mar 31, 2021.

00:00 The tables have turned, and this time I'm the guest and you all are the hosts.

00:03 I get tons of questions over email and Twitter asking me about my thoughts on various trends,

00:09 tools, and behind-the-scenes questions around Talk Python. So I've enlisted two listeners who

00:14 are up for hosting a conversation and taking questions from y'all. Thank you to Patrick

00:19 Lobiel and Kim VanWijk, who guest-hosted this episode, where I answer a bunch of audience

00:23 questions in this Ask Me Anything. This is Talk Python to Me, episode 310, recorded March 31st, 2021.

00:31 Welcome to Talk Python to Me, a weekly podcast on Python, the language, the libraries, the ecosystem,

00:49 and the personalities. This is your host, Michael Kennedy. Follow me on Twitter, where I'm

00:54 at mkennedy, and keep up with the show and listen to past episodes at talkpython.fm,

00:58 and follow the show on Twitter via at Talk Python. This episode is sponsored by OutSystems and us

01:05 over at Talk Python Training. OutSystems is a new sponsor with a really cool product,

01:09 so please check it out with the link in the podcast player show notes. We'll be giving away

01:13 five tickets to attend PyCon US 2021. This conference is one of the primary sources of

01:19 funding for the PSF, and it's going to be held May 14th to 15th online. And because it's online this

01:25 year, it's open to anyone around the world. So we decided to run a contest to help people,

01:30 especially those who have never been part of PyCon before, attend it this year. Just visit

01:35 talkpython.fm/PyCon 2021 and enter your email address, and you'll be in the running for an

01:40 individual PyCon ticket. Compliments of Talk Python. These normally sell for about $100 each. And if

01:47 you're certain you want to go, I encourage you to visit the PyCon website, get a ticket,

01:50 and that money will go to support the PSF and the Python community. Congratulations to Janine

01:56 Van Nykirk. Janine won the fourth of the five tickets that were given away for PyCon, and there's

02:02 still more chances to win. If you want to be in this drawing, just visit talkpython.fm/PyCon 2021.

02:07 Enter your email address. You'll be in the running to win a ticket. Now let's get on to that interview.

02:14 Tim and Patrick, welcome to Talk Python to me.

02:16 Thanks. Thanks for having us.

02:18 Thanks for the invitation.

02:19 Yeah, it's, should I say thank you for having me? It's different, right? This is a bit of a unique

02:24 one. So thank you both for being here. I've selected you to represent the audience in this

02:30 Ask Me Anything episode, which should be a lot of fun. So yeah.

02:33 Cool.

02:33 It's great to have you both here and helping out with that.

02:35 Yeah, looking forward to it. Should be good fun.

02:37 Yeah, definitely.

02:38 Yeah, I agree. So you were sort of guests on the show until you turned the tables, I suppose. So

02:42 let's do a quick introduction, just a little bit of background about yourself. Tim, I'll let you go

02:47 first. How'd you get into programming and how'd you get over here onto this show to interview me?

02:53 Sure. And in terms of programming, I've been, or I'm an electronic engineer by training and I did

02:59 electronic engineering type work for 13, 14 years or so, mainly producing television satellite decoders

03:05 at a facility based in Durban in South Africa. So we made satellite decoders and I did all sorts of

03:10 areas on that with C and low level embedded programming and so forth, but slowly moved my

03:14 way into PC programming. And for the last couple of years, I've actually been full-time software

03:18 development using Python mainly for DevOps work. Although DevOps means different things to different

03:22 people. In my case, it's a process automation and industrial is not the word I'm looking for,

03:27 but process automation and infrastructure as code is the word I'm looking for. That's the kind of thing

03:32 we do. So I've been probably writing Python since university today, which was not terribly far off 20

03:38 years ago. So I've been using it for quite a while.

03:39 Yeah. Those are pretty early days of Python. You know, that's at least two thirds of its life there,

03:43 right?

03:44 Yeah, it was 2.3, 2.4, something I get.

03:46 Yeah, absolutely. Well, that sounds really fun. And were you happy to be moving away from more

03:51 low level stuff or do you miss it?

03:52 I don't miss it a great deal. I don't miss writing C. But I enjoyed it when I did it,

03:57 but I don't miss, I don't know if I'd go back to it now, but yeah, I thoroughly enjoy the kind of

04:00 thing I do now.

04:01 I feel the same way. I did a bunch of C programming at the beginning and I'm really glad I had that

04:05 experience. I'm also really glad I don't have to continue having that experience.

04:08 Exactly.

04:08 Awesome. All right, Patrick, how about yourself?

04:11 I don't have the experience that you two guys have, but I think my first course in programming

04:17 was during my physics studies where I had a C++ course, just basic introduction. And I didn't

04:23 really, I liked the idea of programming, but I didn't like the language that much. During my PhD,

04:28 I was mostly working with Mathematica, also some C++ and old Fortran code for simulations. But after my

04:36 PhD, I started as a traffic planning, in a traffic planning company as a transport model and data

04:43 analyst and-

04:43 Like modeling like traffic and stuff like traffic lights.

04:46 Like traffic, yes. Traffic.

04:48 Okay.

04:48 Yeah.

04:48 We did like traffic modeling for cities and areas and stuff like that. Yeah. Even some countries.

04:53 Keep the A4, A6. Is that, what are the Autobahn names there from clogging up or is that more cities?

05:00 Yeah. Yeah. That was also a big part. Yes. The modeling that stuff because that's the most interesting

05:06 form of some of the economic companies there. Yes. I think my first task there was like converting

05:11 some old VBA code to Python that transferred structural data from an Excel database to another

05:17 program, which was horrible. Oh, wow.

05:18 Yeah. It was absolutely horrible. And I immediately realized that I hated VBA and loved Python.

05:24 That's where I stuck. And later I joined Germany's biggest drugstore company. And I'm now there

05:31 a data engineer for tracking data, mostly working with Python and the Hadoop PySpark ecosystem.

05:37 Yeah.

05:37 I see you live pretty solidly on the big data, data science side of Python.

05:42 Yeah. I'm more in the data science, more data engineering. So it's really a lot of software

05:48 engineering.

05:48 I recently had Tobias Macy on the show. We talked about the whole data engineering landscape and

05:53 it's really interesting. Like I had kind of lost track of paying attention to the data pipeline

05:58 stuff. And there's a lot of neat tools that are coming out in that area.

06:00 Yeah, there is actually, but it's always hard to migrate from an existing infrastructure to

06:06 something new, but you always have to. So it's a steady flow of new stuff that's coming

06:09 in where you have to adapt. Yeah.

06:11 Yeah, absolutely. All right, cool. Well, thank you both for being here. Let me just

06:15 give a quick summary of this. Ask me anything episode for people watching and listening. So

06:21 people ask me questions all the time. They'll, they'll shoot me a message over email and say,

06:25 Hey, here's this thing you should check out, or this is interesting. I got a quick question. Oh,

06:28 by the way, here's some question about the podcast and this way or that those have obviously been,

06:33 you know, building up over time. So I thought it might be fun to just let you guys come on and

06:37 basically ask those questions and you two get to be the audience or for everyone out there. And so

06:43 we collected some questions previously. I tweeted this out and maybe mentioned on one of the podcasts,

06:49 let people fill out a form and ask their question, which is actually how you two got here, right?

06:54 Because you both asked a question and there was a checkbox of, would you like to be the interviewer of this whole thing?

07:00 And that's great. And so what we're going to do is we're going to start with some of those questions we collected previously,

07:05 but then I could see people like Hong and Daniel and Sam all in the audience and other people coming.

07:12 So maybe there's going to be some questions from the live stream that we do on YouTube as well.

07:15 So Patrick, I guess we can kick this off and what's the first question?

07:20 So the first question is by Martin Boris. So a little bit of background. So he's listening from Germany.

07:28 So yeah, sorry. Let me also add like everyone put a little tiny bit of background about themselves.

07:32 And so just to kind of make them more part of the show as well, these guys will like fill that in just a tiny bit as well.

07:38 Yes. So he's listening from Germany. So hello from Germany to Germany.

07:42 I think I started around 2016. I love the show's wide range of Python related topics.

07:48 And also the sound quality is usually very good. Yes, that's true. And yeah, the question is,

07:53 what's the first Python code you ever used professionally or got paid for? And is it still in production?

07:59 That's a great question. So what is the first Python code that I wrote in production?

08:04 Well, the first that I got paid for. So really, when I started in the Python, I was working as a full time as a software trainer.

08:14 So I would go around and teach courses. And I used to previously do C# and .NET and C++ and stuff and was super happy to move over to Python and do more of that.

08:24 So the first things that I did around that space was really like a bunch of little example projects and things I kind of explored so that I could teach those ideas to other people.

08:34 So it's, I don't know, it's possibly someone took that and put it into production, but it's not really the type of code like, oh, I deployed this one website or I built that website.

08:42 And then, you know, after that, I started realizing just how neat and powerful Python was started doing some little small projects with it.

08:51 And probably the thing that's still around, honestly, is Talk Python.

08:54 The site itself is the oldest piece of Python code that I've written that's still around and doing anything.

09:01 Yeah. So I think that's probably the first one back in the day.

09:04 You know, it's tricky.

09:05 I wasn't working at the time writing that much software as a software developer when I found Python, I would say.

09:12 So there you go.

09:13 Yeah.

09:13 I would also probably suggest quite a lot of us who write Python, a lot of our Python, if it's still in production, was only ever run once or twice because it was a quick script to fix some problem that the rest of the system had and so forth.

09:24 So it's still around, but not necessarily ever looked at again because it fixed the problem and it was solved.

09:30 Exactly.

09:30 Yeah.

09:31 I've got a huge folder of like, these are the things I need to run to solve problems or get like little reports really quick.

09:36 And I honestly forget if they even exist.

09:38 You know, I'm like, I'm pretty sure a couple of years ago I wrote this thing that'll, you know, output this data or summarize this for me and I could do it again, but maybe I can find that script.

09:46 And yeah, I just, I did one of those like yesterday or the day before.

09:49 Yeah.

09:49 I think it was Monday.

09:50 Yeah.

09:51 Onto the next question.

09:51 It's maybe just worth noting.

09:53 I think I forgot to mention up front.

09:54 We are very loosely grouped the questions together.

09:57 So we're going through a set of kind of software related questions and then there's a couple on the software industry and then a couple of them more podcast specific itself.

10:04 Yeah.

10:05 Sounds good.

10:05 Hey, before you do that, the follow up from the live stream, Sam, Samuel.

10:09 Yes.

10:09 Does how much of the original Talk Python code do you think is still remaining versus how much has been refactored over the years?

10:15 Super interesting.

10:15 I would say a lot of it has been additive.

10:18 You know, people maybe think, oh, it's just a podcast site.

10:20 Like it does a whole bunch of little small things behind the scenes that are not super obvious.

10:24 Like just a couple months ago, I added the ability to have the site like transform during live stream.

10:30 So if you pulled up talkpython.fm right now, like there's like a sort of the whole site is like lit up.

10:34 Oh, we're live streaming right now.

10:35 And it'll like take you over there.

10:37 Right.

10:37 Like you don't get that with say WordPress.

10:39 So there's a lot of stuff behind the scenes happening.

10:40 In terms of the biggest stuff that was ripped out and refactored is it originally was written with SQLAlchemy talking to a relational database.

10:47 And then after a year or two, I switched it over to MongoDB.

10:51 We collect tons of data.

10:53 There's like, I don't know, gigs, gigs of data in the database and, you know, millions of records.

10:59 So I'm like, all right, it's going to be better to switch this over.

11:01 And I like it better anyway.

11:02 So switch that over.

11:03 It probably just ripped out the data layer and put in this new one.

11:06 That's probably the biggest change.

11:07 Yeah.

11:07 The next question that was submitted is from Mahadi, who describes himself as a longtime listener, first time caller who does automation and data science.

11:15 There's two questions here.

11:17 I'll ask them both at the same time.

11:18 You can tackle them, obviously, together.

11:20 First question.

11:21 Are there any new courses or content or projects that you're currently working on that you can share to your listeners?

11:27 And the second question, you've done an amazing job covering almost every major Python framework, what you love about them and what you dislike.

11:33 As a professional content creator, what are the exact features that you're looking for in a Python web framework?

11:38 Not including the common features, but subtleties that may just make up your mind about switching to that framework.

11:43 Awesome.

11:43 That sounds like three questions, right?

11:45 Well, granted, more or less, yeah.

11:46 Yeah, yeah, yeah, yeah.

11:48 Beautiful.

11:48 So first one, courses and content that I'm working on.

11:52 I have two main areas that we're focusing on.

11:54 We have a couple of authors who are working with me.

11:57 Probably the most exciting one that is just about to come out is we have a course on Dask.

12:01 So a Dask and scaling Dask with Coiled.

12:04 That's going to be a real fun series that we're doing.

12:06 I have a course on Twilio and Flask.

12:09 So basically, how do I build like a chat bot that will talk to like a backend and in Flask and then like let people put in database, create an admin backend, notifications for like a company.

12:19 Then they talk back to their customers over WhatsApp.

12:22 So that's going to be fun.

12:23 That's coming out.

12:24 More FastAPI kind of leads into the next one of the other questions.

12:29 And then I'm thinking I haven't quite got there, but I'm thinking of doing a year long YouTube channel of something like a five minute.

12:36 Here's a cool thing in Python this week thing on YouTube.

12:40 Just like screen sharing and like, hey, let's just jump in and oh, did you know about micro or ultra JSON?

12:45 Look, you can use it like this, but it's faster or something like that.

12:48 That'd be nice.

12:48 Yeah.

12:48 Yeah.

12:49 Thanks.

12:49 A single highlight out of your other podcast, basically.

12:52 Yeah, exactly.

12:53 Python bytes occasionally highlights a package we may not know about.

12:56 Yeah, exactly.

12:57 Like so Python bytes, but like let's actually write some code and see it on the screen real quick sort of style.

13:01 So that's in terms of content.

13:03 And then you, the other one was about the web frameworks.

13:07 Go ahead.

13:08 Yeah.

13:08 About Python web frameworks, effectively.

13:10 What is it that you're looking for in one that, you know, given that you seem to have covered quite a few of them, what is it you like most about the ones you like?

13:17 I would say two things that really attract me to paying attention to those.

13:20 One is the sort of modern features, modern feel, right?

13:25 If it's using type hints, if it's using async, those kinds of aspects, it feels like it's leveraging what Python is now and not what Python used to be.

13:35 So those aspects of it make me excited.

13:37 But the other thing in terms of when would I think that it's important to pay attention to or when would I think is switching to it is I look at the basically the derivative of the popularity, the number of stars, the number of people that tell me I should be covering this, the number of articles you see about it.

13:54 So like how fast is it growing and how much excitement?

13:56 So, for example, like FastAPI, I would say is probably number one in that rate of growth category right now.

14:02 And it also touches on the modern features with Pydantic and whatnot.

14:06 And it's not, say, as popular as Django or Flask.

14:10 I think it's if you look at the adoption, I think it's one third compared to them.

14:14 You look at the GitHub stars, it's 50-50, but it's only been around for two years where those other frameworks have been around 15 years.

14:20 So, you know, to be almost as popular, like 50 percent as popular.

14:24 But from, you know, in just two years, that's a pretty big indicator that people are excited about it, at least at the time.

14:30 That is, yeah, I was about to say, granted, if you're half as popular as Flask or a third as popular as Flask and you've been around far less, that is impressive.

14:37 Yeah, exactly.

14:38 It's a big deal.

14:39 And I think it's because these other frameworks, while amazing, it's hard for them to say, oh, we're going to change how you write the code and use like type hints.

14:47 And you can see both those frameworks are working on their strategy to switch to async.

14:52 Whereas with FastAPI, like literally the getting started out of the box gives you an async server.

14:58 You can write async view methods if you want or not.

15:01 And it's just, it's unencumbered in which I think is one of the things that's given it an advantage over the others.

15:07 Perfect.

15:07 Thanks.

15:08 I think that pretty much comprehensively covers that.

15:10 On a side note, I do recall in previous episodes, you've made reference to building your site on Pyramid.

15:16 Sorry.

15:16 Yes.

15:17 There's a pyramid you've used.

15:18 Yeah.

15:18 That's also obviously one of your favorites.

15:20 Yeah.

15:21 Yeah.

15:21 Yeah.

15:21 There's a little bit of Flask in there as well for like some services that are no one would know about or see.

15:26 Maybe our search microservice type thing might be in Flask.

15:30 But yeah, I'm a fan of Pyramid.

15:32 I like it a lot.

15:33 I think it also is in the situation where it's hard for it to make major changes to adopt some of these new features.

15:40 Right?

15:41 Yeah.

15:41 One of the things that attracted me to Pyramid is it's a clean, simple, small framework that lets me go, oh, I'm going to use relational.

15:48 Oh, no, I'm going to switch to Mongo.

15:49 And I don't have to like negotiate with the ORM and the admin integration.

15:53 And like, I can just decide how it's going to work.

15:55 And for me, that's good because I've been building web apps for a while.

15:57 Like, I don't know that's necessarily good for everyone, but for me, that was a big positive.

16:01 And also, Pyramid was one of the first modern frameworks to massively embrace Python 3.

16:07 I remember when I was choosing between Flask and Pyramid, Pyramid's like Python 3 straight away.

16:12 This is like where we're focused.

16:14 And Flask was like, to get out a statement, like, yeah, it'll probably work on Python 3, but we've tested on Python 2.

16:19 I'm like, if I'm starting from scratch, I don't really want to have that as the way that I feel.

16:24 I really want this to work on Python 3, right?

16:26 So that was part of the thing that I think is not really relevant now, but it's part of the decision when I decided to go with it.

16:32 Yeah, it makes a lot of sense.

16:33 Yep.

16:34 Thanks, Patrick.

16:35 I think that's pretty much that.

16:36 Yeah, that was all of them.

16:37 From that question, yeah.

16:38 This portion of Talk Python to Me is brought to you by OutSystems.

16:43 Just in a recent episode, my guest and I were lamenting about how complex building UI-oriented applications for end users has become across the industry.

16:52 There's an entire class of applications that should be a whole lot easier to build.

16:56 That's why I'm really excited about our sponsor, OutSystems.

16:59 They provide a visual, model-driven, yet powerful development environment.

17:03 You create your app's UI by dragging beautiful widgets.

17:06 You connect those widgets to a plethora of data sources like APIs, SaaS services, and more.

17:10 You can connect it to your Python backend data using something like FastAPI or Flask services.

17:16 Then you publish your app as a web portal, a progressive web app, or even a native mobile app at the iTunes and Google Play stores.

17:23 As someone who has created cross-platform mobile apps, this whole system is very appealing to me.

17:28 I encourage you to check it out.

17:30 It's time to stop telling your team and stakeholders that it's too hard to build a mobile app or a website for your project.

17:36 Impress them with OutSystems.

17:38 Just visit talkpython.fm/OutSystems and check out the animated GIF right at the top to see what I'm talking about.

17:45 Give it a try for free at talkpython.fm/OutSystems or just click the link in your podcast player show notes.

17:50 Next question, which is from me, so I'm not going to introduce myself.

17:56 Yeah, you've already done so.

17:57 Yeah, perfect.

17:58 So, yeah, in my opinion, the addition of F-strings and Python 3.6 was the game changer for convincing many people to finally drop legacy Python.

18:07 Many libraries today only support Python 3.6 and greater, and I know that some of them did so because they wanted to use F-strings.

18:16 Do you think pattern matching in Python 3.10 could be a similar milestone, or do you think the current controversies around pattern matching will continue once Python 3.10 will be released?

18:27 I've been thinking about pattern matching.

18:28 It's really interesting.

18:29 It's like a switch statement, but much, much more, right?

18:33 And that's been approved.

18:35 That's pretty recent news that that's been approved and will be in Python 3.10, as you've hinted, Patrick.

18:40 I also agree that F-strings were major, and I think it's interesting to just look back on the history of Python.

18:46 Like, one of the biggest challenges of switching from Python 2.2.3 was around the way that strings behaved in, like, bytes versus strings, where it used to just kind of be all the one thing.

18:57 And it's kind of amazing to me that such a simple, oh, we're just changing this one data structure.

19:02 It has so much influence, right?

19:05 That you'd switch to Python 3.6 because F-strings are awesome.

19:09 Or you would be resistant to switch from 2.3 because the way strings work changes.

19:13 But I think it's true.

19:14 So will pattern matching be that?

19:16 I don't think so.

19:18 You know, it's...

19:19 Let me rephrase that.

19:19 I think for certain use cases, absolutely.

19:22 And I think the way that pattern matching is set up, particularly, it's really useful for things where you're kind of doing, I don't know, maybe almost a reg X type of work where it's not just this value equals that or that.

19:35 But like the name says, I guess, pattern matching, right?

19:38 And you're writing a parser or something like that.

19:41 Maybe it's really, really important.

19:42 But to me, it doesn't feel like it's going to make a major difference.

19:45 I would say it's more going to be a walrus operator than an f-string, which is not to disparage the walrus operator.

19:51 I've come to love that thing, actually.

19:53 I really like it.

19:54 It's like, especially, I think, in the data science realm where you have to do things as a single statement, a single expression, you know, like a list comprehension or something.

20:03 You can really do nice things with the walrus operator, whereas like you could create a variable and assign it and then test it.

20:10 That's not something that's as easy to do without multiple duplication or whatever in, say, a list comprehension.

20:16 So these sort of places where you work in like one line expressions, I think actually the walrus operator is really super powerful.

20:23 So I think there'll be something like that where there's a slice of types of problems that you solve.

20:28 Like, yeah, this is the answer.

20:30 And, you know, 90% of the world will go like, what's pattern matching?

20:33 Well, yes.

20:35 Well, I must say just to chop onto that in terms of the f-strings, to come at it from the other side, I didn't move to Python 3 because I wanted to get access to f-strings.

20:45 As is, I stuck with 2.7 long after there were threes available for legacy reasons.

20:50 And that's what the stuff I'd written was written in and for a variety of reasons.

20:53 But because it's what I was using for my day-to-day job, it's what I was using for my personal stuff as well.

20:57 I hadn't bothered moving to three.

20:58 But having now finally done that, I'm not prepared to go back without the f-strings.

21:02 And it's not just f-strings, but f-strings are a very nice thing to point to, to say, this is why I'm staying with Python 3.

21:10 I'm not giving these up.

21:12 I totally agree.

21:13 You know, it's interesting.

21:14 Python has many ways to format strings.

21:16 You've got the percent modulo thing.

21:18 You've got .format.

21:19 You could even do a plus if you had to.

21:22 All those kinds of things.

21:23 And now f-strings.

21:24 I think the most underrated thing is this.

21:26 When you have an f-string and an expression and you say letter equals.

21:30 And it prints a nice.

21:32 Yeah, the debugging statement.

21:33 Yeah.

21:34 Output like letter equals the letter value.

21:37 That's pretty nice.

21:38 I think it's a 3.8 thing.

21:39 Yeah, I think that's not.

21:40 Which is bad because we use a lot of 3.6 currently.

21:43 But we hope to.

21:44 Hopefully we're soon switching to 3.8.

21:47 Yeah.

21:48 Why is this crashing?

21:49 I've definitely broken the server.

21:50 I took down the website once.

21:52 So one of my websites by using the Walrus operator and a little side script.

21:56 But it tried to parse it and it was on 3.7 at the time.

21:59 Anyway.

22:00 Yeah.

22:00 You got to stay back a little.

22:02 But the thing that's interesting is like f-strings are now both faster and cleaner and shorter.

22:07 There's always been these trade-offs.

22:08 Like, well, this is faster, but this is more maintainable.

22:11 And now it's just clear.

22:12 So I think f-strings are actually pretty important.

22:15 I would put pattern.

22:16 Patrick, I'd put pattern matching in with like Walrus operator level in my guesstimation here.

22:21 So it breaks your code.

22:23 Yeah, exactly.

22:24 It will.

22:26 Yeah, I think I'm running 3.9 on the server if I try to put it out too soon.

22:29 Before we move on to the next question, a comment's just come on from Piotr.

22:32 There have been a few previous comments to do with courses, which I think might make more

22:35 sense to get to when we get down to the podcast side.

22:38 But this one looks like it might fit in here quite nicely.

22:41 Yeah.

22:41 Basically, Piotr, and my apologies.

22:44 In fact, my apologies to everyone whose name I mangled if I mangled it wrong during the podcast.

22:48 Basically, Piotr saying his question, what do you find more exciting these days?

22:52 Web development with Python on the back end or machine learning and AI?

22:55 That's pretty interesting.

22:56 What is more exciting?

22:58 I mean, I think there's philosophical exciting and then there's what am I excited to sit down

23:03 and write?

23:03 And for me, there's something beautiful about just sitting down and building an API that

23:08 you can put out there and just builds a website that feels super smooth.

23:12 The web is so critical to us today.

23:14 And you go around and so many sites are like loading, loading.

23:16 Maybe they're like, oh, we're going to use a front end framework.

23:19 So it seems fast and then it fills up and then it just has a bunch of spinners for a while.

23:22 You're like, no, that's just another way of loading slowly.

23:25 And having something that's just like instant, no matter where you are.

23:28 It's just like there's a I get a great satisfaction in building things like that.

23:33 And so the day to day sort of I'm going to go with the web development in Python.

23:38 But in the big picture, like let's dream sci-fi sort of world.

23:42 Machine learning AI is, you know, very captivating.

23:46 Cool.

23:46 What do you guys think?

23:47 Thomas, I'm with you there entirely.

23:48 I don't do any web development per se, but similarly in a day to day basis, if it turns

23:53 out that for a lot of my work day, I wrote a script that did something relatively simple.

23:57 You know, I spoke to a couple of databases and processed some numbers and maybe told us

24:02 some stuff we didn't know before.

24:03 And it was a script I wrote in a day and I ran it once or twice on the server and we never

24:08 need to do it again.

24:08 That was a good day.

24:09 That was pleasing.

24:10 You know, I sat down, I wrote some Python.

24:12 It did the job and it worked.

24:14 It's yeah, that was exciting to me.

24:16 I like the fact that here's a problem to solve.

24:18 I've written some patterns to solve it.

24:19 And it's a neat self-contained thing as opposed to being big picture.

24:23 Let's say machine learning stuff you do for months on end.

24:25 And that's also very exciting.

24:26 But there is something very satisfying about solving a particular problem.

24:30 Yeah.

24:30 You know, with some Python.

24:31 I mean, and Python makes a lot of it really simple.

24:33 Exactly.

24:34 Yeah, exactly.

24:35 Yeah.

24:35 Patrick, you live closer to the AI world than probably Kim Aradia.

24:38 I actually started as a data scientist at my company and I quickly switched to be a data

24:44 engineer because I realized that that's where the hard engineering and the kind of real work was happening.

24:50 And without the data engineering, data cleaning, the whole, the proper processing of data, you

24:56 cannot do any machine learning or AI.

24:58 You cannot even start.

25:00 That's why I'm there.

25:01 To build kind of a foundation for me.

25:04 At some point, I want to more go into this data science world, more machine learning, AI, which

25:09 is really, I mean, it's incredible what Google puts out there with image classifiers and crazy

25:16 stuff, especially with video or pictures.

25:20 But I would also say it's really nice to solve a problem the right way.

25:25 That's more an engineering part, which I like, which is more like a web framework where it's

25:29 predictable.

25:30 And data science is, it's really dependent on the data.

25:32 And that's hard.

25:33 The hard problem.

25:34 There's a really cool quote that Eugene Yen mentioned on the show last time in some of

25:39 his writings the time before.

25:41 But recently he was on the show and it was that this cleaning and wrangling of data is

25:46 not the grunt work.

25:47 It is the work of data science, right?

25:49 Like that's, without that, you really get nowhere.

25:51 It's like an iceberg.

25:52 It's a 90% below the water.

25:55 Without it.

25:56 Yeah.

25:57 The top bit falls under.

25:58 All right.

25:59 Kim, other live stream questions you want to focus on now or want to move on?

26:02 Well, actually, the other live stream question that maybe is worth talking while we're talking

26:06 web frameworks, question from, I think it's Nechmat.

26:09 Basically, that's the one.

26:11 Basically, what are your thoughts on creating web applications using Streamlit versus Flask

26:15 versus FastAPI in terms of pros and cons and so forth?

26:18 I know you've had whole episodes talking about these things.

26:21 Yeah, yeah, yeah.

26:21 These are episodes.

26:22 I have classes on some of these.

26:24 Yeah, I've put a lot of thought into this, I guess.

26:26 So let's put Streamlit on one side and Flask and FastAPI to the same other side in a

26:32 moment.

26:32 So Streamlit, if you're unaware, is really interesting for, I want to write some code

26:37 that looks like procedural code, maybe in the data science world or in a Jupyter notebook,

26:42 like do this thing, then this, then this, then this.

26:44 Oh, wait.

26:45 Actually, what I want is an interactive website with like a combo box and a select list and

26:49 a slider that's going to like live time update that on the web.

26:53 To go from that script to that front end dynamic website without something like Streamlit

26:58 is a tremendous amount of work.

26:59 It involves like reactor view, requires writing APIs, and it requires you understanding like

27:05 visualization in the route.

27:07 Like there's a lot of pieces.

27:08 So I think Streamlit is incredible from going from like zero to 80 or zero to 70% on that.

27:14 I've got that script and I want to get that up.

27:16 And for most, for many, many people, 70% is plenty fine.

27:20 Like the decision makers or the team want to have a dashboard where they can play with the

27:24 data and look at it.

27:24 And it kind of looks like this shape and I can't do a lot of design on it maybe, but who cares?

27:28 Like they just want access to this and it's better than an Excel spreadsheet, better than

27:32 like letting them query the data directly.

27:34 So I think like that's one realm.

27:36 And the other realm is I want to build like a production grade API that other things are

27:43 going to depend upon and use and so on.

27:45 So that comes down to Flask versus FastAPI.

27:47 I think on the both new feature neatness, you know, Pydanic types, async and await, FastAPI

27:54 wins.

27:55 The APIs are similar, but it's the new magic plus the derivative of like the growth rate

28:01 of FastAPI is higher.

28:02 So on that regard, I think FastAPI is a really good choice.

28:06 On the other hand, Flask has thousands of extensions you can add to it and just make it do things

28:11 really easily.

28:12 So if you would like to depend on that extension, things that people have done for you, you know,

28:17 you're going to get, and you don't want to try to do those things yourself, you're probably

28:20 better off going with Flask in that regard.

28:22 So, you know, I mean, not everything, Brian had a nice statement quite a while ago that

28:27 you're not Facebook, you're not LinkedIn, you're not Google.

28:29 And I know that's a false statement for a very small slice of the world, but for most people,

28:33 your API doesn't have to support a thousand requests a second.

28:36 Five requests a second is fine.

28:37 And you can do five requests a second on lots of data on a $5 server.

28:41 You know what I mean?

28:42 Like exactly with Flask and with no async.

28:45 So I think like don't go too far.

28:47 And also the Flask versus FastAPI, you can switch from one to the other pretty easily.

28:51 There's similar frameworks you can switch.

28:53 It's not sort of a super huge commitment.

28:56 Streamlit versus those other two, that's a big deal.

28:59 That's a big difference.

29:00 Cool.

29:00 Yeah.

29:00 Yeah.

29:01 And Robert says, I agree.

29:03 Flask and Grapeframe were small and medium sized web apps.

29:05 Yeah, I agree.

29:06 I'm reminded of probably not that many shows ago you had some of the Django chaps on.

29:10 And I recall one of them, one of the chaps saying he basically, he ran almost everything

29:14 on the baked in SQLite.

29:16 You can get quite far on SQLite.

29:18 Yeah.

29:19 Yeah.

29:19 That was Carlton Gibson, I think.

29:21 Yeah.

29:21 That was it.

29:22 Yeah.

29:22 You can get quite far on SQLite.

29:23 And that was a good point.

29:24 And not everybody, not all your problems are Google scale problems.

29:27 You don't need to roll out a cluster of databases up front.

29:29 Yeah.

29:29 And if you write to it rarely and you read from it frequently, it may well be very good.

29:35 It's the thing is, you know, it's a lot of people say, well, you got to use Postgres

29:38 if you're in a relationalist because you could do all this amazing things.

29:41 And that's true.

29:42 But my experience is there's a lot of people out there in the world that go, I want to build

29:45 my first website or API and put it on the internet.

29:48 Oh, I also have to learn Linux.

29:49 And that's really intimidating.

29:50 And now I've got to run another server and the database in production with the, on the

29:55 internet where people could hammer on it.

29:56 And there's all the security and just like, they throw up their hands and they run away.

29:59 They're like, this is just too much for me to connect these different servers securely

30:03 and maintain them.

30:04 And, and, you know, the SQLite answer might be like, I can do this versus, ah, it's too

30:08 much.

30:08 Right.

30:09 And I think in that case, it's a huge win.

30:11 There's a strong, I think sometimes overlooked middle ground.

30:13 I sometimes think we get a lot of resources for how to begin AppStage beginner.

30:17 You've never done any of this Python stuff before and all that stuff is brilliant.

30:20 And you get a lot of writing on, well, this is how we do it at, as you say, a thousand

30:23 requests a second at huge scale and clustering.

30:26 And we're running Kubernetes clusters over 15 different locations around the world and all

30:31 that kind of serious complexity.

30:32 But most of us aren't at either of those extremes.

30:34 We're sitting in the middle where, you know, perhaps MySQL is what your company's chosen to

30:39 use, for example.

30:39 And it's just fine.

30:40 It works.

30:41 You know, it's not as good as Postgres, et cetera.

30:43 But you know, a lot of these tools will meet the need.

30:44 The need isn't always as big as you think it will be.

30:47 Yeah, absolutely.

30:47 I mean, I started out on one server, I think.

30:49 And now we have eight.

30:50 But I didn't start with eight.

30:52 That would have been super intimidating, right?

30:54 Like, yeah.

30:54 Yeah.

30:55 Yeah.

30:55 Fantastic.

30:55 Quick follow from Robert.

30:57 A fair number of internal apps used by 10 to 15 people running on SQLite.

31:01 Easy setup.

31:01 Easy to move from server to server.

31:03 Excellent.

31:03 Oh, exactly.

31:04 Yeah.

31:05 And backup.

31:05 You backup the SQLite file.

31:07 You got it backed up.

31:07 Baked into your Docker container, in my case, for example.

31:09 And the people using it don't even know there's a database at all.

31:11 It's brilliant.

31:12 Yeah.

31:12 Beautiful.

31:13 Michael, there were a couple more live comments.

31:15 Or perhaps, I don't know if any of those left to you that you wanted to take.

31:19 Or we should maybe ask one of the questions that were sent in earlier.

31:22 Yeah.

31:23 Magnus had.

31:24 He was in here early.

31:25 So I'll give him the chance to get his thoughts in as well.

31:28 So it says, in your, great, thank you, courses seem to be favoring starting from scratch,

31:34 creating files in order to learn step by step.

31:36 It's not bad per se, but do you have any thought of maybe doing a TDD workflow?

31:40 Would this add more value?

31:42 I think there's a big difference between the right way to create things for somebody seeing

31:48 it for the first time versus the right way to create things for engineering.

31:52 And I would say in the engineering world, I would lean more towards having tests.

31:57 I'm not sure I want to start that.

31:59 I've gone through some experiences where like, okay, what we're going to do is we're going

32:02 to build this whole project up doing TDD.

32:04 And if the whole idea is not to teach testing per se, but something else, I think for a lot

32:09 of people that can be distracting, even though it might be a nice way to simplify it on the

32:14 small scale and the large scale, I think that it can be, you know, just one more thing

32:18 to talk about.

32:19 Like a similar thing that people say is like, well, wouldn't it be great to like also cover

32:23 Docker in your classes so that it doesn't matter what setup you're on?

32:26 It's like, well, yeah, but then I also have to teach people Docker and that's already its

32:29 own, its own other, you know, thing.

32:32 And I ended up using SQLite a lot in all of my courses because I don't want you to have

32:36 to fire up a server and go, why can't I connect?

32:38 Or why can't I authenticate to this other server?

32:40 Or actually even maybe worse, accidentally put data into it because you're playing around

32:44 and then forget that, oh, that's open on the internet.

32:46 Whoops.

32:46 You know, like there's just a beauty to just having like a laser focus on the thing that

32:50 you're trying to do.

32:51 Exactly.

32:52 That said, yeah, my stuff in production, I would say there's not a hundred percent code

32:55 coverage.

32:56 I find, you know, that that last 30, 20 percent, whatever it is, that marginal returns on that

33:02 is much farther down the road than like, what is the essence of my thing I'm building?

33:08 If I was building like a training engine, sorry, a trading engine for like stock market,

33:12 well, the core part that does the trades and pricing like that should have tests.

33:16 The part that does the logging, eh, probably not.

33:19 Exactly.

33:19 The part that emails everybody once a month of the marketing newsletter.

33:24 Exactly.

33:24 Yeah.

33:24 Exactly.

33:25 Right.

33:25 Like, yeah, I mean, it might be great to have it, but it could be that you could be making

33:30 important contributions in the other.

33:32 I don't know.

33:32 I feel like that if you take it too far in any direction that like looking for a hundred percent

33:37 code coverage or something like that, often you can end up spending huge amounts of time.

33:42 Then you get in a situation where you're like, well, it would be really better if we refactored

33:45 this.

33:45 And that's not that much work, but there's like 500 tests that go with that.

33:49 Now we've got to rewrite the tests.

33:50 And it's just like, oh gosh, well, maybe we won't make that architectural shift because

33:54 what was a benefit to like supporting us getting there.

33:58 Now all of a sudden I've got all this like baggage to take along for lack of a better word.

34:01 Anyway, that's, that's probably a larger answer than he was looking forward.

34:05 That's kind of my thoughts on that.

34:06 There were a couple of other comments that you've, you've pretty much covered as well

34:09 on the same vein of, of testing in your courses and so forth.

34:12 Yeah.

34:12 Yeah.

34:12 Okay, cool.

34:13 I don't think that Brian Okun would agree with you, Michael, but, but maybe you should offer

34:18 a course on, on test driven development.

34:20 So that's, that's fine.

34:21 Right.

34:21 Yeah.

34:21 Yeah.

34:22 And if we do a course, probably Brian is going to be the one doing it.

34:25 So I'll let him, you know, I just, I have a very pragmatic feel about that stuff.

34:29 Like, yes, you should have tests.

34:30 You should be able to say, if I push this to production, is it going to crash or not crash?

34:35 Like, is it going to do the thing that it's supposed to do?

34:37 But at the same time, I think it's also really valuable to not put too much extra burden and

34:43 calcify your architecture and your decisions on this package or this structure and be able

34:47 to move quick when you find that there's other things.

34:50 Anyway, that's, that's about this, the tension I'm trying to balance in my world, not speaking

34:54 for Brian, right?

34:55 If I could just maybe pick up on one of the statements you were making on the way through.

34:58 The other thing also to all of us really have to bear in mind is if somebody is paying you

35:02 to do this, there are a number of things you could be doing in any given part of the day

35:06 that would help the company, which is ultimately what you're being paid for.

35:09 It may be that writing the last 90 to 95% test coverage isn't actually nearly as useful

35:14 as the next problem on the list.

35:16 So you got to ask yourself, how upset will either the users or the business folks be

35:22 if that part of the site stops working or if that part of the application stops working?

35:27 And would they be willing to accept a little bit of uncertainty there for, you know, more

35:31 features or something like that?

35:32 Exactly.

35:33 Yeah.

35:33 Yeah.

35:33 That said, when I was, when people would ask, oh, we need this new feature.

35:37 How much is it going to cost in terms of time and energy?

35:40 I wouldn't say, well, here's how much the feature costs and here's how much the test costs.

35:43 And here's how much the refactoring to keep things clean.

35:46 I would just put that into one and go, that's the price.

35:49 And then, you know, then you don't have to have that discussion of, well, yes, you can

35:53 add this feature, but please don't take time to refactor the code so that it's not building

35:58 up technical debt.

35:59 It's like, no, no, no, that's the price.

36:00 This is what it costs.

36:01 Yeah.

36:02 Yeah.

36:02 Yeah.

36:02 Exactly.

36:03 Yeah.

36:03 I think maybe while, while we're going through some excellent comments that have come through,

36:07 it might be worth, there's two more that just leapt out to me that are probably

36:10 interesting discussions for most people.

36:12 There's one from Data Goose, which I can just read.

36:14 It's basically pipn versus vn versus conda, pros and cons at scale.

36:18 Yeah.

36:18 That's an excellent question, I think, of a discussion.

36:21 And then not quite, well, not actually related at all, but in the kind of Python vein, Prodvan

36:26 has asked, with all the new pips being merged and people calling it a bit bloated with features,

36:30 what are your thoughts on that?

36:33 Yeah.

36:33 Yeah.

36:33 I can touch on those.

36:34 I'll get your, both of your thoughts on this as well.

36:37 So for me, I'm not using pipn, I'm not using poetry, I'm not using the other things.

36:44 And that's not necessarily a negative view on those.

36:47 But for me, I've just focused on using Python's virtual BNV stuff.

36:52 A couple of reasons.

36:53 For me, it just works.

36:54 I find it simple.

36:56 I don't have to make sure I have anything installed.

36:58 I don't have to alter my workflow.

37:00 I'm going through a tutorial.

37:01 I don't have to go, oh, well, they said do this thing, but I'm going to do it in this

37:05 other way.

37:05 When I'm talking to other people or maybe making a part of a course, these things, they wax

37:10 and they wane.

37:11 And, you know, it might be poetry this year, but next year it's pipn or vice versa.

37:16 And you bake that flow into a course.

37:18 And then people are like, well, really, you told me to use pipn, but I feel like poetry

37:22 is kind of the thing.

37:23 It's like, oh, yeah, well, you can't change those once they're, you know, solidified.

37:27 Right.

37:27 And these are like months long projects.

37:28 I don't know for me, it's not ideal.

37:30 What I would really like to see is the PEP that has dunder py packages accepted and kind

37:35 of like a, there's this convention of, you know, if I've got an environment, I just run

37:39 Python stuff.

37:39 It just finds the environment.

37:41 But until something like that comes along, I'm going to go with virtual environment versus

37:45 the pep, poetry, et cetera.

37:46 I think conda is a different story, right?

37:49 I think that the, all these other things versus conda, this is, do I choose like the conda

37:54 data stack sort of style of supply?

37:57 Or do I go straight with PyPI and like the raw Python things?

38:00 So I don't do conda because I'm, I'm mostly doing stuff on the web.

38:04 And something that really bothers me about the web is every now and then you're like, oh,

38:09 you need to update, I don't know, some random library that either your framework is your framework

38:15 or your framework depends upon because there's a CVE security vulnerability.

38:19 And you need to update that hours from the time, you know, not, not days from the time,

38:23 you know, that because you pull up the logs, your website, you can see it's just whack, whack,

38:27 whack.

38:28 And it's constantly being hammered, looking for problems.

38:30 And if something major comes along, you want to be able to go now.

38:33 It like the release is out, put it up now.

38:35 And the conda side often has a little bit of a lag because there's more that you've got to

38:40 like build the binaries and stuff, go along with it.

38:43 That latency that's not normally a big deal is a big deal when you are running stuff that people

38:48 could be messing with.

38:49 You want to make sure like if anything needs to be updated, it's updated straight away.

38:53 So that's another consideration.

38:54 But if I was doing, say, Patrick's job, that wouldn't matter probably, right?

38:59 Because it's behind the scenes.

39:00 I mean, so actually I'm using conda on my computer, but mainly because I'm used to.

39:04 So when I worked at my first company, I had to use a Windows computer and I had to do data

39:10 science stuff.

39:11 So it's incredibly hard to install.

39:13 Especially there.

39:14 Yeah.

39:15 Yeah.

39:15 Some packages like Geopandas.

39:17 It's hard to install with pip there.

39:19 Now I'm using Linux most of the time and there it works fine.

39:23 So I'm just using conda for my environments to setting them up.

39:27 Just I have all environments in one with one comments, something like pip.

39:31 But then I normally install just via pip.

39:34 So I'm kind of using it wrong.

39:36 But I know some of my colleagues, especially data science people, they use conda for running

39:42 jobs in the Spark cluster.

39:43 And yeah.

39:44 But we personally, so my team, we're using pecs, which is something like.

39:49 Oh, right.

39:49 Okay.

39:50 Interesting.

39:50 So VNs, something like this.

39:52 Yes.

39:52 Yeah.

39:53 So pecs, that's a way to like package the things you're running up and deliver almost

39:58 like a zip file, if I remember correctly.

40:00 And then like running that directly.

40:01 Right.

40:01 Oh, interesting.

40:02 It's something like a zip VNs.

40:04 You need an interpreter, by the way.

40:06 So you have to have the correct interpreter on your system to then run this file or run

40:12 this pecs file.

40:13 So it's kind of a nice thing.

40:15 It's kind of a binary, which you can chip everywhere.

40:18 That's the thing.

40:19 You can test it on your CI and then deploy it, this thing on a CI and deploy exactly that

40:23 on production system.

40:24 So you don't have to install, alter that in any way from your test environment to production,

40:30 which is nice.

40:30 Yeah.

40:31 That's pretty interesting.

40:32 It's almost like halfway to Docker, but not quite.

40:34 Yes.

40:35 Yeah.

40:37 Yeah.

40:37 And I don't mean that in a positive or negative way.

40:39 It's just, it is like, it doesn't come with the runtime stuff of the machine, like the

40:44 interpreter and other stuff you might set up like Docker would, but it comes with everything

40:48 else, like exactly as you would put it, right?

40:50 Like the packages and stuff.

40:51 So yeah.

40:51 I think this whole packaging and deploying and moving Python applications around, it's interesting.

40:56 It's interesting.

40:57 It's changing.

40:58 We're also looking into poetry now and let's see how this works.

41:01 Yeah.

41:02 Speaking of which, Magnus says, I use poetry to get into pyproject.toml.

41:06 However, not be able to do editable install via pip is a big hurdle.

41:11 Yeah.

41:11 Okay.

41:11 Interesting.

41:12 Yeah.

41:12 Yeah.

41:13 I'm going to have the next question.

41:14 Do it a notebook themes?

41:16 Yes or no?

41:17 If yes, which one?

41:18 Oh, interesting.

41:19 Yeah.

41:19 So for me, I do some stuff in notebooks.

41:22 I have like accounting and what courses are popular and what are people doing there?

41:27 What podcasts are popular?

41:29 And so that kind of analysis and graphic do that in notebooks.

41:32 But boy, it's just the plain old JupyterLab notebooks.

41:37 Nothing super fancy.

41:38 Right.

41:38 I looked at some notebook themes after this question and they look really cool.

41:42 Maybe I should check it out.

41:43 You probably spend more time in notebooks than I do anyway.

41:46 Have you done anything with it?

41:47 I don't know if I spent more time, just some basic stuff to play around with code.

41:51 I think it's superior to IPython, just the way the UI works.

41:55 And I played with Jupyter themes once.

41:58 I tried some, but always at some point there was some weird thing with the CSS, in my opinion,

42:04 and I didn't like it.

42:05 Some table formatting stuff.

42:07 Right.

42:07 That's a challenge, right?

42:08 Like it's how comprehensive is that theme going to be?

42:13 And, you know, once it breaks through and then like, oh, it looks good.

42:16 It looks good.

42:16 Oh, it looks horrible.

42:17 I can't even hardly, and if it gets to the point where you can't even make sense of

42:21 like some graph or something, then it's out, right?

42:23 Yeah.

42:24 And also like the, for example, I played a lot with Bokeh or Bokeh.

42:28 And then also it didn't really fit in that theme, right?

42:32 It's just like wide and everything around it is as dark.

42:35 And it didn't, it didn't just feel right.

42:37 I like the idea of themes.

42:38 I've also, you know, say like PyCharm has like themes and stuff that I put in there.

42:42 And I'm always, oh, well, this one is great.

42:43 Then I'm like, ah, but this part, I don't know.

42:46 And I don't know.

42:46 There's something to be said for just going back to the plain old standard.

42:50 I guess another important question, Patrick, is font ligatures or no font ligatures?

42:54 Are you familiar with those?

42:55 I have no feeling about that.

42:57 Really?

42:58 Okay.

42:59 I kind of like them, but I know that people seem like, why does your code look weird and whatnot?

43:04 So I also normally don't use them because I have the feeling that it's changing stuff.

43:09 That's not in the code.

43:11 It's a bit weird.

43:12 It is weird.

43:13 I kind of like it.

43:14 It's, I kind of feel like I live for the future when I use them.

43:17 The thing that's okay with me is they've changed the way it looks, but only for me.

43:20 Right.

43:21 They don't actually change the meaning, but yeah, they are a little bit crazy.

43:24 All right.

43:25 Maybe catch a live stream question from Moomten.

43:27 What would be the best Python project types to do to learn some cool things you can do,

43:33 things Python can do maybe?

43:35 So, you know, Patrick, I want to get your thoughts on this one as well.

43:38 I think it depends on where you want to go.

43:40 You know, people ask me this often.

43:42 It's like, well, do you want to go down?

43:43 I want to build websites and web APIs and stuff.

43:45 Then there's a series of things that you can do there to get you down that path.

43:50 And if what you want to do is you want to go down the data science or data engineering side,

43:55 there's a different set of things that you should learn to go to it.

43:58 Like, so you should be playing with Jupyter and you should be playing with pandas.

44:01 But if you want to go do stuff on the web, you could completely ignore those things.

44:04 So I think you want to decide in the path that you'd like to take.

44:07 And then start on really small stuff, right?

44:10 Like, you know what?

44:11 I'll give you an example of something I have to do all the time.

44:13 I've got a course folder full of MP4s and I need to turn those into what is the file name?

44:19 How many seconds long is it?

44:21 What is the resulting title of that going to be?

44:24 And I used to just go through by hand and go, okay, we'll go to the finder.

44:28 All right, this is a minute 30.

44:29 So that's 90 seconds.

44:31 And then the title of the file is this.

44:33 And so the title is going to be that.

44:34 And you do that for 150 files.

44:36 You're probably going to make a mistake.

44:38 Not fun.

44:39 So I decided instead of complaining about that week after week, I wrote a script that would

44:44 read the file structure, do some stuff to figure out the time, generate the output that it needed.

44:48 And boom, like all of a sudden, now that's an instant answer instead of an hour long project

44:53 that's AirPro.

44:54 Everybody's life is full of these little types of projects.

44:56 Like, oh, this is super annoying.

44:58 I have to do this all the time.

44:59 Why do I have to keep doing this?

45:00 I keep making the mistakes.

45:01 I do it at the end of the week because I want to put it off.

45:05 And if I do it frequently, like the repetition of it just turns up my day.

45:09 And there's probably a way to do that in Python and make it run in five seconds or less.

45:13 And then that just completely changes the way that you approach like that part of your

45:17 life.

45:17 Like, well, that's solved.

45:18 That's automatic now.

45:19 What else can I do?

45:20 And so, you know, Patrick and Kim as well.

45:23 What do you guys think?

45:24 Sorry, I dropped out just slightly.

45:25 Fair enough.

45:26 The best Python project types to do cool things.

45:29 I would find, in all honesty, exactly what you're saying there.

45:32 What I ended up doing a lot of with Python when I learned it, I didn't have any.

45:38 I wasn't employed at the time.

45:39 I was at university level.

45:40 I wasn't using it for the master's thesis I was pretending to do.

45:44 I basically, a friend of mine knew it and used it.

45:47 And he found it quite nifty.

45:49 And when I looked at him using it, I thought, well, this could be quite a powerful tool.

45:52 And ultimately, I had a few small tasks to automate.

45:55 And for the life of me, I can't remember what they were now.

45:58 But similar kind of things.

45:59 I had, say, 100 files to rename.

46:01 Or I needed to extract the text out of 15 Excel files and join them together.

46:06 Simple things like that that you could do by hand.

46:08 It would take you a couple of hours and it would be done.

46:11 Or it would possibly be error prone.

46:13 You'd make mistakes and so forth.

46:14 So I turned to Python for that kind of thing.

46:15 And ultimately, I found before I knew it that I had, you know, 50 Python files on my system

46:20 doing one thing well once.

46:23 Every now and then, I, you know.

46:24 And you learn other little, you're like, oh, this time I got to work with Excel.

46:27 Now you know how to talk to Excel files.

46:29 Oh, this one's got to talk to a database.

46:30 Now I know how to talk to a database.

46:31 But they're not huge projects.

46:33 No.

46:33 And to think of an example, I acquired some music of some form.

46:37 And I didn't like the way it was, the MP3 tags were done at the time.

46:41 So then I had to figure out a Python library to do whatever those things are called.

46:44 Itags or tags or whatever.

46:46 Yeah, yeah, yeah.

46:46 I've forgotten exactly what the ID.

46:48 Effects.

46:49 That's the one.

46:49 Yeah, something like that.

46:50 Yeah.

46:50 So I had to figure that out.

46:51 And so suddenly I know about a new module.

46:53 And I would suggest, in all honesty, that the best way to learn some Python easily is the

46:58 next time you find yourself sitting down and thinking, well, this will take me a couple

47:00 of hours to click and drag and push the buttons, whatever.

47:03 Think about if you can write some Python to do it instead.

47:06 And keep doing that.

47:07 And before you know it, anything you can't write some Python to do becomes an irritating

47:10 problem because you're used to writing scripts to solve all your problems.

47:13 That's right.

47:13 That's why we both said, oh, we have all these scripts just laying around for these types of

47:17 things, right?

47:18 Patrick, what do you think?

47:18 I can just agree.

47:19 Automate everything.

47:20 That's my answer to that.

47:22 And if you want to go forward, the problem is you only come so far with that, like self-learning.

47:28 At some point, you have to get in touch with more experienced people.

47:32 And that's my second answer to that.

47:35 So at some point, go to a company or go to an open source project where you have a mentor,

47:40 where you have people, where you can grow, which give you the experience, which you only

47:46 can get from other people.

47:47 Yeah.

47:47 You take some online courses.

47:48 You can join something like Python Discord.

47:50 There's a lot of options for where you can find that.

47:53 But yeah, maybe even that means find a job where you can, you're not the smartest person

47:58 in the room in that area, right?

48:00 It's like, because at that point, you've got no one left to ask.

48:03 You're just on the internet on Stack Overflow hunting around.

48:06 And so that can be also a big help.

48:07 Yeah.

48:07 Yeah.

48:08 Never be the smartest person in the room.

48:10 Yeah.

48:10 If you can avoid it.

48:11 Exactly.

48:11 Exactly.

48:12 Also says there are some good books out there, like Automate the Boring Stuff by Al Swagger.

48:15 That's excellent.

48:16 Yeah.

48:16 Yeah.

48:17 I recommend that.

48:18 It's in the same vein of what we've been talking about.

48:20 I was going to say, I would have recommended Mark Pilgrim's Dive into Python, but it's quite

48:24 a few years out of date now.

48:25 And it does really assume you know how to develop software.

48:28 But that's also, that was what taught me a lot of useful things.

48:31 For sure.

48:31 Okay.

48:32 Yeah.

48:32 Other questions?

48:33 You guys.

48:33 Got a few more minutes.

48:34 We've got maybe five minutes and then lots of call it.

48:37 Kim, I think you can choose, right?

48:39 I already stole one of your questions.

48:40 The one about notebook themes or was it?

48:44 Yeah.

48:44 I mean, SQLite and MongoDB, right?

48:46 There's one which we kind of.

48:48 Well, that's because maybe we've got a few minutes, Michael.

48:50 I'm going to self-indulgently ask you the question I submitted, if that makes sense.

48:54 Because I think there's some value there to a lot of people.

48:56 What I basically asked you was more about the software industry.

48:59 And what you're saying, in your conversations with people over the last year or, you know,

49:02 year plus, et cetera, particularly in the COVID times, have you noticed an increased

49:06 appetite for employers to be willing to have their developers work fully remotely?

49:09 I know, as I said, this has become more common in the US and it's common in other

49:13 countries in recent years.

49:14 Yes.

49:14 I'm wondering if the pandemic is...

49:16 Yeah, work for...

49:17 Remotely.

49:17 Yeah.

49:17 Yeah.

49:18 Has the pandemic hastened acceptance of employees from outside the employee's country?

49:22 And I also, this is a South African.

49:24 Yeah.

49:24 And there's lots of small nations like mine.

49:26 We've got a small software industry, for example.

49:28 But if I think of, say, Lesotho and Swaziland, our two nearest neighbors, if you want to do

49:32 any kind of software work, you have to at least come to South Africa.

49:34 And I'm wondering if the scope is opening for those of us from countries like ours to do more

49:39 global work.

49:40 Short answer.

49:41 Absolutely.

49:41 Yeah.

49:42 I definitely, definitely think so.

49:43 I have worked from home in some fashion or another remotely since 2006.

49:49 Those are earlier days of working from home.

49:52 Where it was a little bit...

49:54 And I ended up there in an interesting way.

49:56 I worked for this company that I love working for.

49:58 And they said, well, my wife got her PhD and we moved from the West Coast to the East Coast

50:03 so she could go to a university and teach there.

50:05 And so I spoke to the company I worked at and I said, hey, you know, I'm not going to completely

50:09 hold back my wife's career just so I can stay working here.

50:12 I have to move.

50:12 Like, we're moving.

50:13 Would you like me to still work for you?

50:15 We can figure out something remote.

50:16 And they're like, I don't think it's going to work.

50:18 I don't think we can do this remotely.

50:19 All right.

50:19 I'll find something else.

50:20 Coincidentally, like a great company reached out to me a week later, said, hey, we'd love

50:24 to give you a job.

50:25 And it's like traveling around or remote, whatever.

50:27 It's fantastic.

50:28 I'll take it.

50:28 I moved away.

50:30 The other company called me and said, well, are you going to maybe keep working for us?

50:33 Like, no, I asked.

50:34 You said no.

50:35 Like, I took another job.

50:36 I'm gone.

50:37 Right.

50:37 That was kind of the thing.

50:38 It was like there was maybe we can make it work.

50:40 I really like working with you, but probably not was the way I think that's less so, but

50:46 it's still persisted up until 2019.

50:48 There were many companies like we're going to get this campus.

50:51 We're going to get an amazing office.

50:53 Yeah.

50:53 You got to commute for an hour and a half and be on some random bus and a bunch of traffic,

50:56 but it's fine.

50:57 We'll give the bus Wi-Fi.

50:58 You're going to love it.

50:59 Right.

50:59 And then it was you can either go out of business or you can let people work from home.

51:04 Like, oh, you know what?

51:04 Work from home is good.

51:05 We don't mind this at all.

51:06 And I think specifically for software developers, what has happened is we built up tools and

51:12 ways of working that are already perfect for this disconnected async way.

51:17 We've got Git.

51:18 We've got GitHub or something like it.

51:20 We've got Zoom and other forms of screen sharing.

51:22 We've got voiceover IP.

51:23 We've got the way to say, hey, let's work on this project together.

51:27 I'll just fire up screen sharing.

51:29 And it's better than if I sit at your desk because we both can control the keyboard and

51:33 I can see it right in front of me.

51:34 I'm not like off to the side.

51:36 And I think there's a lot of companies that are realizing that worked.

51:40 We are privileged for that.

51:42 Yeah, absolutely.

51:42 And so what I think is there were many companies who still believed maybe it's possible, but

51:48 it's not for us.

51:49 And they were forced to realize that didn't break everything.

51:52 So maybe we can hire the best talent around the world, not the best talent out of just

51:58 Oklahoma City or out of Lisbon or whatever, right?

52:02 Like we can all of a sudden really expand out and do something amazing.

52:06 So not every company is going to come along for that ride.

52:09 But I think many, many more are open to it.

52:12 That's encouraging.

52:13 Yeah.

52:13 I do agree with you there entirely.

52:14 I mean, particularly in the South African context, quite a lot of employers have had to have

52:20 some of the employers work from home.

52:21 There was a time we all worked from home, but quite a lot of people are now back in the office.

52:24 But, you know, restrictions on gatherings to kind of try to combat spread of COVID have

52:29 meant you can't fill the building.

52:30 So in many cases, the IT staff are the ones who are working from home, because you're

52:34 right, we are the best equipped to do this.

52:35 You know, we're using Git collaboratively, et cetera.

52:37 We're mainly talking to servers that aren't our computers anyhow.

52:40 So whether you do that from inside the office or do it.

52:43 Exactly.

52:43 All this move to the cloud.

52:45 Exactly.

52:45 Yeah.

52:45 The data center is no longer down the hall behind the VPN.

52:48 The data center is out on Amazon or Linode or DigitalOcean or whatever.

52:53 For that matter, even if it is down the hall behind the VPN, you're still talking to

52:56 it remotely.

52:56 You just were sitting at a desk above it instead of at home.

53:00 So that I think I would like to think, and I must hasten to add, if my employer happens

53:04 to be listening, I'm not looking myself, but there are lots of people in the South African

53:08 or otherwise environment, smaller, with smaller industries like ours, where there's talent

53:12 there.

53:13 And most people don't necessarily know you wouldn't come looking to South Africa for a

53:16 software developer, for example, if, as you say, you're not Oklahoma City.

53:19 But maybe there is kind of a good sign that we'll start expanding this kind of look.

53:23 I think so.

53:24 I think it's going to be progressive, but certainly the trends are in that way.

53:28 Everybody's been, you know, shocked.

53:30 Into realizing how this is going to work.

53:32 And I think, honestly, there's probably some managers and some business leaders who are

53:37 like, I actually don't even want to go back to the office.

53:39 I kind of like working from home too.

53:41 Like, I hate the traffic.

53:42 Yeah.

53:42 You know, not everyone.

53:43 Like, yeah, I heard Tim Cook is like, oh, I can't wait till we get back together.

53:46 This is what makes us special.

53:48 And I don't know, maybe it is, maybe it isn't.

53:49 But I think there's still going to be a lot of people who are like, actually, this is a pretty

53:53 good way for us to work.

53:54 Exactly.

53:54 Yeah.

53:55 I think there's a lot of people along the lines of, I'd like to see my colleagues every

53:58 now and then, but keep working from home most of the time.

54:00 Well, we did that company that I was remote for from 2006, 2000, I guess it was.

54:05 We would have once or twice yearly retreats.

54:08 We would go to some fun town and spend a week together.

54:11 Yes.

54:12 And, you know, do basically like sprints or do presentations to each other.

54:16 And it was great team building.

54:18 We'd go to Toronto.

54:19 We'd go to New Orleans or go to London or something like that and spend a week together.

54:25 And then we'd all go back to our remote places.

54:27 But even just that, that one week together and that face-to-face time creates deeper relationships

54:31 that when you reach you, oh, now I feel comfortable to reach out to that person on Slack and ask

54:36 for help or whatever.

54:37 Right.

54:38 Exactly.

54:38 Yeah.

54:39 I think the pandemic showed a lot of people who are a lot of deciders that there are many

54:45 advantages in allowing remote work.

54:47 It's like you get better people, you get more people.

54:49 You don't have to pay for offices, which is like a big deal.

54:53 And people are more willing to come to your company.

54:57 It's incredible.

54:58 You might even like, don't tell all the people who are working, but you might even get more

55:02 time, productive hours from people.

55:04 They're less likely to get distracted walking down the hall.

55:07 Yes.

55:08 They're not spent commuting.

55:09 If they had a long commute, they might just show up a half hour early.

55:12 You could actually get more productivity out of people.

55:15 Yes.

55:15 Essentially.

55:16 Not always.

55:16 For sure.

55:17 There's people who might just go watch TV, but you don't necessarily want to keep working

55:20 with those people.

55:20 Yeah.

55:21 Right.

55:21 Thanks.

55:22 That covers what I was asking quite well.

55:24 Thank you very much.

55:25 All right, gentlemen.

55:26 I think we might be out of time, even if we're not out of questions entirely.

55:29 Yeah.

55:30 There's probably another half set of questions we could ask.

55:33 I don't know if you want to do this again sometime.

55:35 Clearly, it looks like there's a lot of...

55:37 Good stuff, yeah.

55:38 So let's just do the wrap up real quick and just do a quick favorite editor or editor

55:43 use for Python code and then a notable PyPI project.

55:47 So I'm using VS Code.

55:48 I like the GUI, although I like the auto-completion and intelligence of PyCharm more.

55:55 But PyLens is making a great leap in that regard.

56:00 Oh, yeah.

56:00 You started using PyLens over just the Python add-in and you find it makes a difference?

56:04 It's pretty good.

56:05 It's not PyCharm level yet, but I think it's getting there.

56:08 In my case, I would love to use VS Code.

56:11 I've tried to use VS Code.

56:12 I know I should be using a modern tool that's under active development like VS Code because

56:17 new things are added to it all the time.

56:19 But my fingers remember Emacs and I keep coming back to Emacs because that's how I'm most

56:24 productive.

56:24 It is too frustrating to try anything else.

56:26 And I'm sure at some point I will just bite the bullet, sit down and work through the frustration

56:30 point and just get myself under VS Code or something that actually sees development.

56:34 I'm being slightly unfair to the Emacs guys, but I think nobody would disagree VS Code sees

56:38 more development than, say, Emacs does.

56:40 Yeah.

56:40 VS Code has got a ton of people working on it.

56:43 Emacs is where my fingers are.

56:45 Yeah.

56:45 It's muscle memory demands Emacs.

56:47 Yeah.

56:48 And if you said Vim, like a lot of PyCharm and VS Code have Vim modes and stuff.

56:53 And I don't know about the Emacs version.

56:54 It's a set of Emacs bindings for VS Code, but it's just not the same.

56:58 You know, I rely on things that only Emacs can do.

57:00 Things that were probably designed roughly before I was born, but I want those things, you

57:04 know, and I keep coming back to them.

57:05 Yeah.

57:05 And you just like your code to run on, your editor to run on Lisp.

57:09 Well, yes.

57:09 I have actually tried to write some Lisp, but that's in its own right terrifying.

57:13 Yeah, it definitely is.

57:15 And Samuel says, yeah, you might be able to find some Emacs.

57:19 I've tried them and they work, but there's underlying Emacs behavior that I'm just so used

57:24 to that I just can't get around not having, basically.

57:27 Yeah.

57:27 It's just the window is freaking you out.

57:29 They have a window and tabs and stuff.

57:31 Yeah.

57:31 Exactly.

57:31 It's basically, I boiled down to being kind of old and stuck in my ways is what I'm politely

57:35 saying.

57:35 Yeah.

57:35 Yeah.

57:35 Got it.

57:36 All right.

57:36 And then you guys, a quick notable PyPI package.

57:39 Kim, you want to start?

57:39 Sure.

57:40 I actually was going to suggest, I was going to highlight two Python packages that I don't

57:45 think see a lot of attention.

57:46 And they are fairly niche, but I use them a great deal, particularly when I was doing embedded

57:51 type software.

57:52 Because a lot of those devices you talk to have RS-232 serial and that kind of low level stuff.

57:57 And there's a project called PyCereal and a project called PyVisa, both of which are under

58:01 active development.

58:02 And both of which I would recommend to anybody who is getting into RS-232, Arduino type embedded

58:07 work, et cetera.

58:08 PyCereal more so for that.

58:10 It's brilliant at RS-232 work.

58:12 It can simplify a lot of things.

58:14 You might be finding yourself spending a lot of time in a terminal doing this by hand.

58:18 So PyCereal can really help with a lot of this kind of automation.

58:21 And then PyVisa is very niche, but I just wanted to give it a shout out because I don't

58:25 think anybody ever does.

58:26 And it's helped me a lot for years.

58:28 Visa is the protocol effectively that controls industrial automation.

58:31 So things like industrial power supplies and multimeters and the kind of things you find

58:37 scattered around factories that turns valves on and moves things around.

58:39 Yeah, and Python is a perfect way to automate that kind of stuff, right?

58:41 Exactly.

58:42 Yeah.

58:42 Python does it very well.

58:44 I mean, National Instruments has built LabVIEW to be exactly the tool for this.

58:49 But if you're tired of LabVIEW and if you're into Python, I would hazard a guess to say LabVIEW

58:54 maybe annoys you like it annoyed me.

58:56 Something like PyVisa is a great way to get away from LabVIEW for everything.

58:59 And Python can do a lot of this kind of automation work instead.

59:02 Yeah, fantastic.

59:03 Patrick, package?

59:04 I want to mention Click, which is a package for building CLI applications.

59:09 I don't know.

59:10 I think you already mentioned it at some point, but I just love it.

59:13 Oh, awesome.

59:13 So instead of using like argparse or just like sys.args.

59:17 Yeah, it has some Flask wipe with decorators.

59:20 It's because the creator of Flask also created Click, I think.

59:23 Yeah, it's interesting.

59:24 There's these CLI frameworks and these web frameworks.

59:27 And it seems like multiple times the people that make the web stuff also created CLI framework,

59:31 I guess.

59:32 There must be something going on there.

59:33 Yeah.

59:34 But yeah, that's cool.

59:35 Yeah, Click is quite popular and neat.

59:36 Yeah.

59:37 There's also Typer, which is built on Click, which is even easier.

59:42 And it uses type hints if you're interested, Michael.

59:45 Yeah.

59:45 Yeah, Typer's cool.

59:47 That's nifty.

59:47 They're both cool.

59:48 Yeah.

59:48 Yeah.

59:49 It's maybe just worth pointing out that you mentioned sys.org and argpars.

59:52 Since argparse is bundled into the Python library, if you are still using sys.org, argparsemight

59:57 be worth a look.

59:58 You know, if you don't want to install any third-party applications, you can now get your

01:00:02 hands on at least argparseto get yourself closer to...

01:00:05 Yeah, cool.

01:00:05 Good advice.

01:00:05 There's definitely something to be said for you don't have to pip install anything.

01:00:09 You don't have to create a virtual environment.

01:00:11 You can just run it.

01:00:12 And so if using Click would push you into that boundary, maybe it's worth sticking with

01:00:16 this, right?

01:00:16 Yeah.

01:00:17 We also have it in our production system because at some point we weren't able to use a personal

01:00:22 VN.

01:00:23 Now we can with specs, but before we had to use what was there, which was basically nothing.

01:00:28 Oh, interesting.

01:00:29 Okay.

01:00:29 Yeah, yeah.

01:00:30 That's part of the advantage of specs.

01:00:32 All right, Patrick, Kim, thank you for hosting this.

01:00:35 This has been great.

01:00:35 Oh, thank you, Michael.

01:00:36 My apologies that I wasn't here for some of it.

01:00:38 I hope I didn't miss too much.

01:00:39 But yeah.

01:00:40 No, it was great.

01:00:41 The bulk of it that I was here for, I thoroughly enjoyed.

01:00:43 Thank you.

01:00:43 That was very good.

01:00:44 Thank you for having us, Michael.

01:00:46 Yeah.

01:00:46 Yeah.

01:00:46 Thanks, guys.

01:00:47 See you later.

01:00:48 Bye.

01:00:48 Thank you very much.

01:00:49 Cheers, guys.

01:00:49 This has been another episode of Talk Python to Me.

01:00:53 Our guest hosts on this episode have been Patrick Lobiel and Kim VanWijk.

01:00:57 Stop telling your team and stakeholders that it's too hard to build a website or mobile app

01:01:02 for your project.

01:01:03 Wow them with OutSystems.

01:01:05 Give it a try for free at talkpython.fm/OutSystems.

01:01:09 Be sure to subscribe to the show.

01:01:11 Open your favorite podcast app and search for Python.

01:01:14 We should be right at the top.

01:01:15 You can also find the iTunes feed at /itunes, the Google Play feed at /play,

01:01:20 and the direct RSS feed at /rss on talkpython.fm.

01:01:25 We're live streaming most of our recordings these days.

01:01:28 If you want to be part of the show and have your comments featured on the air,

01:01:31 be sure to subscribe to our YouTube channel at talkpython.fm/youtube.

01:01:36 This is your host, Michael Kennedy.

01:01:38 Thanks so much for listening.

01:01:39 I really appreciate it.

01:01:40 Now get out there and write some Python code.

01:01:42 Now get out there and write some Python code.

01:01:42 We'll see you next time.

01:01:42 Bye.

01:01:43 Bye.

01:01:43 Bye.

01:01:43 Bye.

01:01:43 Bye.

01:01:44 Bye.

01:01:44 Bye.

01:01:44 Bye.

01:01:44 Bye.

01:01:44 Bye.

01:01:45 Bye.

01:01:46 Bye.

01:01:47 Bye.

01:01:48 Bye.

01:01:49 Bye.

01:01:50 Bye.

01:01:51 Bye.

01:01:52 Bye.

01:01:53 Bye.

01:01:54 Bye.

01:01:55 Bye.

01:01:56 Bye.

01:01:57 Bye.

01:01:58 Bye.

01:01:59 you you you Thank you.

01:02:02 Thank you.

Back to show page
Talk Python's Mastodon Michael Kennedy's Mastodon