#404: Clean Code in Python Transcript
00:00 Clean code is one of those aspects of your programming career that's easy to put on the back burner, sometimes more by management than yourself.
00:07 But it's important in the short term for writing more debuggable and readable code.
00:11 And it's important in the long run for avoiding having your program take on the dreaded legacy code moniker.
00:18 We're fortunate to have Bob Bilderbos back on the show.
00:22 He's been thinking and writing about clean code and Python a lot lately.
00:26 And we'll dive into a bunch of tips that you can use right away to make your code cleaner, more fun to work with, easier to read, and more maintainable.
00:33 This is Talk Python to Me, episode 404, recorded February 12th, 2023.
00:53 Welcome to Talk Python to Me, a weekly podcast on Python.
00:56 This is your host, Michael Kennedy.
00:58 Follow me on Mastodon, where I'm @mkennedy, and follow the podcast using @talkpython, both on fosstodon.org.
01:05 Be careful with impersonating accounts on other instances.
01:08 There are many.
01:09 Keep up with the show and listen to over seven years of past episodes at talkpython.fm.
01:14 We've started streaming most of our episodes live on YouTube.
01:18 Subscribe to our YouTube channel over at talkpython.fm/youtube to get notified about upcoming shows and be part of that episode.
01:27 This episode of Talk Python is brought to you by Taipy.
01:29 They're here to take on the challenge of rapidly transforming a Bayer algorithm in Python into a full-fledged decision support system.
01:36 Check them out at talkpython.fm/taipy, T-A-I-P-Y.
01:41 And it's also brought to you by my friends over at Brilliant.
01:44 Stay on top of technology and raise your value to employers, or just learn something fun in STEM at Brilliant.org.
01:51 Visit talkpython.fm/brilliant to get 20% off an annual premium subscription.
01:57 Bob, welcome back to Talk Python to me.
02:00 Thanks for having me back.
02:01 I'm excited to be here.
02:02 Yeah, it's really great to have you back.
02:04 You've been on a couple of times.
02:05 We started, I believe, our first discussion on the show way back when was around 100 days of code.
02:12 We went on quite the 100 days of code journey, writing a couple of really, really long courses that were well-received,
02:20 but it took a long, you know, like nine months for us to write, which was amazing.
02:23 And we're back together.
02:25 It's great to catch up.
02:25 Yeah, indeed.
02:26 Yeah, that goes way back to 100 days of Python and two courses that came from it.
02:31 And I think we also did a Django episode.
02:33 Yeah.
02:34 That's right.
02:35 We did do a Django episode as well.
02:36 Good stuff.
02:37 So this time we're back to talk about code quality, writing clean code.
02:42 What are some of the tools?
02:44 And also what are some of the techniques, maybe even mindset people are using to help them write clean code, better code, both for yourself to make yourself happy and as well to be a better teammate if you're working in a team, right?
02:56 Yeah.
02:56 This topic really excites me.
02:58 It's in my everyday work.
02:59 And I think there's a lot to gain from it.
03:01 So, yeah.
03:02 Excited to share today.
03:03 I am too.
03:04 It's something I really, really care a lot about.
03:07 It's one of those topics that I think is long lived, which is rare in software development.
03:12 It's not the latest JavaScript framework or something that's going to last for a year or two.
03:18 These ideas are the kind of ideas that just no matter what you're doing, even if you don't do Python, if you do something else, it's very likely that these are solid foundations.
03:26 Some of the ideas will be specifically for Python, but many of them not.
03:29 Yeah.
03:30 Timeless stuff.
03:31 Yeah.
03:31 Those are the good ones that are worth putting your time into learning.
03:34 Now, before we jump into that topic, which is going to be good, maybe just tell people what you're up to.
03:39 These days, so I left Oracle a while back.
03:42 I think we spoke about that in 2020.
03:44 So, I've been working on PyBytes full-time for almost three years.
03:47 Congratulations.
03:48 I know you and I spoke about this beforehand, and it was really a big step for you.
03:53 It was.
03:54 And something you were absolutely looking forward to.
03:56 But just walking away from a good-paying job, that's scary.
04:00 Stuff.
04:01 So, congratulations.
04:02 Yeah, thanks.
04:03 This is also around the time that, as PyBytes, we pivoted to doing coaching.
04:07 So, helping people one-on-one.
04:09 We created our PyBites Developer Mindset Program.
04:12 It's a coaching program.
04:13 And that really took off.
04:15 And we have worked now with 100-plus people.
04:17 But I'm super excited about every day when I wake up.
04:21 Because helping people overcoming the tutorial paralysis, embracing imposter syndrome, helping them shipping to projects end-to-end is just the best thing ever.
04:30 Yeah.
04:30 To see people grow and become more confident and get better.
04:34 And, yeah, it's really excellent work, isn't it?
04:36 Yeah, it's really fulfilling.
04:37 Yeah.
04:38 So, congratulations.
04:39 It's awesome to hear that you're doing that now.
04:41 And you can put all of your energy, not just your side hustle energy, into these types of things, which I can tell you feels real good.
04:49 You know how it is, right?
04:50 I do, I do.
04:51 I've been in it for a while.
04:52 Very fortunate.
04:53 Okay.
04:53 So, let's start this off with looking at your article.
04:58 But before we do, Toon Army Captain has a really great way to sort of kick off this topic, I believe.
05:03 Clean Code has less or fewer WTFs per line.
05:07 How about that?
05:08 Nice.
05:08 So, yeah, I think that's going to be a theme here.
05:12 Let's go ahead and jump into it.
05:14 I reached out to you and said, hey, let's get together and talk about this.
05:17 Because I knew that you were passionate about these things.
05:20 But then you also put together a blog post here called Tips for Clean Code in Python.
05:26 And I thought, okay, that's interesting.
05:27 Let me start flipping through here.
05:29 And there's just a bunch of nice ideas that, you know, resonate with me.
05:32 So, what I thought we could do is maybe you could set the stage.
05:36 Like, what the heck is Clean Code?
05:37 Toon Army did do a pretty good job of kind of giving us the colloquial term.
05:41 But, you know, maybe something a little more saphoric and formal we could go with.
05:45 Set the stage for what is Clean Code?
05:47 Why does it matter?
05:48 And then we could dive into some of the stuff you put into your article.
05:52 And then also, you and I both threw some ideas in after, like, kind of expanded as well.
05:57 So, let's start with what is this Clean Code thing?
05:59 Yeah.
06:00 To set off the stage.
06:01 So, Clean Code is really when you write code that's easier to maintain, easier to test, easier to update.
06:07 Because, as we all know, a project usually starts simple.
06:11 But over time, it grows.
06:12 New requirements comes in.
06:14 You constantly have to change things.
06:16 And, yeah, the cleanness of your code will definitely determine how easy you can move a project forward, how you can make changes.
06:25 And, of course, it's also a contribution to your team to make it a more pleasant experience and decrease the amount of WTFs.
06:34 Yeah, you don't want it.
06:35 The fewer, the better.
06:36 Although, maybe you'll leave in one or two in just for a good war story at a conference five years from now.
06:40 But you don't want too many of those.
06:42 And I was a free.
06:42 Yeah.
06:42 I think it's appreciating the fact that if you do a good job, what you're building will live a long time.
06:49 Right?
06:49 It will possibly be still the main focus in five years from now.
06:54 And you don't always know what that's going to be.
06:56 Right?
06:56 It could be, oh, here's a little side project.
06:59 It's just going to pull in some data.
07:01 And, like, you know, you show it to somebody at work.
07:03 I'm like, you know what?
07:03 That's great.
07:04 Ship it.
07:04 Like, whoa, whoa, whoa, whoa.
07:05 Ship it.
07:06 No, no.
07:06 I just threw it together.
07:07 It's good.
07:08 Let's go.
07:09 Like, we're in a hurry, you know?
07:10 And then that just starts to build and layer on, like, a sediment that just makes it harder and less fun to work with.
07:17 And so as you just think about this thing is small now, but as it grows, what are both the practices and maybe technological programming things to bring and put in place so that as it grows, you don't start to just really come to a screeching halt to make changes.
07:33 Tests start to fail randomly as you just touch it in various places.
07:38 Things like that, right?
07:38 Yeah.
07:39 It was also kind of my introduction to Python in 2012, so almost 11 years ago when I made this automation framework at work.
07:48 And I was a big fan of Perl back then.
07:52 I guess that goes back to my Unix shell scripting initial programming exploration.
07:57 And it was not maintainable at all, right?
08:00 There were no classes in Perl.
08:01 And it was just a mess.
08:02 Also, of course, because I did no Perl maybe very well.
08:05 But then I discovered Python and I could refactor it, make it more modular.
08:09 And that project was then just a joy to work with.
08:13 And maybe that says a lot about Python as well, that it does inherently things well.
08:18 Because if you type import this in the REPL and you get the send of Python, a lot of these statements sound so simple, but they're actually so profound.
08:26 And they actually tell you a lot about writing clean code, in a sense.
08:30 So anyway, but I digress.
08:32 They do.
08:33 They also talk a lot about the community that cares enough that, you know, you have these sorts of commands in the language itself that remind you to think about writing clean code.
08:43 And going way back to the late 80s, Gito, when he came up with it, really broke with tradition to use white space for the code structure.
08:53 And I think it's a little bit less so now, but back then, that was very much to encourage people to write code in a way that was very readable.
09:01 Whereas, you know, I say less so than now, because you've got a lot of IDEs and stuff where you can push a button and it'll auto format it.
09:07 And you've got things like black.
09:08 And, you know, if you had curly braces, like it could still be quickly put into a pretty shape now.
09:13 But in the early days, that was a really important concept of it.
09:16 These days were spoiled, right?
09:18 Having black out of formatting and all that.
09:20 When we started, that was not the case.
09:22 How did you react to the white space thing when you first?
09:26 At first, I hated it.
09:27 I just really didn't like it.
09:28 I thought this is just weird.
09:29 Like, okay, everything else about this language seems pretty nice, but this is weird stuff.
09:34 Then I started using editors that understood it.
09:36 It's like, okay, well, I just hit, you know, colon and enter.
09:39 And it kind of does it for me automatically.
09:41 And sure, there's three spaces, four spaces.
09:43 But if I backspace, then it goes back forward.
09:46 It kind of treats the structure.
09:48 The tools know the structure.
09:49 And it makes it really nice.
09:51 And at the time, this is long ago, I was doing C#, which is an okay language.
09:55 It's pretty good.
09:55 But it has all the symbols and formulas and all the angle brackets and all the stuff that you would do, right?
10:02 It's not quite C++, but it's pretty close.
10:04 And when I came to Python, I was like, gosh, it's just weird that a lot of that stuff's not here.
10:08 And it seemed kind of out of place.
10:10 But then when I went back, I realized, wait, these languages are lying to me.
10:14 They're saying I need all these symbols to make it hold together.
10:16 And you just don't.
10:17 And it's so much nicer to look at code that is not laden with these support structures.
10:23 So a bit of a diversion.
10:25 But yeah, I reacted weirdly to it.
10:27 But after about a week, I'm like, but it's better.
10:30 It really is.
10:31 Even though I'm so weirded out by it, it's better.
10:33 And then, you know, pretty quickly, I'm like, yeah, this is sweet.
10:35 I'm going with this.
10:36 Yeah, same here.
10:37 Indeed.
10:38 All right.
10:39 Well, let's start, I guess, start at the top with your article.
10:42 What inspired you, by the way, to write it right now?
10:45 Because it's pretty recent, isn't it?
10:46 Yeah, that's a good question.
10:47 It's not the first time I've been writing content about clean code and stuff.
10:53 In 2016, I actually did a formal certification, building maintainable software from the SIG group.
11:01 Software Improvement Group.
11:03 They have formal certifications.
11:05 And yeah, it's a passion of mine, clean code.
11:08 It's also coming back a lot in the coaching I do day to day, helping people.
11:12 So it's not the first content piece.
11:14 But I think this one kind of summarizes in a bit more of a concise way.
11:19 And of course, there's way much more to it.
11:22 But I think the 10 points here get you pretty far.
11:25 So those are pretty important.
11:27 They sure are.
11:28 So let's start with one that you already kind of laid this out.
11:32 Like so many of these techniques, they are like, they sound so simple.
11:35 And I guess, sure.
11:37 Obviously, Michael, I wouldn't write a 1000 line function.
11:43 And you're like, but it's already 700.
11:46 How did this happen?
11:47 Right?
11:47 How do we get here?
11:48 And the first one solidly falls into that realm, which is smaller units, smaller functions,
11:54 smaller classes, smaller modules, single responsibility principle, all that, right?
11:58 Tell us about this.
11:59 Yeah, so indeed, a function or a unit of code should ideally do one thing.
12:04 And just a side note, right?
12:06 That's usually not how we start.
12:08 Sometimes I do want to make this side note that when we're figuring out a design, it can
12:14 definitely happen that you have very large units because you're basically trying to figure
12:18 out what you're building, right?
12:19 And that's where refactoring comes in.
12:21 But we'll talk about that a bit later.
12:23 But yeah, I give a very simple example in this article, right?
12:26 Where a function parses a CSV file, builds up a result list and also prints it out, right?
12:31 So now this function is doing three things and it probably works and it's fine, but you
12:37 cannot really easily reuse something that's doing three things, right?
12:41 So if you want to plug in this function now into something else, there's a lot of things
12:45 happening that probably doesn't make it a candidate to reuse in its current shape or form.
12:49 It's also harder to test because now you have to test parsing and printing.
12:54 And this is kind of a silly example, but just think about another thing you have in your
12:59 code that's doing multiple things.
13:00 It will just be harder to test because there's a lot more going on.
13:03 And yeah, as you said, right?
13:05 These things sound very intuitive and easy to grasp.
13:09 Yet when we look at code bases, right?
13:11 This happens all the time.
13:12 And yeah, so if something's harder to test, it's harder to guarantee that it works.
13:17 If it's hard to extend and when requirements change, it will be harder to update.
13:21 Yeah.
13:22 Yeah.
13:22 It's harder to maintain.
13:23 It's harder to reason about the more things that it's doing, the more state that it might
13:29 be changing.
13:29 It's harder to onboard new people.
13:31 It's just the whole spectrum.
13:35 This portion of Talk Python Amey is brought to you by TypePy.
13:38 TypePy is the next generation open source Python application builder.
13:42 With TypePy, you can turn data and AI algorithms into full web apps in no time.
13:47 Here's how it works.
13:48 You start with a bare algorithm written in Python.
13:51 You then use TypePy's innovative tool set that enables Python developers to build interactive
13:56 end user applications quickly.
13:58 There's a visual designer to develop highly interactive GUIs ready for production.
14:03 And for inbound data streams, you can program against the TypePy core layer as well.
14:07 TypePy core provides intelligent pipeline management, data caching, and scenario and cycle management
14:13 facilities.
14:13 That's it.
14:14 You'll have transformed a bare algorithm into a full-fledged decision support system for
14:19 end users.
14:20 TypePy is pure Python and open source.
14:23 And you install it with a simple pip install TypePy.
14:26 For large organizations that need fine-grained control and authorization around their data,
14:30 there is a paid TypePy Enterprise Edition.
14:32 But the TypePy core and GUI described above is completely free to use.
14:36 Learn more and get started by visiting talkpython.fm/taipy.
14:41 That's T-A-I-P-Y.
14:43 The link's in your show notes.
14:44 Thank you to TypePy for sponsoring the show.
14:47 And it's write functions that are 10 lines long, not 100 lines long.
14:55 Now, I'd like to hear your thoughts on this, but my feeling about all of this stuff is,
14:59 as a general rule, this is what you should do.
15:02 There will be situations where you might, you know, this is just, I can't think of something
15:06 better.
15:06 Or it's way over the top to try to, like, adhere to all of these rules within this small
15:12 context.
15:12 I could tell you, like, on the Talk Python training website, there's a couple of places where
15:18 it's like 75, 80, 90 lines of code for a single function.
15:22 I'm just like, this is so bad.
15:24 I just, I wish it wasn't like this, but it's just weirdly unique.
15:29 And, but there's, you know, 19,000 other lines of code that are all three lines, five lines
15:35 type of thing.
15:36 Right?
15:36 So it's, for me, at least, this isn't a 100% or you're doing it wrong.
15:41 It's a 98% or 95% of the time you should be doing this.
15:45 There may be some weird case.
15:46 And I don't know.
15:47 What do you, how do you feel about that in general on these topics?
15:50 Yeah, as I said before, sometimes you just don't get the design right from the start.
15:54 So inevitably you're going to write some longer classes or functions.
15:58 But then when you break them out in smaller units, the other thing I have to say about
16:02 that is it gives you an opportunity to better documents.
16:05 Because in this example, if you break this long function that's doing three things out
16:10 into three separate functions, parse CSV, buildup list, bad name, but you get the point,
16:16 and print results, then all of a sudden we have three units now and every unit has a
16:20 name.
16:20 Hence, when you go back after a year and look at that code, which always happens and you
16:25 always wonder, what the heck was I thinking?
16:27 Now you add a glance just by looking at the function numbers, you have just already a better
16:33 idea.
16:33 These functions, you can give doc strings as well.
16:36 So at a glance, you have just a much easier time figuring out what this was about.
16:41 Yeah, I appreciate you actually gave a shout out to me in this first piece here, which
16:44 I didn't expect.
16:45 But one of my rules, you know, and I see in this article, you referenced Martin Fowler's
16:50 refactoring book.
16:51 And one of the most important things that came out of that is not all the refactorings, but
16:54 this idea of code smells, right?
16:56 And maybe tell people real quick about code smells.
17:00 Yeah, that's a funny name, right?
17:02 Code smell.
17:02 It is.
17:03 Code that is smelling and this analogy I came up with.
17:06 But yeah, basically it's code that's not following these best practices and it's just not maintainable,
17:13 not easy to extend, going back to these principles we mentioned.
17:16 So yeah, not following these guidelines.
17:18 That code can be smelly.
17:20 It can be as code smell.
17:21 Yeah.
17:22 Yeah.
17:22 Well, the idea is like, it's not actually not working.
17:24 It's just not nice.
17:26 You know, you kind of turn your nose up at it when you see it, but it's, you can't say it's
17:30 broken because it is working.
17:32 It's a good point.
17:33 It smells a little bit off, right?
17:35 It's not quite broken code, but it's sour.
17:38 I don't know.
17:39 And so the reason I bring this up is one of my rules of thumb here that I love, and I think
17:44 you're kind of hinting that with this little shout out that you gave me in this article
17:47 is when we come across these code smells often, especially when you're earlier in your career,
17:53 often the reaction or the first thought is, you know, this part is not nice.
17:58 This part is complicated.
17:59 This part is hard to maintain or hard to understand.
18:01 So I need to put a nice comment.
18:02 I've been told, comment your code.
18:04 Don't be a jerk.
18:05 Comment it.
18:05 So I'm going to put a big comment that describes why it's really bad and all that.
18:09 And in the code smells sort of world, these code comments can be seen as deodorant for
18:15 the smell.
18:16 Like it doesn't remove the smell, but it kind of obscures.
18:19 Like it's not as bad if you saw the comment saying why the code that follows is bad, but maybe
18:25 you could just make it not need a comment.
18:27 And a lot of times, like you just said, that's, well, these five lines of code need an explanation
18:32 because they're mixed in with all this.
18:34 But if they had a function that had a name, whose name was basically the comment, well,
18:39 then you don't need a comment because the name is the comment and it's now small and understandable,
18:43 right?
18:43 Like there's a lot of these little iterative things that go on.
18:46 Exactly.
18:46 Yeah.
18:46 That's a great point.
18:47 And it's, it's very easy to make that refactoring, just turn the comment into a function and we
18:52 have created another unit.
18:53 We can extend, we can test, et cetera.
18:55 So yeah, it's interesting because there are kind of mixed feelings about commands.
18:59 Some people say, voracious, command your code.
19:01 But mostly in their literature, I'm reading that commands are actually not that good.
19:06 And sometimes even labeled as a code smell in the sense that your code should be kind of
19:10 intuitive.
19:11 I think it's going back to that, like half of the time when people are actually putting
19:15 those comments, it's because there's something kind of messy and they're trying to help you
19:19 get over that mess or deal with that mess rather than just fixing the mess, you know?
19:23 Yeah.
19:24 It might be an indication of a bad design, right?
19:26 Yes, exactly.
19:27 So you might even have to go back to the drawing board.
19:28 Yeah.
19:29 But that said, I definitely see a place for commands where sometimes you just want to remind
19:33 yourself and of course your team members, not what the code is.
19:37 They can read the code obviously, but why you took that decision, what the,
19:41 and there might just be some extra context that you need to give.
19:44 Yeah.
19:44 And it's also different than doc strings potentially, right?
19:48 Which is kind of a form of a comment, but it's meant to talk about the API and not cover
19:53 up some weird thing you had to do in the middle.
19:55 Exactly.
19:56 Yeah.
19:56 Doc strings are basically your API documentation.
19:59 Yeah.
19:59 Yeah, exactly.
20:00 All right.
20:01 Well, good, good stuff.
20:03 I mean, even GitHub Copilot and those things are starting to use these comments as a way to
20:07 say, well, I'll just make a function that kind of is named what the comment is.
20:11 You know, so it's even our robot overlords are going to, going to help us by taking these
20:18 comments and removing them, putting the functions there.
20:20 Help us not replace us, right?
20:22 Yeah.
20:22 Hopefully.
20:23 Fingers crossed.
20:24 Another thing related to this, strongly related to this, because the more things you have
20:29 that are smaller units, you're probably still going to have about the same amount of codes.
20:32 You have more functions, more classes, more modules, files, and so on.
20:36 What gets tricky can be naming them, right?
20:39 So choosing good names also, I think, probably is a message in here that people need to think
20:44 about.
20:44 What are your thoughts on that?
20:45 Yeah, it's funny.
20:46 Again, another thing that sounds very intuitive, but wasn't there a saying like there are two
20:51 or three complex problems in science.
20:53 One other one I'm going to remember and naming things.
20:56 Yeah.
20:57 There's two problems that are hard in computer science.
21:00 Naming things, cache and validation, and off by one errors.
21:04 Thank you.
21:05 Yeah.
21:05 So naming is up there, right?
21:07 It is.
21:08 That's right.
21:09 I think it goes back again to knowing your design and what you're building.
21:12 And sometimes you don't really have that figured out.
21:15 Hence, naming becomes more difficult.
21:16 But I think that's an experience thing.
21:18 You get better at it the more code you write.
21:21 Maybe we can touch on the magic numbers there.
21:24 Yeah, absolutely.
21:24 That's the next one is magic numbers.
21:26 And magic, the number part, I think, could be expanded a little bit broader even.
21:31 You know, magic, static values, right?
21:34 But if you see 360, you know, think just in the code, like, well, is that like a degrees?
21:40 Right.
21:41 What is that?
21:41 Probably degrees, but it might not be.
21:43 I don't know.
21:44 So yeah, magic numbers.
21:45 That's your next recommendation.
21:46 I'm presuming to avoid them.
21:48 Yes.
21:48 So when you see some sort of random integer in the code, which might be an intuitive one,
21:54 like 365 number of days in a year, which we might kind of guess.
21:59 But yeah, any guessing in code is not good.
22:02 So it's very easy to then replace that 365 integer with a number on the score of days in a year
22:10 or something like that.
22:11 Something that expresses meaning.
22:12 Yeah.
22:13 And again, this is kind of an obvious example.
22:15 But if you have some magic five or six or 87 in there, the reader of this code probably
22:20 doesn't know what that means.
22:21 Right.
22:21 So if you have a constant uppercase with underscores as per PEP 8 defined at the top
22:26 of your module, I usually put them in the top of the module.
22:29 And then in the code, you see like max underscore, blah, blah, blah, a real name.
22:34 And then immediately makes sense.
22:36 Right.
22:36 And it's also a nice way of grouping those variables in one place.
22:41 I said top of the module.
22:42 But if you have many, you could also have a constants.py in your package to group them.
22:47 You can also use enums to group various constants together if they logically pertain to the same
22:53 group.
22:54 And again, all more readable.
22:55 Yeah.
22:56 Enums are a great recommendation because instead of saying, well, the default Sunday, so we're
23:00 just going to put Sunday here.
23:01 Oh, why is Sunday here?
23:02 This is weird.
23:03 Or you say the function can take a string when really it can only take seven strings,
23:07 seven particular values of string.
23:10 Right.
23:10 You could really clearly communicate that with enums, which is quite nice.
23:15 Yeah.
23:15 And the use of constants.
23:16 That's really a win.
23:17 I'm 100% on board with you on constants.
23:19 All caps, maybe snake case, otherwise, you know, underscores, separate them.
23:25 But I have tooling that I work with all the time that'll say, oh no, you misnamed this variable.
23:30 Python, you know, they want to use lowercase variables, not uppercase variables.
23:33 I'm like, but for constants, they say to use uppercase.
23:36 Why don't you know this?
23:37 It drives me crazy.
23:38 So I was thinking about sort of expanding on this a little bit.
23:41 Are you familiar with this, the typing.final when this was added in 3.8?
23:47 Recently discovered that.
23:48 Yes.
23:49 Super nice.
23:50 So it's also the idea is that it comes along and is meant to be used in this scenario in
23:56 addition to what we've already discussed.
23:59 So the example in the Python docs says there's a max size, capital max, capital size.
24:04 Instead of just saying it's 9,000, you say it's colon final 9,000.
24:09 Now that means nothing to Python.
24:12 It means nothing.
24:14 But it means something to type checkers, right?
24:16 So if you have like mypy or something and somebody tries to change the value of it later, it's
24:22 going to come at least as a linting type of error, if not a full on runtime error.
24:27 So this is something people could leverage to go a little bit further.
24:30 That's super nice type hinting.
24:31 As you will paraphrase, it's meaningless to Python.
24:35 But if you now run your checker and it's somewhere in your code that you try to overwrite that
24:40 variable, which is perfectly fine, right?
24:42 Because Python doesn't enforce, doesn't say anything if you now would assign a new value
24:46 to max size.
24:47 But there is no constant Python.
24:49 Yeah.
24:50 Mypy picks up on that final.
24:51 Hey, it was final.
24:52 Shouldn't overwrite that.
24:54 So super powerful.
24:55 I should use this more.
24:56 Yeah.
24:57 Yeah.
24:57 And so you can say final or I would think preferably final of int.
25:01 So you say what type it actually is in addition to that.
25:04 Now related to this, related to this is it's not exactly a magic number, but I often find
25:10 magic numbers appearing in this situation.
25:13 And that has to do with, I want this thing to have maybe a default value or a value that
25:20 says clearly communicates.
25:21 I haven't set a value for it, but I want it to be.
25:24 Like in this case, a number, it still needs to be an int.
25:26 Well, what integer are you going to put there?
25:28 Hmm.
25:28 Zero.
25:29 Well, zero valid value.
25:31 Do you want it to be falsy?
25:32 Like there's all, could it be negative?
25:34 Which negative values?
25:35 And then you see weird tests.
25:37 Like if it's less than zero, you're like, why is it testing that?
25:39 Well, that means it's not set.
25:40 Like, okay, that's weird.
25:42 So another idea that I think comes in that's pretty interesting is the sentinel pattern where
25:48 you come up with a particular number and sort of store that and say this number or value,
25:54 something you say, if it is this, that means it's unset or it's just a weird case or it's,
26:00 it's kind of like setting it to none.
26:02 But if that's not going to be a case you can use, sometimes none is, is that thing.
26:06 Right.
26:06 But not always.
26:08 Are you familiar with this?
26:09 Do you do anything like this?
26:10 I have not really used it.
26:11 So I had to read a read up on it.
26:13 I think I use none quite a bit, but I think the, I mean, that's at least better maybe than
26:19 minus one is the find example at the top.
26:22 Yeah.
26:22 But I think this even wins by being more explicit even, right?
26:27 By having an object then to test against.
26:29 Yeah.
26:29 So one example of a standard library is the config parser has a underscore unset as a global variable
26:36 and that's its, its sentinel value.
26:38 BZ2 has the underscore sentinel to just call out the pattern directly there and so on.
26:45 My favorite one out of this actually is a variation is this, this null object pattern.
26:51 So null, because it comes, you know, sort of earlier in the life cycle of languages where
26:56 null, not none, but where a lot of times if you see if this value is none, then we can
27:01 do a thing other than otherwise we can do actually work with it.
27:05 Right.
27:05 And you'll see that these, these tests all over.
27:08 And so this pattern is come up with a sentinel value that is not none, but kind of behaves
27:14 in a no op way.
27:16 Right.
27:17 So you could give it a value of a person who is a sentinel and it just, if you try to
27:21 get his name, it's just going to return, I don't know, nothing or something like that.
27:24 Right.
27:25 So you can, if it makes sense in that situation, you could remove a lot of these, if thing is
27:29 none, do something else, do something else just globally, which is really nice.
27:34 So you don't have to do this non check here.
27:36 So are you saving an if else?
27:38 Yeah.
27:39 You're saving, you're saving the cyclomatic complexity.
27:41 You're saving like branching at the cost of maybe allocating more stuff or, or having it
27:47 possibly have this value that doesn't really mean it has a value, you know?
27:50 So you got to trade it off and figure out what it makes.
27:52 That's interesting because I think I do the is none quite a bit.
27:55 So yeah, I do too, because sometimes it's kind of hard to deal with that or you don't control
28:00 it.
28:00 A lot of times kind of going back to when I was talking before about the 95% case and I told
28:05 you about this really gnarly bit of code.
28:07 that's because I'm consuming two other APIs that I have no control over and they're junky.
28:12 They're real junky.
28:13 So, you know, I've kind of got to like fit it together and, and you don't get a pick whether
28:18 they use the null object pattern or not.
28:20 You just got to make it work.
28:21 Right.
28:21 Yeah.
28:21 That's a good point.
28:22 Yeah.
28:22 Yeah.
28:22 Indeed.
28:23 All right.
28:24 So those are all kind of under the magic number category, I would say, even though technically
28:28 some of them are not, like I said, not numbers.
28:29 This one, honestly, I'll tell you, you asked me about kind of my first experiences with Python
28:34 long ago.
28:35 And this one weirded me out a little bit because it's something I had spent a long time in C,
28:39 C++, C#, trying to just remove and weed out.
28:43 And that's global scope.
28:44 Tell us about this and maybe why to watch out for it.
28:47 If we leave something in global scope, for example, we have a global variable and we're
28:52 pulling that into a function with a global keyword and we're making changes to it.
28:58 Now this function has side effects, right?
29:00 So you have the stuff that's going in the function, but it's also mutating some external object,
29:05 right?
29:06 And when you write it, you probably are very aware of it because you're doing it, but you're
29:11 going back and all of a sudden this might bite you.
29:13 It will surprise you.
29:15 Side effects overall are bad, right?
29:17 And the global scope has that potential because it's global scope.
29:20 It's not local scope to some class or function.
29:24 Yeah.
29:24 Basically that, that, it can lead to surprises and we don't want surprises in code.
29:29 Yeah.
29:30 And it's not an argument you pass to a function.
29:32 It's not a return value from a function.
29:34 You might not even know that that is in play as you interact with part of your code, but
29:38 somewhere out there, it's something deep down is reaching up and seeing that value.
29:44 This portion of talk Python to me is brought to you by brilliant.org.
29:47 You're a curious person who loves to learn about technology.
29:50 I know because you're listening to my show.
29:52 That's why you would also be interested in this episode's sponsor, brilliant.org.
29:57 Brilliant.org is entertaining, engaging, and effective.
30:00 If you're like me and feel that binging yet another sitcom series is kind of missing out
30:05 on life, then how about spending 30 minutes a day getting better at programming or deepening
30:09 your knowledge and foundations of topics you've always wanted to learn better like chemistry
30:14 or biology over on brilliant.
30:16 Brilliant has thousands of lessons from foundational and advanced math to data science, algorithms,
30:22 neural networks, and more with new lessons added monthly.
30:25 When you sign up for a free trial, they ask a couple of questions about what you're interested
30:29 in as well as your background knowledge.
30:31 Then you're presented with a cool learning path to get you started right where you should be.
30:35 Personally, I'm going back to some science foundations.
30:38 I love chemistry and physics, but haven't touched them for 20 years.
30:41 So I'm looking forward to playing with PV equals NRT, you know, the ideal gas law, and all
30:48 the other foundations of our world.
30:50 With Brilliant, you'll get hands-on on a whole universe of concepts in math, science, computer
30:55 science, and solve fun problems while growing your critical thinking skills.
30:59 Of course, you could just visit brilliant.org directly.
31:02 Its URL is right there in the name, isn't it?
31:04 But please use our link because you'll get something extra, 20% off an annual premium subscription.
31:09 So sign up today at talkpython.fm/brilliant and start a seven-day free trial.
31:15 That's talkpython.fm/brilliant.
31:17 The link is in your podcast player show notes.
31:19 Thank you to brilliant.org for supporting the show.
31:21 You know, one of the areas where this comes back, where I think maybe things might have
31:28 been different if the people were able to see the future 30 years in advance, which I'm not
31:32 expecting they should be able to, would be the global interpreter lock itself with its global
31:37 description right there.
31:39 It's first G, you know?
31:41 You know, Eric Snow did a lot of work to try to create a per sub-interpreter lock.
31:47 So it's still a global lock, but global for that sub-interpreter and try to share those
31:51 within processes.
31:52 And I recall I talked about just reworking hundreds or thousands of global variables so that they
31:58 were no longer globally shared, but were more local.
32:01 And it's this kind of stuff that can grow and really makes it hard to go forward and make
32:06 changes.
32:07 Yeah.
32:07 Interesting.
32:08 I think you have had an episode on the GIL and then the refactoring, right?
32:12 Yes.
32:13 A couple.
32:14 I have not had Sam Gross on for his, his like true gill-less Python work, but he's welcome
32:19 anytime he wants to come.
32:21 Of course.
32:21 Cool.
32:21 All right.
32:22 Let me see.
32:22 I want to make sure I don't skip my, anything on my list.
32:25 So let me just throw this out here for you.
32:27 So global scope says, look, you should avoid having global variables.
32:30 Sometimes you'll need global variables, but they shouldn't be the default.
32:34 Is there a way you can pass this down to the thing that needs it and not just make it global
32:38 and shared and right.
32:39 So minimize what is global, I guess, is one of the things.
32:43 When I first saw that, well, Python is full of just a bunch of modules and functions and
32:49 the shared state is going to end up in these global variables.
32:51 And that seems really, oh my gosh.
32:52 But if you look at languages like Java, C#, C++, you end up with a lot of static
32:58 classes.
32:59 And just because there's a class namespace between your global variable and not, they're
33:05 the same thing.
33:06 Right.
33:06 And so modules and module level variables are very, there's something that you don't want
33:11 to have too many of, but they're not as out of whack with the rest of programming as
33:17 I think maybe people might initially see them are.
33:19 Right.
33:20 There's no real difference between a static class and a function and a global variable
33:24 and a module.
33:25 Because there's scope to that module.
33:26 Right.
33:27 So unless you bring in that module, you don't have access to it.
33:31 You mean?
33:31 Yeah, exactly.
33:32 And if, if you have a class and it has one value and if you have a module as one value,
33:37 it's really like, it's really semantics on whether they're different.
33:40 So I don't feel like putting them in, you know, some languages I'm maybe thinking like, oh,
33:44 well, we don't have that because we have classes and we put stuff in class.
33:46 Like if they're static classes, so the same thing.
33:49 Yeah.
33:49 Sometimes what confuses people though is, is having a static method or just a plain function
33:55 outside of the class.
33:56 And I've been a long time proponent of just a function, but I did see a Raymond Hattinger
34:02 talk where he showed that if you do now, if you have the static method, so you bring basically
34:08 the function in the class without it handling the instance, it does show it as part of your
34:15 API.
34:15 So if you now do a DURL on an instance of that class, that static method shows up, which
34:19 I found kind of interesting.
34:21 That is interesting.
34:22 Yeah.
34:22 I think there's a lot of places where people think I need a static class or, or something
34:26 where it could just be a module with functions and maybe a global variable or two.
34:30 But if you do have classes and I guess, and you're trying to look what's part of this class,
34:34 then making that, go ahead and bringing that into the class.
34:37 That totally makes sense to me.
34:37 I can see that.
34:38 I didn't see that presentation, but that makes sense.
34:40 Yeah.
34:40 As a form of grouping, but I'm with you with modules and functions.
34:44 It can get very far.
34:45 I would say more people error on trying to build up stuff into a bunch of classes because
34:50 they've seen that in other languages they're coming from, then they don't quite group it
34:55 right.
34:56 You know, like it's, it's more of not idiomatic code, which I guess we're going to get to as
35:00 well as some of the other stuff.
35:01 So one of the things that we kicked off this conversation with is, well, you've, you said
35:07 we were spoiled.
35:08 I take offense to that.
35:09 I think it's absolutely fair.
35:11 We are so spoiled with the tools we have these days.
35:14 And I remember pre-internet, it was hard programming.
35:16 So pre-web anyway.
35:18 So linters, tools that automatically fix things.
35:22 That's what's next.
35:23 Tell us about this.
35:24 Yeah.
35:24 This is really a no brainer in the times we live.
35:27 And actually, yeah, we're spoiled because there was a time I was just manually fixing
35:31 stuff like 8 was giving back till the Blackout of 4 Murder came along.
35:35 Bob, it was worse.
35:36 There was a time where we would argue about how it should be fixed.
35:39 And we, this would be like a conversation.
35:41 And we'd be like, no, no, no.
35:42 We put the commas here and we space it like this.
35:45 It's like, is this really how we should spend our day?
35:48 And it was time for somebody to step in and give them defaults.
35:51 Because I asked the questions.
35:53 And enter Lukas.
35:54 Yeah.
35:54 I asked the question on Twitter the other day, like single or double quotes.
35:57 And there were like 50 commands.
35:59 And there's really, yeah, people are really torn about the default style should be.
36:03 But yeah, Flake 8 to be compliant with Pep 8, which every Python developer should be.
36:08 Black for auto-formatting.
36:10 mypy to the type checks, right?
36:12 As we said before, Python doesn't enforce it.
36:15 So you need a tool for that.
36:16 mypy, hence.
36:17 Yeah.
36:18 And then there's also, Flake 8, for example, has a lot of plugins we might go into next.
36:23 But first I want to highlight, like, you want to automate as much as possible.
36:27 And a pre-commit tool is just awesome.
36:29 It's called pre-commit, right?
36:31 So it's a tool that runs before you try to make a commit.
36:34 So it's a great enforcer locally to not commit any code that's not formatted or that has style
36:40 violations in it.
36:41 Very easy to use.
36:43 Very easy to set up.
36:44 It might take a little bit of work fixing.
36:46 I mean, black auto-formatting is automatic, but maybe you need to fix some Flake errors.
36:51 But I see that as a little bit of work or a little price to pay with every commit to
36:56 avoid a lot of technical debt over time.
36:59 One of the challenges I see for a lot of these things, and linters and testing, they're both
37:03 in this category, is different people on the team who are all working on the same code base
37:08 have very different levels of commitment to, say, writing or running unit tests or formatting
37:16 your code so that it all looks nice and clean before they check it in, right?
37:20 I mean, I've had experiences where like, oh, the build is broken again.
37:23 It's like, well, why is the build broken again?
37:25 Well, someone checked it in.
37:26 Why didn't they notice?
37:27 Well, because they don't run any of the tools.
37:29 Why do they run the tools?
37:30 Because they don't really want to run the tools.
37:31 Like, okay.
37:32 But we're all in this together.
37:35 Let's see what...
37:36 And tools like the pre-commit stuff just mean it's just automatic, right?
37:40 There's not a person to be blamed for saying, well, why didn't you run it?
37:43 Or that person makes me run this tool?
37:45 Or, you know, like there's none of that weird friction.
37:48 It's just like the software said, this is what we agreed on and it's not ready.
37:52 You've got to go, you know, format this line or it'll often just do that itself, right?
37:56 Yep.
37:57 What's the saying?
37:58 If it's not automated, it is broken.
38:02 Yeah, that's a good saying.
38:02 This is as simple as putting the YAML in place.
38:05 I mean, that's a one-time thing.
38:07 Then that's committed to a version control.
38:09 So your collaborator teammate pulls that in.
38:11 You do a pre-commit install.
38:13 It installs the hook locally in your .git folder and you get to go.
38:17 Now all this stuff is enforced.
38:19 You can also do that with GitHub actions.
38:21 But I think the more you do locally at the individual developer level, the better.
38:25 You do both.
38:26 Oh, yeah.
38:26 So ideally, none of the malformed code gets checked in.
38:31 And then fixed.
38:32 And then it shows up as a git diff and, you know, things like that.
38:35 So you could have it on the server just as a safety net.
38:39 Who knows how that got checked in?
38:40 But also have it as a pre-commit hook for most cases.
38:43 Yeah, exactly.
38:43 All right.
38:44 Let's see.
38:45 Next up.
38:46 Narrow exception blocks.
38:48 Yeah, this is something I see quite a lot when I'm reviewing code.
38:53 It's these long blocks between the try and accept.
38:56 So 20, 30 lines.
38:57 And my code review command is always the same.
39:01 Are all these lines of code susceptible to this exception?
39:04 And of course, often they're not.
39:06 Right?
39:06 So narrow your exception blocks, meaning put only code in the try accept that can actually raise that exception.
39:14 And yeah.
39:14 I think it's good advice.
39:15 I think it doesn't have to be just one try accept block either for one huge function.
39:19 You know, you could have try accept do some stuff.
39:22 Try something else if you really need to.
39:24 You know, you can't disambiguate them by exception type.
39:27 But also worth pointing out, I suppose, is the accept star stuff, which is not going to be maybe Python.
39:35 Oh, that's new.
39:35 Yeah.
39:36 Yeah, that's new.
39:37 And I don't know how to search for it because it seems like the star.
39:41 Well, whatever.
39:42 I could just describe.
39:42 But it's really interesting because you can catch multiple exceptions.
39:45 And maybe this actually changes the advice a tiny bit.
39:48 You can catch multiple exceptions for something going wrong.
39:51 So if you were doing it primarily around async, but not just.
39:55 You could say, I'm going to try to talk to the database and this API.
39:59 And if they both break, you know, the accept star will let you catch both.
40:03 Add two separate executions.
40:05 The database exception and the API exception potentially.
40:08 Right.
40:08 Right.
40:09 Interesting.
40:09 Yeah.
40:10 I think that's new.
40:11 Brand new in 3.11.
40:12 Yeah.
40:13 That's a 3.11 thing.
40:14 So it'll be a while till we see that really coming along.
40:16 But, you know, one of the places that might be relevant is there's cool libraries like Tenacity,
40:20 which say, I want to call this function.
40:23 And if it fails, just wait and call it again.
40:25 And then try again and, you know, maybe back off until too much time has passed or give
40:29 it a certain number of attempts.
40:30 But if it failed, but it failed differently three times, you might want to know, well, what
40:35 are all the ways in which it failed?
40:36 Not just the last or the first.
40:38 Right.
40:38 And with the accept star, you could actually catch all of those errors, say, from like a
40:42 retry block or something.
40:43 So.
40:44 Yeah.
40:44 Interesting.
40:45 I don't know.
40:45 It doesn't make this cleaner or simpler to go with.
40:49 It only complicates it.
40:50 But still the idea of like smaller code blocks, I think good advice there.
40:54 Yeah.
40:54 And always name your exceptions.
40:55 Right.
40:55 So sometimes I see try except colon.
40:58 And then we're like, what's exception?
41:00 Because that will just catch anything.
41:03 Right.
41:03 So always be explicit in the exception.
41:05 I just think you understand how this works.
41:07 When you put that in there, there's no more errors in the code.
41:10 It just keeps running.
41:11 It used to crash.
41:12 And if I put except colon and just keep going, way more reliable.
41:17 Yeah.
41:17 What's the saying in the center of Python?
41:21 Errors should not pass silently.
41:22 Yes.
41:23 Exactly.
41:24 I remember.
41:24 Yeah.
41:25 Yeah.
41:25 I've seen some of this code.
41:27 And it's not always bad.
41:28 But it's usually.
41:29 It's usually not good.
41:31 So.
41:31 Yeah.
41:32 Yeah.
41:32 Catch the exceptions by specific type.
41:34 Handle different ones potentially differently.
41:36 And so on.
41:37 Yeah.
41:37 And maybe that kind of leads us into number six here more broadly.
41:41 That's one of the Pythonic or idiomatic things of Python.
41:44 Right.
41:44 But there's a whole bunch more.
41:45 And that's your next tip.
41:46 Yeah.
41:47 This can, of course, be a whole series of articles because there's so much what can be considered idiomatic or not.
41:53 But yeah, there's sometimes you see people reinventing the wheel and they can perfectly well use the standard library.
42:00 There's also a very rich set of built-ins.
42:03 Right.
42:04 Like all any those built-ins.
42:06 But then also style-wise, there's this concept of leaping, checking before you leap versus just do something and ask for forgiveness rather than permission.
42:16 So, for example, code that overly checks if a file exists or can be opened.
42:21 So all these conditionals.
42:22 What's often considered more Pythonic is to just try open a file, try to do something and then catch the exception.
42:29 Right.
42:30 So that's why it's called.
42:31 Right.
42:31 It's easier to ask for forgiveness than permission.
42:34 Yeah.
42:35 Yes, exactly.
42:36 As opposed to like a C language or something where you do six or seven checks to make sure everything is set up just right.
42:44 And then, you know, I'm going to check that the file is not just that it's not null and that it has a null terminating character.
42:52 And now I'm going to check that I have access to the files.
42:56 You know, you're like there's if you write, if you go and read a lot of C code, there's like check, check, check, check, check, check, check.
43:00 Do the thing.
43:01 And usually that's because there's a page fault and the program just goes, whoof, and just goes away if you touch it wrong.
43:07 Whereas Python almost always, not always, but almost always the result is here's an exception you can catch and handle.
43:13 And it tells you what went wrong instead of trying to think of all the checks.
43:17 Because there's probably a check you forgot.
43:18 Just give it a shot.
43:20 Right?
43:20 Right.
43:20 And it's also, it's kind of the positive mindset.
43:23 Like usually it works.
43:25 And if it doesn't work, we deal with the consequences.
43:27 Whereas if you do all these if checks, that's code that always runs and might not be necessary.
43:31 So it might even be faster.
43:33 Right?
43:33 For sure.
43:34 So more of the ideas that came to mind when I was reading your idiomatic code section was just one use that you did mention this a little bit, but I'll reinforce it is you said use a standard library.
43:43 And one of the things I learned from working with you is there, you know, sort of, I would try to pull another, other library, other things from IPI or something.
43:51 You're like, look, this is built in.
43:52 Why don't you just use, you know what?
43:53 You're right.
43:54 That's built in.
43:55 That seems pretty handy.
43:56 Right?
43:56 Like, for example, counter is definitely one that I use all the time because of you.
43:59 And a collection.
44:00 Yeah, exactly.
44:02 So embracing what's there instead of maybe getting another library or just trying to write that algorithm yourself because who knew it existed?
44:10 Didn't study the standard library enough.
44:12 I just came from C and this is how we do it in C.
44:14 So I'm going to try to do that here.
44:15 Right?
44:16 Like the idiomatic steps that you can take are really good there.
44:19 But also lambdas versus regular functions for like little inline things.
44:24 You mentioned some of the ones that take generators for like all and any and these other tests.
44:29 And there's just a lot of cool little language features.
44:31 Yeah.
44:32 Decorators.
44:32 I've got to use later.
44:33 Yeah.
44:34 Yeah.
44:34 Decorators.
44:34 Definitely.
44:35 I would put it in that space as well.
44:37 And the wit statement.
44:38 Context managers to automatically clean up resources.
44:41 Pretty important feature.
44:42 Mm-hmm.
44:43 Mm-hmm.
44:44 Another one that is near and dear to my heart.
44:46 You're number seven on this list.
44:48 Near to dear because I have suffered badly the consequences of choosing wrongly is choose the right data structures.
44:56 Tell us about that.
44:56 Yeah.
44:57 This is very important for performance.
44:59 Right?
44:59 So where you kind of need to understand just the fundamentals here.
45:02 Right?
45:03 Like list versus sets and dictionaries and how they perform.
45:06 For example, if you're very big collection of a million items and you would leave that as a list, yet you want to do searches or lookups.
45:13 Then how this works is they traverse them item by item.
45:16 Right?
45:16 Where if you would make that a set, it's now in big O notation, O1, because there's hashing going on.
45:22 So it can immediately find the value because they're hashed.
45:26 Right?
45:26 So that's a small, well, a pretty fundamental example because we work with lists and sets and dictionaries all the time.
45:34 But those are very important to know.
45:36 They are.
45:37 The dictionary and the set one can't emphasize that enough.
45:40 I mean, obviously, we use dictionaries so much.
45:43 But if you've got a list, but you think you might need to look up stuff in that list based on, say, if it's a list of objects, some field of that object.
45:51 If you would make that a dictionary, make the key of the field, then it's not a little bit faster.
45:56 It's unbelievably faster.
45:58 My mind just never ceases to be amazed.
46:01 Like, you know what?
46:02 It really, it went through a million items that fast, like instantly.
46:05 Right?
46:06 It's so different.
46:07 Or if you see people writing code, here's some more of the idiomatic stuff is I'm looping over a thing.
46:12 I want to build up this list of them.
46:14 But I want duplicates.
46:15 You might see if this thing is not, you know, look in the list for it, then you can add it.
46:20 Like, just don't do that.
46:21 Just make a set and just jam it in there.
46:22 And it'll reject the duplicates.
46:24 Right?
46:25 Like, there's a lot of things where this ties back to your idiomatic one as well.
46:28 Yeah.
46:28 Another one I highlight there are decks.
46:30 Right?
46:31 So when you do inserts and leads on both ends of a sequence, that's also generally slow with a list for the same reason.
46:39 Because as the search example is that it needs to remap values, right?
46:43 Massively.
46:44 And a deck is designed to do that fast on both ends.
46:48 So anytime I need to, if it's a big collection and I need to insert stuff on at the start, then a collection, a deck from the collections module is a better choice.
46:57 Yeah, absolutely.
46:58 Another thing, I guess, giving a little quick shout out here is the default dict is really nice, too.
47:02 Oh, yeah.
47:03 For removing tests and checks and initialize.
47:07 Because you don't have to check if a key is in the dictionary.
47:09 You can just assume it's there.
47:11 And, yeah, that leads.
47:13 You can ditch a whole bunch of if statements.
47:15 Yep, exactly.
47:17 It's the job of the dictionary, not your job now.
47:19 Yeah, nice.
47:20 All right.
47:20 Let me see.
47:21 Do I have any others that I want to squeeze in here?
47:23 Okay, so here's one that I wanted to sneak in before we get further beyond this is to use guarding clauses.
47:32 And so often what I'll see, especially when people are new, is writing code that looks like this.
47:37 It's like, it looks to me like a saw, a lumberjack would use or something.
47:42 It's just like zip, zip, zip, out, and then back, and then out, and then back.
47:45 And you're like, whoa.
47:46 You know, it's usually like it starts in the middle of the editor window and it goes to the right.
47:52 You're like, what is this?
47:53 And, you know, it's usually if some case and then if some other case, maybe do a little step, then if another case.
48:00 And it's like assuming that it's got to build up a bunch of these steps before it can sort of go down the happy path.
48:07 So it's going to check everything that could possibly go wrong before it tries to do that.
48:11 And I don't know how you feel about guarding clauses.
48:13 I love guarding clauses.
48:14 Yeah, you do?
48:15 So tell people about this sort of alternative.
48:17 It's so simple, but it makes such a difference.
48:20 Yeah, it goes back to the Xenophython again.
48:22 Flat is better than nested.
48:24 So nested, deeply nested code is just inherently, it's more complex.
48:29 Painful.
48:30 And it's harder to maintain.
48:32 Yeah, it's called the arrow, arrow shape.
48:35 I learned from this article that it's also called the staircase, which is nice.
48:39 But yeah, you can, if you look up our arrow shape, it explains more about it.
48:42 But yeah, it's just more complex because, yeah, it's again, that cyclomatic complexity, right?
48:48 The amount of paths you have through your code.
48:50 And if you, in this case, reverse all these statements and early return, the code is now way flatter.
48:56 It's easier to read and less complex, right?
48:59 So this code looks really reasonable.
49:02 And yeah, the other is just very hard to follow.
49:05 You don't even know.
49:07 And there's all these weird interplays of like, well, what if this is true, but that's false?
49:11 And could you actually get into that if case?
49:13 Whereas these guarding clauses just say, if this is wrong, bail out of this function.
49:18 If this other case is wrong, bail out.
49:20 Just like return false or don't do the thing or whatever.
49:22 And then eventually, once you've hit all those with very little indentation, then you have the thing you wanted to get to.
49:28 And it also kind of makes them a little bit unrelated to each other.
49:32 Yeah.
49:32 Unlike the super nested style.
49:34 Yeah, I guess, yeah, the cyclomatic complexity, right?
49:36 The amount of combinations you can have with this is less.
49:40 Hence, it's less complex.
49:41 Hence, it's easier to maintain.
49:43 And readability here is way better.
49:46 Yeah, I totally agree.
49:47 So when you see things like flat is better than nested in the Xen of Python, you're like, cool, but I got to test some stuff.
49:53 So it's getting nested, right?
49:55 Like it's, I just, there's nothing in that statement that it tells you, well, what are the mechanics that you consider to avoid that, right?
50:03 It's just, well, it's complicated.
50:04 I'm sorry.
50:05 It's going to be nested, right?
50:06 It would be better if it weren't, but it is.
50:07 Well, not actually.
50:08 Like guarding clauses almost always will lessen it, if not really, really decrease it.
50:13 Yeah, that's a good point.
50:14 Like the Xen tells you the what, but not necessarily the how.
50:18 And the guard clause is that this is a particular technique.
50:21 Another one in refactoring is the extract method refactoring.
50:24 And that's where we pull out a bunch of code into a new function, which also often leads to less nested code.
50:30 Indeed.
50:30 Is that your next one?
50:32 Refactoring?
50:32 Could be.
50:33 I think that was a bonus.
50:34 I skipped over a bunch.
50:35 That's right.
50:36 It was a bonus here at the end.
50:38 Absolutely.
50:39 So that's where you gave a shout out to the refactoring book, which also had those code smells in them.
50:44 Totally.
50:44 Yeah.
50:45 Do you still recommend people read that?
50:46 I mean, I read it with bated breath in 1999 or whenever that thing came out.
50:51 It was really an important book.
50:53 And I'm like, this has changed everything.
50:54 But, you know, that's 2025 years ago, something like that when it actually came out.
50:59 Yeah, that's a good point.
50:59 I think the tooling has improved.
51:02 But still, I think conceptually, this is still an important book.
51:04 I was reading parts of it again.
51:06 Honestly, I do like the first edition a bit better sometimes because it uses Java, where this uses JavaScript.
51:13 And I don't know, the paper quality.
51:16 I'm not a fan of Java, but I think I might actually prefer the Java or the JavaScript, to be honest.
51:22 But it's very fundamental stuff.
51:23 And, yeah, it still has, it's also nice that it's a catalog, so you don't have to read it end to end.
51:30 You can just zoom into parts that are relevant for you at that moment.
51:34 So, the collection of things.
51:36 Okay.
51:36 Very interesting.
51:37 So, I guess another one, I don't remember if it was directly in your article.
51:42 I don't think so.
51:43 It was just sort of implied throughout, was, you know, embrace refactoring.
51:48 You opened the, alluding to this at the beginning of this conversation, is I see a lot of people who get stuck thinking about, I've got to get this right.
51:56 I know if I start wrong, I'm going to hate it.
51:59 It's going to be hard to work on.
52:00 It's going to be so bad.
52:01 So, I've got to sit here and think for, like, two weeks to get started on this project so I get it right.
52:06 You know what you could also do?
52:08 Just pick one way that seems right at the time.
52:10 Spend a week on it.
52:12 If it's not working, it'll refactor to the other.
52:14 Or even throw it away and go to the other option, right?
52:17 Like, there's a lot of people kind of stuck.
52:19 And I think embracing the idea of refactoring, maybe not in the traditional by-hand effect of Martin Fowler's original book, but, you know, we have tools that do a lot of this automatically or almost automatically, you know, right-click and ask it to.
52:31 So, what are your thoughts?
52:33 I definitely think people should at least catch the zen of it, if not go through the book, because the tools will take care of it.
52:39 Yeah, you still want to know the concepts here.
52:42 The different techniques, as I already highlighted, the extract method, for example, grouping constants into enums we've mentioned before.
52:49 So, a lot of actually what we've spoken about so far can be found in this book as well, right?
52:54 And I think overall, the goal of refactoring is to keep that technical depth in check, right?
53:01 So, avoid, suffer a rut, they also say.
53:04 And, yeah, as you start to add more code and cover more scenarios and the always changing requirements, inevitably, what happens?
53:14 You kind of need to learn.
53:16 You thought it was one thing, but as you got further into it, you're like, you know, that part's actually, that's where the problem is.
53:21 I didn't think so, but now I know, because that's where it's hard, right?
53:24 Which you only learn as you go, right?
53:26 Yeah, that's a good point.
53:27 Right, but the people who are stuck, I don't think they internalize that, like, the only way to know where you're going to get stuck further down the road is to go down the road.
53:34 You can't think about it very well.
53:36 There are a couple of interesting comments here with respect to garden clauses.
53:41 Alvaro says, you could go and put assert statements at the beginning of functions to indicate some of the assumptions that could be disabled in production and give nice errors on pytest.
53:51 So, pytest, rather.
53:52 Yeah, possible.
53:54 What do you think, Bob?
53:55 Yeah, asserts are great, but, yeah, I'm happy that you highlight the caveat that they can be disabled, so you cannot always rely on them.
54:01 I definitely see a place for asserts.
54:04 And, of course, you use them in pytest all the time to assert different ways your function performs.
54:10 I use most of my asserts in pytest, yeah.
54:13 But I can definitely see a place for this scenario that Alvaro highlights.
54:18 Yeah, and then Brandon Outlawdience has a very realistic problem that people run into is, I work a lot of places where they don't give you the time to do the refactoring and make things better.
54:28 And so it just builds up until it becomes not ideal, as his second comment here implies.
54:34 That's a good point.
54:34 That is a real problem.
54:35 And I'll tell you how I solved it.
54:38 Bob, maybe we're getting short on time, but you can tell me real quick how you solved it.
54:42 For me, when my managers and folks would say, Michael, what is your estimate for how long this is going to take?
54:47 If I thought it was going to take five hours, but maybe I would need to do an hour or two of refactoring, this feature took seven hours.
54:53 Like, you want it done, it's going to take seven hours.
54:56 Because if I'm going to do it, I'm going to do it right, because I'm going to have to work on it afterwards.
54:59 And I know it's just going to get worse if I...
55:02 So I wouldn't exactly lie to them, but I would include the time to make it right, not just make it work as part of my estimate.
55:10 Yeah, maybe even double it, right?
55:12 Because there's the initial version, and there's the testing, there's the documentation, there are bug fixes, there are inevitably are requirement changes in the sprint meetings.
55:23 So it might, five hours might just be that initial draft almost, but maybe it should then be 10 hours.
55:30 Yeah, perhaps.
55:31 But whatever it is, clean code should be part of the deliverable, in my opinion.
55:36 And it should be part of the estimate.
55:38 I was saying estimate times three plus two weeks.
55:41 That's a nice one.
55:42 Exactly.
55:43 That is a good one.
55:45 Very good.
55:45 All right.
55:46 Well, we're getting short on time here.
55:48 Let me do something.
55:49 I want to do something that's kind of different, Bob, because I think this is actually...
55:53 Let me share a different thing here.
55:55 I think that this is something people are going to have to start thinking about as well as maybe an option here.
56:00 So what about asking AI assistants?
56:03 Things like Google Copilot or ChatGP.
56:06 There might be some cases where people say, here's my code.
56:09 Hey, thing, could you make it better?
56:11 So for example, let's do this to close out the show.
56:13 Okay, Bob?
56:13 Sure.
56:14 How can we write plain code in Python?
56:17 I'm going to ask ChatGP.
56:19 We have already covered this.
56:23 Are you sharing?
56:23 What else?
56:24 All right.
56:24 Oh, hold on.
56:25 I thought so.
56:26 I did.
56:26 I forgot to add a stream.
56:27 So I'm going to ask ChatGP here.
56:29 And what else does it say?
56:30 So I gave it all of your tips.
56:32 And I said, we already discussed that.
56:33 What else can we do?
56:34 Is it based in the article as well?
56:35 No, I didn't give it the article.
56:37 Just the headings.
56:38 So it says you can document your code.
56:40 Keep it simple, which it didn't understand.
56:42 Follow PEP 8.
56:43 This is an interesting one.
56:45 Exceptions are for error handling.
56:46 So anyway, I just thought it might be kind of fun to think about, you know,
56:50 could we ask these things to help us and improve our code?
56:53 I don't know if you played with any of this, but it could be fun.
56:56 I have not yet.
56:56 That's Julian's department.
56:58 He's geeking out over the tool.
57:00 But I'm high on my list to try it out because it's really mind-blowing.
57:06 What this tool is doing.
57:08 Yeah.
57:08 I had a program that I wrote.
57:11 And let's just see if I can go to, let's grab some random gist here on GitHub.
57:17 Let's see.
57:18 So you can take these things, this random code, and say, you know, things like,
57:23 here's a Python program.
57:25 How can I make it more readable?
57:28 You can give it this long thing and it'll say, you know, look, there were no doc strings.
57:33 There's some places where you had meaningful variables are missing.
57:36 Yeah.
57:36 Could have grouped the imports better.
57:38 Could add more type hints.
57:39 And then it rewrites it.
57:41 So, for example, this is my code to integrate with turnstile.
57:45 And so it's, you know, you see it putting type hints and stuff as it goes.
57:49 Typing there.
57:49 Yeah.
57:49 Yeah.
57:50 Yeah.
57:50 Yeah.
57:51 So anyway, I don't know.
57:52 I feel like this is, as much as looking back 25 years and seeing refactoring go that seminal,
57:58 you know, maybe, maybe, maybe this kind of stuff is going to have to be something that we take into consideration going forward.
58:05 So.
58:05 Yeah.
58:06 Wow.
58:06 Pretty impressive.
58:07 Yeah.
58:08 Yeah.
58:08 We will see.
58:08 We'll see.
58:09 So anyway, that was fun, but let's, let's close it out.
58:12 Yeah.
58:12 Excellent work.
58:13 I mean, I feel like we only just touched the surface, right?
58:16 This could go on and on.
58:17 JF out in the audience asked about unit testing and we barely touched on that.
58:22 There's a bunch of, bunch more areas, but I think we're out of time.
58:25 Yeah.
58:25 The sun is coming up in your world.
58:27 I can see.
58:27 Thousand pager books have been written about this.
58:31 It's, it's a whole field.
58:32 So yeah.
58:33 Yeah.
58:33 We, we mentioned a couple of important things, I think.
58:36 Yeah.
58:36 We sure did.
58:37 Well, it's certainly a lot of things that people can take and they do sound simple.
58:41 Like, well, write smaller programs and give them good names or write smaller parts of your program and give it good names.
58:46 But that it's like a never ending journey to do that right.
58:49 And to keep on it.
58:51 Right.
58:51 So there's a lot of nuance as well.
58:53 Right.
58:54 There is.
58:54 Absolutely.
58:55 All right.
58:55 Well, quick two final questions before you get out of here.
58:58 If you're going to write some clean Python code editor, what do you use these days?
59:03 Well, the coaches, the team almost had me on VS Code, which is an amazing idea.
59:09 But it was not as super fast, of course, but for me, for my workflow, it was not as fast as Vim.
59:16 So I stuck with Vim and I do everything.
59:19 It's just a setup, right?
59:20 With all the plugins and shortcuts.
59:23 And it makes me fast.
59:25 And the muscle memory.
59:26 Yeah.
59:27 Yeah.
59:27 It's one of those things where, you know, you could change to something potentially, but how long are you going to be in a degraded state of working before you gain enough experience?
59:36 And will it actually be that much better?
59:38 You know, there's a lot to it, right?
59:39 And once a tool introduces a mouse and I have the other option of only using a keyboard, then it's still the keyboard.
59:45 Yeah.
59:46 Yeah.
59:46 Yeah, for sure.
59:47 Okay.
59:47 And the notable PyPI package you've come across lately?
59:50 Yeah.
59:50 Yeah.
59:50 This is such a hard question.
59:52 I think I've mentioned requests in the past.
59:55 These days there's HTTPX because we're using a lot of code to consume APIs.
01:00:01 And now you can do that synchronously.
01:00:03 But now maybe the shout is really for Sebastian Ramirez.
01:00:06 SQL model, FastAPI, typer.
01:00:09 Apart from amazing tools, the documentation, it's so pleasant.
01:00:13 I keep going back.
01:00:14 And because I teach those tools as well, right?
01:00:16 To other developers.
01:00:17 I can just teach them from the documentation.
01:00:20 It's really good with solid code samples.
01:00:23 So FastAPI, I guess it is then.
01:00:25 Right on.
01:00:27 And all those things you named there, an important piece in the mix are Python types and Pydantic.
01:00:31 Yeah, exactly.
01:00:32 Exactly.
01:00:33 That really took the type hints to the next level.
01:00:36 It did.
01:00:37 Yeah.
01:00:37 Yeah.
01:00:38 It definitely did.
01:00:38 All right.
01:00:39 Well, final call to action.
01:00:40 People are interested in this.
01:00:42 Maybe they're interested in getting to know some of your coaching stuff better as well.
01:00:46 What do you tell them?
01:00:47 Yeah.
01:00:47 Go to pybit.es, our website.
01:00:49 Check out our PDM program.
01:00:51 And yeah, for anything, you can also join the PyBite Slack.
01:00:55 Hit me up there with any questions or comments.
01:00:57 I'm pybob, add pybob there.
01:00:59 And I think that's it.
01:01:01 Pybob?
01:01:02 Yeah.
01:01:02 That's a good name.
01:01:03 I like it.
01:01:04 It's on my handle.
01:01:05 So yeah, perfect.
01:01:06 So people can go check it out there.
01:01:09 And you've got the cool aspects of the Spanish domain suffix.
01:01:14 So pybit.es.
01:01:18 You can actually have just the word.
01:01:19 That's very cool.
01:01:20 Yeah.
01:01:20 Yeah.
01:01:21 Excellent.
01:01:21 All right.
01:01:22 Well, thank you for being on the show.
01:01:23 Thanks for writing the article.
01:01:24 And good to catch up with you as always.
01:01:26 Yeah.
01:01:26 Thanks for having me.
01:01:26 This was so much fun.
01:01:27 I'm happy to share.
01:01:28 You bet.
01:01:29 Bye.
01:01:29 Bye, everyone.
01:01:30 Bye-bye.
01:01:30 This has been another episode of Talk Python to Me.
01:01:34 Thank you to our sponsors.
01:01:36 Be sure to check out what they're offering.
01:01:38 It really helps support the show.
01:01:39 TypeI is here to take on the challenge of rapidly transforming a bare algorithm in Python into a full-fledged decision support system for end users.
01:01:48 Get started with TypeI Core and GUI for free at talkpython.fm/typeI.
01:01:53 T-A-I-P-Y.
01:01:54 Stay on top of technology and raise your value to employers or just learn something fun in STEM at brilliant.org.
01:02:02 Visit talkpython.fm/brilliant to get 20% off an annual premium subscription.
01:02:08 Want to level up your Python?
01:02:10 We have one of the largest catalogs of Python video courses over at Talk Python.
01:02:15 Our content ranges from true beginners to deeply advanced topics like memory and async.
01:02:20 And best of all, there's not a subscription in sight.
01:02:22 Check it out for yourself at training.talkpython.fm.
01:02:25 Be sure to subscribe to the show.
01:02:27 Open your favorite podcast app and search for Python.
01:02:30 We should be right at the top.
01:02:32 You can also find the iTunes feed at /itunes, the Google Play feed at /play,
01:02:36 and the direct RSS feed at /rss on talkpython.fm.
01:02:41 We're live streaming most of our recordings these days.
01:02:44 If you want to be part of the show and have your comments featured on the air,
01:02:47 be sure to subscribe to our YouTube channel at talkpython.fm/youtube.
01:02:52 This is your host, Michael Kennedy.
01:02:54 Thanks so much for listening.
01:02:55 I really appreciate it.
01:02:56 Now get out there and write some Python code.
01:03:01 I really appreciate it.
01:03:19 I really appreciate it.
01:03:19 I really appreciate it.
01:03:19 I really appreciate it.