Monitor performance issues & errors in your code
00:00 Clean code is one of those aspects of your programming career that's easy to put on the back burner, sometimes more by management than yourself.
00:08 But it's important in the short term for writing more debuggable and readable code.
00:12 And it's important in the long run for avoiding having your program take on the dreaded legacy code moniker.
00:19 We're fortunate to have Bob Bilderbos back on the show.
00:22 He's been thinking and writing about clean code and Python a lot lately.
00:26 And we'll dive into a bunch of tips that you can use right away to make your code cleaner, more fun to work with easier to read and more maintainable.
00:34 This is talk Python to me Episode 404, recorded February 12 2023.
00:44 Welcome to Talk Python to Me, a weekly podcast on Python.
00:56 This is your host, Michael Kennedy.
00:58 Follow me on Mastodon, where I'm @mkennedy, and follow the podcast using @talkpython, both on fosstodon.org.
01:05 Be careful with impersonating accounts on other instances.
01:08 There are many.
01:09 Keep up with the show and listen to over seven years of past episodes at talkpython.fm.
01:15 We've started streaming most of our episodes live on YouTube.
01:19 Subscribe to our YouTube channel over at talkpython.fm/youtube to get notified about upcoming shows and be part of that episode.
01:27 This episode of Talk Python to Me is brought to you by Taipy.
01:30 They're here to take on the challenge of rapidly transforming a bare algorithm in Python into a full-fledged decision support system.
01:36 Check them out at talkbython.fm/taipy.
01:41 And it's also brought to you by my friends over at Brilliant.
01:44 Stay on top of technology and raise your value to employers or just learn something fun in STEM at brilliant.org.
01:52 Visit talkpython.fm/brilliant to get 20% off an annual premium subscription.
01:57 Bob, welcome back to Talkpython to me.
02:00 - Thanks for having me back.
02:01 I'm excited to be here.
02:02 - Yeah, it's really great to have you back.
02:04 You've been on a couple of times.
02:06 We started, I believe our first discussion on the show way back when was around a hundred days of code.
02:12 We went on quite the a hundred days of code journey, writing a couple of really, really long courses that were well-received, but it took a long, you know, like nine months for us to write, which was amazing.
02:23 And we're back together.
02:25 It's great to catch up.
02:25 Yeah, indeed.
02:26 Yeah.
02:27 That goes way back to a hundred days of Python and two courses that came from it.
02:31 And later, I think we also did a Django episode.
02:33 Yeah.
02:34 That's right.
02:35 We did do a Django episode as well.
02:36 Good stuff.
02:37 So this time we're back to talk about code quality, writing clean code.
02:43 What are some of the tools and also what are some of the techniques, maybe even mindset people are using to help them write clean code, better code, both for yourself to make yourself happy and as well to be a better teammate.
02:55 If you're working in a team, right?
02:56 Yeah.
02:56 So this topic really excites me.
02:58 It's in my everyday work and I think there's a lot to gain from it.
03:01 So, yeah.
03:02 Excited to share today.
03:03 I am too.
03:04 but it's something I really, really care a lot about.
03:07 It's one of those topics that I think is long-lived, which is rare in software development.
03:13 It's not the latest JavaScript framework or something that's gonna last for a year or two.
03:18 These ideas are the kind of ideas that just no matter what you're doing, even if you don't do Python, if you do something else, it's very likely that these are solid foundations.
03:26 Some of the ideas will be specifically for Python, but many of them not.
03:30 - Yeah, timeless stuff.
03:31 - Yeah, those are the good ones that are worth putting your time into learning.
03:34 Now, before we jump into that topic, which is gonna be good, maybe just tell people what tripped you.
03:39 - These days, so I left Oracle a while back.
03:42 I think we spoke about that in 2020.
03:44 So I've been working on PyBytes full-time for almost three years.
03:47 - Congratulations.
03:48 I know you and I spoke about this beforehand and it was a really a big step for you.
03:53 - It was.
03:54 - And something you were absolutely looking forward to, but just walking away from a good paying job, that's scary.
04:00 - Stuff, yeah. - So congratulations.
04:02 - Yeah, thanks.
04:03 This is also around the time that as PyBytes, we pivoted to doing coaching.
04:07 So people helping people one-on-one, we created our PyBytes developer mindset program, a 12-week coaching program.
04:13 And then that really took off and we have work now with a hundred plus people.
04:17 And what I'm super excited about every day when I wake up, because helping people overcoming the tutorial paralysis, embracing imposter syndrome, helping them shipping two projects end to end is just the best thing ever.
04:30 Yeah.
04:31 to see people grow and become more confident and get better.
04:34 And yeah, it's really, it's excellent work, isn't it?
04:36 - Yeah, it's really fulfilling.
04:38 - Yeah.
04:38 So congratulations.
04:39 It's awesome to hear that you're doing that now and you can put all of your energy, not just your side hustle energy into these types of things, which I can tell you, it feels real good.
04:49 - You know how it is, right?
04:51 - I do, I do.
04:51 I've been in it for a while.
04:52 Very fortunate.
04:53 Okay, so let's start this off with looking at your article.
04:58 But before we do, Toon Army Captain has a really great way to sort of kick off this topic, I believe.
05:04 Clean Code has less or fewer WTFs per line.
05:07 How about that?
05:08 - Nice.
05:09 - So yeah, I think that's gonna be a theme here.
05:12 Let's go ahead and jump into it.
05:14 I reached out to you and said, hey, let's get together and talk about this because I knew that you were passionate about these things, but then you also put together a blog post here called Tips for Clean Code in Python.
05:26 I thought, okay, that's interesting.
05:28 Let me start flipping through here And there's just a bunch of nice ideas that resonate with me.
05:32 So what I thought we could do is maybe you could set the stage, like what the heck is clean code?
05:37 Toon Army did do a pretty good job of kind of giving us the colloquial term, but you know, maybe something a little more safe, forward, conformal.
05:44 We could go with, set the stage for, for what is clean code?
05:47 Why does it matter?
05:48 And then we could dive into some of the stuff you put into your article.
05:52 And then also you and I both threw some ideas in after like to kind of expand it as well.
05:57 So let's, let's start with what is this clean code thing?
05:59 Yeah.
06:00 So set off the stage.
06:01 So clean code is really when you write code, that's easier to maintain, easier to test, easier to update.
06:07 Because as we all know, a project usually starts simple, but over time it grows, new requirements comes in.
06:14 You constantly have to change things.
06:16 And yeah, the cleanness of your code will definitely determine how easy you can move a project forward, how you can make changes, and of course it's also contribution to our team to make it a more pleasant experience and decrease the amount of WTFs.
06:33 Yeah.
06:34 You don't want to have fewer the better, although maybe you'll even want to, and just for a good war story at a conference five years from now, but you don't want too many of those.
06:42 I think it's appreciating the fact that if you do a good job, what you're building will live a long time, right?
06:49 It will possibly be still the main focus in five years from now.
06:54 And you don't always know what that's going to be, right?
06:57 It could be, oh, here's a little side project.
06:59 It's just going to pull in some data and like, you know, you show it to somebody at work and like, you know what, that's great.
07:04 Ship it.
07:05 Like, whoa, whoa, whoa, whoa, ship it.
07:06 No, no.
07:07 I just threw it together.
07:08 It's, it's good.
07:09 Let's go.
07:10 Like we're in a hurry, you know?
07:11 And then that just starts to build and layer on like a sediment that just makes it harder and less fun to work with.
07:17 And so as you just think about this thing is small now, but as it grows, what are the, both the practices and maybe technological or programming things to bring and put in place so that as it grows, you don't start to just really come to a screeching halt to make changes.
07:34 it's test start to fail randomly as you just touch it in various places, things like that, right?
07:39 Yeah.
07:40 It was also kind of my, introduction to Python in 2012.
07:44 So almost 11 years ago when I made this automation framework at work and, I was a big fan of Perl back then.
07:52 I guess that goes back to my Unix shell scripting, initial programming exploration, and it was not maintainable at all, right.
08:00 There were no classes in Perl and it was just a mess.
08:02 Also, of course, because I didn't know Perl maybe very well, but then I discovered Python and I could refactor it, make it more modular and that project was then just a joy to work with.
08:13 And maybe that says a lot about Python as well, that it does inherently things well, because if you type import this in the REPL and you get the scent of Python, a lot of these statements sound so simple, but they're actually so profound and they actually tell you a lot about writing clean code in a sense.
08:30 So anyway, but I digress.
08:32 They do.
08:33 They also talk a lot about the community that cares enough that, you know, you have these sorts of commands in the language itself that remind you to think about writing clean code and going way back to the late eighties, you know, when he came up with it, really broke with tradition to use white space for the code structure. And I think it's a little bit less so now, but back then that was very much to encourage people to write code in a way that was very readable. Whereas, you know, I say less so than now, because you've got a lot of IDEs and stuff where you can push a button and it'll auto format it. And you've got things like black and, you know, if you had curly braces, like it could still be quickly put into a pretty shape now. But in the early days, that was a really important concept of it.
09:16 days we were spoiled, right? Having black out of formatting and all that. When we started, that was not the case. How did you react to the white space thing when you first had to use it?
09:27 At first I hated it. I just really didn't like it. I thought this is just weird. Like, okay, everything else about this language seems pretty nice, but this is weird stuff.
09:34 Then I started using editors that understood it. It's like, okay, well I just hit, you know, colon and enter and it, it kind of does it for me automatically. And sure there's three spaces, four spaces, but if I backspace, then it goes back four.
09:46 And it, it kind of treats it, the structure, the tools know the structure and it makes it really nice.
09:51 And at the time, this is long ago, I was doing C#, which is an okay language, it's pretty good, but it has all the symbols and formulas and all the angle brackets and all the stuff that you would do, right?
10:02 It's not quite C++, but it's pretty close.
10:04 And when I came to Python, I'm like, gosh, it's just weird that like a lot of that stuff's not here and it seemed to kind of out of place.
10:10 But then when I went back, I realized, wait, these languages are lying to me.
10:14 They're saying I need all these symbols to make it hold together.
10:16 And you just don't.
10:17 And it's so much nicer to look at code that is not laden with these support structures.
10:23 So a bit of a diversion, but yeah, that's, I reacted weirdly to it, but after about a week, I'm like, but it's better.
10:30 It really is.
10:31 Even though I'm so weirded out by it, it's better.
10:33 And then, you know, pretty quickly I'm like, yeah, this is sweet.
10:35 I'm going with this.
10:36 Yeah.
10:36 Same here.
10:37 Indeed.
10:38 All right.
10:39 Well, let's start, I guess, start at the top with your article.
10:42 What inspired you, by the way, to write it right now?
10:45 Cause it's pretty recent, isn't it?
10:46 Yeah, that's a good question.
10:47 It's not the first time I've been writing content about clean code and stuff.
10:53 In 2016, I actually did a formal certification building maintainable software from the SIG group, software improvements group.
11:03 They have formal certifications.
11:05 Yeah.
11:05 So it's a passion of mine.
11:07 A clean code.
11:08 it's also coming back a lot in, in the coaching I do day to day helping people.
11:12 So it's not the first content piece, but I think this one kind of summarizes in a bit more of a concise way.
11:19 And of course there's way much more to it, but I think the 10 points here get you pretty far.
11:25 So those are pretty important.
11:27 They sure are.
11:28 So let's start with one that you already kind of laid this out.
11:32 Like so many of these techniques, they are like, they sound so simple and like, Yeah, sure. Obviously, Michael, I wouldn't write a 1000 line function.
11:43 And you're like, but it's, it's already 700. What is how, how did this happen?
11:47 Right? How do we get here? And the first one solidly falls into that realm, which is smaller units, smaller functions, smaller classes, smaller modules, single responsibility principle, all that, right? Tell us about this.
11:59 Yeah. So indeed, a function or a unit of code should ideally do one thing. And And just, just a side note, right.
12:06 That usually not how we start.
12:09 Sometimes I do want to make this side note that when we are figuring out a design, it can definitely happen that you have very large units because you're basically trying to figure out what you're building.
12:19 Right.
12:20 And that's where refactoring comes in.
12:21 But we'll, we'll talk about that a bit later.
12:23 But yeah, I give a very simple example in this article, right.
12:26 Where a function parses a CSV file, builds up a result list and also prints it out.
12:31 Right now
12:32 function is doing three things and it probably works and it's fine, but you cannot really easily reuse something that's doing three things, right? So if you want to plug in this function now into something else, there's a lot of things happening that probably doesn't make it a candidate to reuse in its current shape or form. It's also harder to test because now you have to test parsing and printing and this is kind of a silly example, but just think about another thing you have in in your code that's doing multiple things that will just be harder to test because there's a lot more going on.
13:03 And yeah, as you said, right, these things sounds very intuitive and easy to grasp.
13:09 Yet, when we look at code bases, right, this happens all the time.
13:12 And, yeah, so if something's harder to test, it's, it's harder to guarantee that it works.
13:17 If it's hard to extend and when requirements change, it will be harder to update.
13:21 Yeah.
13:22 Yeah.
13:22 It's harder to maintain.
13:24 It's harder to reason about the more things that it's doing, the more state that it might be changing, it's harder to onboard new people, it's just a whole spectrum.
13:33 This portion of Talk Python to Me is brought to you by Taipy. Taipy is the next generation open source Python application builder. With Taipy, you can turn data and AI algorithms into full web apps in no time. Here's how it works. You start with a bare algorithm written in Python.
13:51 You then use Taipy's innovative toolset that enables Python developers to build interactive end-user applications quickly. There's a visual designer to develop highly interactive GUIs ready for production, and for inbound data streams, you can program against the Taipy core layer as well. Taipy core provides intelligent pipeline management, data caching, and scenario and cycle management facilities. That's it! You'll have transformed a bare algorithm into a full-fledged decision support system for end-users. Taipy is pure Python and and open source and you install with a simple pip install taipy.
14:25 For large organizations that need fine grained control and authorization around their data, there is a paid taipy enterprise edition, but the taipy core and GUI described above is completely free to use, learn more and get started by visiting talkpython.FM/taipy.
14:41 That's T A I P Y the links in your show notes.
14:44 Thank you to Taipy for sponsoring the show.
14:50 and it's write functions that are 10 lines long, not 100 lines long.
14:55 Now, I'd like to hear your thoughts on this, but my feeling about all of this stuff is, as a general rule, this is what you should do.
15:02 There will be situations where you might, you know, this is just, I can't think of something better, or it's way over the top to try to adhere to all of these rules within this small context.
15:13 I could tell you, on the Talk Python Training website, There's a couple of places where it's like 75, 80, 90 lines of code for a single function.
15:23 I was like, this is just so bad.
15:24 I just, I wish it wasn't like this, but it's just weirdly unique.
15:29 And, but there's, you know, 19,000 other lines of code that are all three lines, five lines type of thing.
15:36 Right.
15:36 So it's, for me, at least this isn't a 100% or you're doing it wrong.
15:41 It's a 98% or 95% of the time you should be doing this.
15:45 And it may be some weird case and I don't know, what do you, how do you feel about that in general on these topics?
15:50 Yeah.
15:50 As I said before, sometimes you just don't get the reside and design right from the start, so inevitably you're going to write some longer classes or functions.
15:58 But then when you break them out in smaller units, the other thing I have to say about that is it gives you an opportunity to better documents, because in this example, if you break this long function, that's doing three things out into three separate functions, parse CSV, build a list, bad name, but you get the point and print results, then all of a sudden we have three units now and every unit has a name.
16:20 Hence, when you go back after a year and look at that code, which always happens and you always wonder what the heck was I thinking?
16:27 Now you add a glance just by looking at the function numbers, you have just already a better idea.
16:33 this function, she can give doc strings as well.
16:36 So at a glance, you have just a much easier time figuring out what this was about.
16:41 Yeah.
16:41 I appreciate you actually gave a shout out to me in this first piece here, which I didn't expect, but what are my rules?
16:47 You know, and I see in this article, you referenced Martin Fowler's refactoring book, and one of the most important things that came out of that is not all the refactorings, but this idea of code smells, right.
16:56 And maybe tell people real quick about code smells.
17:00 Yeah.
17:00 That's a funny name, right?
17:02 Code smell.
17:02 Code that is smelling and it's an analogy I came up with.
17:06 But yeah, basically it's code that's not following these best practices, and it's just not maintainable, not easy to extend, going back to these principles we mentioned.
17:16 So yeah, not following these guidelines that code can be smelly, it can be as code smell.
17:22 Yeah, well, the idea is like, it's not actually not working.
17:24 It's just not nice.
17:26 You know, you kind of turn your nose up at it when you see it, but you can't say it's broken, because it is working.
17:32 That's a good point.
17:33 It smells a little bit off, right?
17:35 It's not quite broken code, but it's sour.
17:38 I don't know.
17:39 And so the reason I bring this up is one of my rules of thumb here that I love, and I think you're kind of hinting that with this little shout out that you gave me in this article is when we come across these code smells, often, especially when you're earlier in your career, often the reaction or the first thought is, you know, this part is not nice.
17:58 This part is complicated.
17:59 This part is hard to maintain or hard to understand.
18:01 So I need to put a nice comment.
18:03 I've been told comment your code, don't be a jerk, comment it.
18:05 So I'm going to put a big comment that describes why it's really bad and all that.
18:09 And in the code smells sort of world, these code comments can be seen as deodorant for the smell.
18:16 Like it doesn't remove the smell, but it kind of obscures, like it's not as bad if you saw the comment saying why the code that follows is bad.
18:24 But maybe you could just make it not need a comment.
18:28 And a lot of times, like you just said, that's, well, these five lines of code need an explanation because they're mixed in with all this.
18:34 But if they had a function that had a name, whose name was basically the comment, well, then you don't need a comment because the name is the comment and it's now small and understandable, right?
18:44 Like there's a lot of these little iterative things that go on.
18:46 Exactly.
18:47 Yeah, that's a great point.
18:48 And it's very easy to make that refactoring, just turn the comment into a function.
18:52 And we have created another unit we can extend, we can test, etc.
18:56 So yeah, it's interesting because there are kind of mixed feelings about commands.
18:59 Some people, they voracious command your code, but mostly in their literature, literature I'm reading that commands are actually not that good.
19:06 And sometimes even labeled as a code smell in the sense that your code should be kind intuitive.
19:11 I think it's going back to that, like half of the time when people are actually putting those comments, it's because there's something kind of messy and they're trying to help you get over that mess or deal with that mess rather than just fixing the mess, you know?
19:23 Yeah.
19:24 It might be an indication of a bad design.
19:25 Right.
19:26 Yes, exactly.
19:27 You might even have to go back to the drawing board.
19:28 Yeah.
19:29 But that said, I definitely see a place for comments where sometimes you just want to remind yourself and of course your team members, not what the code is.
19:37 They can read the code obviously, but why you took that decision.
19:40 What the, and there might just be some extra context that you need to give.
19:44 Yeah.
19:44 And it's also different than doc strings potentially.
19:47 Right.
19:48 Which is kind of a form of a comment, but it's, it's meant to talk about the API and not cover up some weird thing you had to do in the middle.
19:55 Exactly. Yeah. Doc strings are basically your API documentation.
19:59 Yeah, exactly.
20:01 All right. Well, good stuff.
20:03 I mean, even GitHub Copilot and those things are starting to use these comments as a way to say, "Well, I'll just make a function that kind of is named what the comment is." You know, so it's...
20:13 Even our robot overlords are going to help us by taking these comments and removing them, putting the functions there.
20:20 - Help us in a number of places, right? - Another thing...
20:22 Yeah, hopefully, fingers crossed.
20:24 We'll see.
20:24 Another thing related to this, strongly related to this, because the more things you have that are smaller units, you're probably still going to have about the same amount of codes.
20:32 You have more functions, more classes, more modules, files, and so on.
20:36 What gets tricky can be naming them, right?
20:39 So choosing good names also, I think probably is a message in here that people need to think about.
20:44 What are your thoughts on that?
20:45 Yeah, it's funny.
20:46 Again, another thing that sounds very intuitive, but it wasn't, wasn't there a saying like there are two or three.
20:52 complex problems in science.
20:53 One other one I cannot remember and naming things.
20:56 Yeah, it's, there's two problems that are hard in computer science, naming things, cache invalidation and off by one errors.
21:04 Thank you.
21:05 Yeah.
21:05 Yeah.
21:05 So naming is up there, right?
21:07 And it is, that's right.
21:08 I think it goes back again to knowing your design and what you're building.
21:12 And sometimes you don't really have that figured out.
21:14 Hence naming becomes more difficult.
21:16 But I think that's an experience thing.
21:18 You get better at it, the more code you write.
21:21 Maybe we can touch on the magic numbers there.
21:24 - Yeah, absolutely.
21:25 That's the next one is magic numbers.
21:26 And magic, the number part, I think, is could be expanded a little bit broader, even.
21:31 You know, magic static values, right?
21:34 But if you see 360, you know, think just in the code, like, well, is that like a degrees?
21:40 - Right. - What is that?
21:41 Like, probably degrees, but it might not be, I don't know.
21:44 So yeah, magic numbers, that's your next recommendation.
21:46 I'm presuming to avoid them.
21:48 - Yes.
21:49 So when you see some sort of random integer in the code, which might be an intuitive one, like 365 number of days in a year, which we might kind of guess, but yeah, any guessing in code is not good.
22:02 So it's very easy to then replace that 365 integer with a number on the score of days in a year or something like that.
22:11 Something that expresses meaning.
22:12 Yeah.
22:13 And again, this is kind of an obvious example, but if you have some magic five or six or 87 in there, the reader of this code probably doesn't know what that means.
22:21 Right?
22:21 So if you have a constant uppercase with underscores as per PEP eight, defined at the top of your module, I usually put them in the top of the module and then in the code, or you see like max underscore, blah, blah, blah, a real name.
22:34 And then immediately make sense.
22:36 Right.
22:36 And it's also a nice way of grouping those variables in one place.
22:41 I said top of the module, but if you have many, you could also have a const NumPy can also use enums to group various constants together if they logically pertain to the same group.
22:51 And again, all more readable.
22:52 Yeah.
22:53 Enums are a great recommendation because instead of saying, well, the default Sunday, so we're just going to put Sunday here, like, Oh, I said, Hey, this is weird.
23:00 Or you say the function can take a string when really it can only take seven strings, seven particular values of string, right.
23:07 You could really clearly communicate that with enums, which is, you know, communicate that with enums, which is quite nice.
23:15 - Yeah, and the use of constants, that's really a win.
23:18 - I'm 100% on board with you on constants.
23:20 All caps, maybe snake case, otherwise, you know, underscores, separate them.
23:25 But I have tooling that I work with all the time that'll say, oh no, you misnamed this variable.
23:31 Python, you know, they want you to use lowercase variables, not uppercase variables.
23:34 I'm like, but for constants, they say to use uppercase.
23:36 Why don't you know this?
23:37 It drives me crazy.
23:38 So I was thinking about sort of expanding on this a little bit.
23:41 Are you familiar with this, the typing.final, when this was added in 3.8?
23:47 - Recently discovered this, yes, super nice.
23:50 - So it's also, the idea is that it comes along and is meant to be used in this scenario in addition to what we've already discussed.
23:59 So the example in the Python docs says there's a max size, capital max, capital size.
24:05 Instead of just saying it's 9,000, you say it's colon final 9,000.
24:09 Now that means nothing to Python.
24:12 It means nothing, but it means something to type checkers, right?
24:16 So if you have like MyPY or something, and somebody tries to change the value of it later, it's going to come at least as a linting type of error, if not a full on runtime error.
24:27 So this is something people could leverage to go a little bit further.
24:30 That's super nice.
24:31 Type hinting.
24:31 As you will paraphrase it's, it's meaningless to Python, but if you now run your checker and at C somewhere in your code that you try to overwrite that variable, which is perfectly fine, right?
24:42 Because Python doesn't enforce, doesn't say anything if you now would assign a new value to max size, but, there is no constant Python.
24:50 Yeah.
24:50 My PyPI picks up on that final, Hey, it was final.
24:53 Shouldn't overwrite that.
24:54 So super powerful.
24:55 I should use this more.
24:56 Yeah.
24:57 Yeah.
24:57 And so you can say final, or I would think preferably final of int, so you say what type it actually is in addition to that.
25:04 Now related to this, related to this is, it's not exactly a magic number, but I often find magic numbers appearing in this situation.
25:13 And that has to do with, I want this thing to have maybe a default value or a value that says clearly communicates.
25:22 I haven't set a value for it, but I want it to be like in this case, a number.
25:25 It still needs to be an int.
25:26 Well, what integer are you going to put there?
25:29 Zero, zero valid value.
25:31 Do you want it to be falsy?
25:32 Like there's all, could it be negative?
25:34 Which negative values?
25:36 Then you see weird tests.
25:37 Like if it's less than zero, like why is it testing that?
25:39 Well, that means it's not set.
25:41 Like, okay, that's weird.
25:42 So another idea that I think comes in that's pretty interesting is the sentinel pattern where you come up with a particular number and sort of store that and say this number or value something.
25:54 You say, if it is this, that means it's unset or it's just a weird case or it's, It's kind of like setting it to none, but if that's not going to be a case you can use, sometimes none is, is that thing.
26:06 Right.
26:07 But not always.
26:08 Are you familiar with this?
26:09 Do you do anything like this?
26:10 I have not really used it.
26:12 So I had to read a read up on it.
26:13 I think I use non quite a bit, but I think the, I mean, that's at least better maybe done minus one is defined example at the top.
26:22 Yeah.
26:23 Yeah.
26:24 I think this even wins by being more explicit even right.
26:27 By having an object then to test against.
26:29 Yeah. So one example of a standard library is the config parser has an underscore unset as a global variable and that's its sentinel value.
26:38 BZ2 has the underscore sentinel to just call out the pattern directly there.
26:44 And so on.
26:45 My favorite one out of this actually is a variation is this null object pattern.
26:51 So, null because it comes, you know, sort of earlier in the life cycle of languages where null not none, but where a lot of times if you see if this value is none, then we can do a thing other than otherwise we can do actually work with it.
27:05 Right.
27:05 And you'll see these, these tests all over.
27:08 And so this pattern is come up with a Sentinel value that is not none, but kind of behaves in a no op way.
27:16 Right.
27:17 So you could give it a value of a person who is a Sentinel and it just, if you try to get his name, it's just going to return, I don't know, nothing or something like that.
27:24 Right.
27:25 So you can, if it makes sense in that situation, you can remove a lot of these, if thing is none, do something else, do something else just globally, which is really nice.
27:34 So you don't have to do this non-check here.
27:36 So are you saving an if else?
27:38 Yeah, you're saving, you're saving the cyclomatic complexity.
27:41 You're saving like branching at the cost of maybe allocating more stuff or, or having it possibly have this value that doesn't really mean it has a value, you know?
27:50 So you got to trade it off and figure out what it makes.
27:52 That's interesting because I think I do the is not quite a bit.
27:55 So yeah, I do too, because sometimes it's kind of hard to deal with that or you don't control it.
28:00 A lot of times, kind of going back to when I was talking before about the 95% case, and I told you about this really gnarly bit of code, that's because I'm consuming two other APIs that I have no control over.
28:11 And they're junky. They're real junkies.
28:13 So, you know, I've kind of got to like fit it together.
28:16 And you don't get a pick whether they use the null object pattern or not.
28:20 You just got to make it work, right?
28:21 Yeah, that's a good point.
28:22 So yeah, indeed.
28:23 All right.
28:24 So those are all kind of under the magic number category, I would say, even though technically some of them are not, like I said, not numbers.
28:30 This one, honestly, I'll tell you, you asked me about kind of my first experiences with Python long ago, and this one weirded me out a little bit.
28:37 Cause it's something I had spent a long time in C, C++, C#, trying to just remove and weed out and that's global scope.
28:44 Tell us about this and maybe why to watch out for it.
28:46 If we leave something in global scope, for example, we have a global variable and we're pulling that into a function with a global keyword.
28:56 And we're making changes to it.
28:58 Now this function has side effects, right?
29:00 So you have to stop that's going in the function, but it's also mutating some external objects.
29:05 Right.
29:06 And when you write it, you probably are very aware of it because you're doing it, but you're going back and all of a sudden this might bite you.
29:13 It will surprise you.
29:15 Side effects overall are bad, right?
29:17 And the global scope has that potential because it's global scope.
29:20 It's not local scope to some class or function.
29:24 Yeah, basically that, that, it can lead to surprises and we don't want surprises in code.
29:29 Yeah.
29:30 And it's not an argument you pass to a function.
29:32 It's not a return value from a function.
29:34 You might not even know that that is in play as you interact with part of your code, but somewhere out there, it's something deep down is reaching up and seeing that value.
29:42 This portion of talk Python to me is brought to you by brilliant.org.
29:47 You are a curious person who loves to learn about technology.
29:50 I know because you're listening to my show.
29:52 That's why you would also be interested in this episode's sponsor, Brilliant.org.
29:57 Brilliant.org is entertaining, engaging, and effective.
30:00 If you're like me and feel that binging yet another sitcom series is kind of missing out on life, then how about spending 30 minutes a day getting better at programming or deepening your knowledge and foundations of topics you've always wanted to learn better, like chemistry or biology over on Brilliant.
30:17 Brilliant has thousands of lessons from foundational and advanced math to data science, algorithms, neural networks, and more, with new lessons added monthly.
30:26 When you sign up for a free trial, they ask a couple of questions about what you're interested in, as well as your background knowledge.
30:31 Then you're presented with a cool learning path to get you started right where you should be.
30:36 Personally, I'm going back to some science foundations.
30:38 I love chemistry and physics, but haven't touched them for 20 years.
30:42 So I'm looking forward to playing with PV equals NRT, you know, the ideal gas law, and all the other foundations of our world.
30:50 With Brilliant, you'll get hands-on on a whole universe of concepts in math, science, computer science, and solve fun problems while growing your critical thinking skills.
30:59 Of course, you could just visit brilliant.org directly.
31:02 Its URL is right there in the name, isn't it?
31:04 But please use our link because you'll get something extra, 20% off an annual premium subscription.
31:10 So sign up today at talkbython.fm/brilliant and start a seven-day free trial.
31:15 That's talkpython.fm/brilliant.
31:17 The link is in your podcast player show notes.
31:19 Thank you to brilliant.org for supporting the show.
31:21 You know, one of the areas where this comes back, where I think maybe things might've been different if the people were able to see the future 30 years in advance, which I'm not expecting, they should be able to would be the global interpreter lock itself with its global global description right there.
31:39 It's first G, you know, you know, Eric Snow did a lot of work to try to create a per sub interpreter lock.
31:47 So it's still a global lock, but global for that sub interpreter and try to share those within processes.
31:52 And I recall, I talked about just reworking hundreds or thousands of global variables so that they were no longer globally shared, but were more local.
32:01 And it's this kind of, this is the kind of stuff that can grow and really makes it hard to go forward and make changes.
32:07 Yeah.
32:07 Interesting.
32:08 I think you have had an episode on, on the gil and then the refactoring, right?
32:12 Yes.
32:13 A couple, I have not had Sam Gross on for his true Gil-less Python work, but he's welcome anytime he wants to come, of course.
32:21 Cool.
32:22 All right, let me see. I want to make sure I don't skip anything on my list. So let me just throw this out here for you.
32:27 So Global Scope says, look, you should avoid having global variables. Sometimes you'll need global variables, but they shouldn't be the default.
32:34 Is there a way you can pass this down to the thing that needs it and not just make it global and share it?
32:39 So minimize what is global, I guess, is one of the things.
32:44 When I first started that, well, Python is full of just a bunch of modules and functions, and the shared state is going to end up in these global variables.
32:51 And that seems really, oh my gosh.
32:53 But if you look at languages like Java, C#, C++, you end up with a lot of static classes.
32:59 And just because there's a class namespace between your global variable and not, they're the same thing.
33:06 Right. And so modules and module level variables are very, there's something that you don't want to have too many of, but they're not as out of whack with the rest of programming, as I think maybe people might initially see them are, right?
33:20 There's no real difference between a static class and a function and a global variable and a module.
33:25 Because they are scoped to that module, right? So unless you bring in that module, you don't have access to it, you mean?
33:31 You mean?
33:31 Yeah, exactly. And if you have a class and it has one value and you have a module has one value, it's really semantics on whether they're different.
33:40 So I don't feel like putting them in, you know, some languages, maybe things like, "Oh, well, we don't have that because we have classes and we put stuff in classes." I get if they're static classes, so the same thing.
33:49 Yeah, sometimes what confuses people though is having a static method or just a plain function outside of the class.
33:56 And I've been a longtime proponent of just a function, But I did see a Raymond Hettinger talk where he showed that if you do now, if you have the static method, so you bring basically the function in the class without it handling the instance, it does show it as part of your API. So if you now do a DIR on an instance of that class, that static method shows up, which I found kind of interesting.
34:21 - That is interesting. Yeah. I think there's a lot of places where people think, I need a static class or something where it could just be a module with functions and maybe be a global variable or two, but if you do have classes and I guess, and you're trying to look, what's part of this class, then making that go ahead and bringing that into the class.
34:37 That totally makes sense to me.
34:37 I can see that.
34:38 I didn't see that presentation, but that makes sense.
34:40 Yeah.
34:40 As a form of grouping, but I'm with you with, modules and functions, you can get very far.
34:45 I would say more people error on trying to build up stuff into a bunch of classes because they've seen that in other languages that are coming from, then they don't quite group it right.
34:56 You know, like it's, it's more of not idiomatic code, which I guess we're going to get to as well as some of that stuff.
35:01 So one of the things that we kicked off this conversation with is worse.
35:06 Well, you, you said we were spoiled.
35:08 I take offense to that defense.
35:10 Now I, I think it's absolutely fair.
35:11 We are so spoiled with the tools we have these days.
35:14 And I remember pre-internet it was hard programming.
35:17 So pre-web anyway.
35:18 So lenders tools that automatically fix things.
35:22 That's what's next.
35:23 Tell us about that.
35:24 This is really a no brainer in the times we live.
35:27 And actually, yeah, we're spoiled because there was a time I was just manually fixing stuff like eight was giving back till the black out of four murder.
35:34 Him along.
35:35 Bob, it was worse.
35:36 There was a time where we had argue about how it should be fixed.
35:39 And we, this would be like a conversation and it would be like, no, no, no.
35:42 We put the commas here and we spaced it like this.
35:44 It's like, is this really how we should spend our day?
35:47 And it was time for somebody to step in and give them defaults because I asked the questions.
35:53 And enter Lukas.
35:54 Yeah.
35:54 I asked the question on Twitter the other day, like single or double quotes.
35:57 And there were like 50 commands and there's really, yeah, people are really torn about the default style should be.
36:03 But yeah, Flake 8 to be compliant with PEP 8, which every Python developer should be, Black for auto-formatting, mypy to the type checks, right?
36:12 As we said before, Python doesn't enforce it.
36:15 So you need a tool for that.
36:16 mypy hence.
36:18 Yeah.
36:18 And then there's also Flake 8, for example, has a lot of plugins.
36:21 we might go into next, but I, first I want to highlight, like you want to automate as much as possible and a pre-commit tool is just, just awesome.
36:29 It's, it's called pre-commit, right?
36:31 And so it's a tool that runs before you try to make a commit.
36:34 So it's a great enforcer locally to not commit any code that's not formatted or that has a style violations in it.
36:41 Very easy to use, very easy to set up.
36:44 It might take a little bit of work fixing.
36:46 I mean, black auto-formatting is automatic, but maybe you need to fix some flake errors.
36:51 But I see that as a little bit of work or a little price to pay with every commit to avoid a lot of technical debt.
36:58 Over time.
36:59 One of the challenges I see for a lot of these things and linters and testing, they're both in this category is different people on the team who all work in the same code base, have very different levels of commitment to say, writing or running unit tests or formatting your code so that it all looks nice and clean before they check it in.
37:20 Right.
37:20 I mean, I've had experiences where like, oh, the build is broken again.
37:23 It's like, well, why is the build broken again?
37:25 Well, someone checked it in.
37:26 Why didn't they notice?
37:27 Well, cause they don't run any of the tools.
37:29 Why do they run the tools?
37:30 Cause they don't really want to run the tools.
37:31 Like, okay, but we're all in this together.
37:35 Let's let's see what, and tools like the pre-commit stuff just mean is just automatic.
37:40 Right.
37:40 There's not a person to be blamed for saying, well, why didn't you run it?
37:43 Or that person makes me run this tool or, you know, like there's none of that, that, that weird friction.
37:48 And it's just like the software said, this is what we agreed on.
37:51 And it's not ready.
37:52 You've got to go, you know, format this line or, or it'll often, it'll just do that itself.
37:56 Right.
37:56 Yep.
37:57 What's the saying?
37:58 If it's not automated, it is broken.
38:00 And this is as simple as putting the, the YAML in place.
38:05 I mean, that's a one-time thing.
38:07 Then that's committed to a version control.
38:09 So your collaborator teammate pulls that in, you do a pre pre-commit install.
38:13 It installs the hook locally in your dot git folder and you get to go.
38:17 No, all this stuff is enforced.
38:19 You can also do that with github actions, but I think the more you do locally at the individual developer level, the better.
38:25 >> You do both.
38:26 >> Oh, yeah.
38:26 >> Ideally, now the malformed code gets checked in and then fixed, and then it shows up as a git diff and things like that.
38:35 So you could have it on the server just as a safety net.
38:39 Who knows how that got checked in, but also have it as a pre-commit hook for most cases.
38:43 >> Yeah, exactly.
38:43 >> All right. Let's see next up.
38:46 narrow exception blocks.
38:49 Yeah, this is something I see quite a lot when I'm reviewing code.
38:53 It's these long blocks between the try and accept.
38:56 So 20, 30 lines, and my code review command is always the same.
39:01 Are all these lines of code susceptible to this exception?
39:04 And of course, often they're not, right?
39:06 So narrow your exception blocks, meaning put only code in the try except that can actually raise that exception.
39:14 I think it's good advice.
39:16 I think it doesn't have to be just one try-except block either for one huge function.
39:20 You know, you could have try-except do some stuff, try something else if you really need to.
39:24 You can't disambiguate them by exception type.
39:27 But also worth pointing out, I suppose, is the except star stuff, which is that going to be maybe Python.
39:35 Oh, that's new, yeah.
39:36 Yeah, that's new, and I don't know how to search for it because it seems like the star...
39:41 Well, whatever, I could just describe.
39:42 But it's really interesting because you can catch multiple exceptions.
39:45 And maybe this actually changes the advice a tiny bit.
39:48 You can catch multiple exceptions for something going wrong.
39:52 So, if you were doing, primarily it's run async but not just.
39:55 So, you could say, I'm going to try to talk to the database and this API, and if they both break, the accept star will let you catch both at two separate executions, the database exception and the API exception, potentially.
40:08 Right. Interesting. Yeah, I think that's new, brand new in 3.11.
40:12 Yeah, that's a 3.11 thing. So it'll be a while till we see that really coming along.
40:16 But one of the places that might be relevant is there's cool libraries like Tenacity, which say, I want to call this function, and if it fails, just wait and call it again.
40:25 And then try again, and maybe back off until too much time has passed, or give it a certain number of attempts. But if it failed, but it failed differently three times, you might want to know, well, what are all the ways in which it failed, not just the last or the first.
40:38 Right.
40:38 And with the star, you could actually catch all of those errors, say from like a retry block or something.
40:43 So, yeah, interesting.
40:45 I don't know.
40:45 It doesn't make this cleaner or simpler to go with it.
40:49 It only complicates it, but still the idea of like smaller code blocks.
40:52 I think good advice there.
40:54 Yeah.
40:54 And always name your exceptions.
40:55 Right.
40:55 So sometimes I see try except colon, and then we're like, what's exception?
41:00 We got that.
41:01 We'll just catch anything.
41:03 Right.
41:03 So always be explicit in the exception.
41:05 I don't think you understand how this works.
41:07 When you put that in there, there's no more errors in the code.
41:10 It just keeps running. It used to crash.
41:13 And if I put except colon and just keep going, way more reliable.
41:17 Yeah.
41:18 Well, it's a saying in the Zen of Python, errors should not pass silently.
41:22 Yes, exactly. I remember.
41:25 Yeah, I've seen some of this code.
41:27 It's not always bad, but it's usually not good.
41:31 So, yeah.
41:32 Couch the exceptions by specific type, handle different ones potentially differently and so on.
41:37 And maybe that kind of leads us into number six here more broadly.
41:41 That's one of the Pythonic or idiomatic things of Python, right?
41:44 But there's a whole bunch more.
41:45 And that's your next tip.
41:46 Yeah, this can of course be a whole series of articles because there's so much what can be considered idiomatic or not.
41:53 But yeah, there's sometimes you see people reinventing the wheel and they can perfectly well use the standard library.
42:01 There's also a very rich set of built-ins, right?
42:04 like all any those built-ins, but then also style wise, there's this concept of leaping, checking before you leap versus just do something and ask, ask for forgiveness rather than a permission. So for example, code that overly checks if a file exists or can be opened.
42:21 So all these, these conditionals, what's often considered more Pythonic is to just try open a file, try to do something and then catch the exception. Right. So that's why it's called, it's easier to ask for forgiveness than permission.
42:34 Yeah.
42:35 Yes, exactly.
42:36 As opposed to like a C language or something where you do six or seven checks to make sure everything is set up just right.
42:44 And then, you know, I'm going to check that the file is not just, that it's not null and that it has a null terminating character.
42:52 Now I'm going to check that I have access to the files.
42:56 You know, you're like, there's, if you write, if you go and read a lot of C code, there's like check, check, check, check, check, check, do the thing.
43:01 And usually that's because there's a page fault and the program just goes hoof and just goes away if you touch it wrong, whereas Python almost always, not always, but almost always the result is here's an exception you can catch and handle, and it tells you what went wrong instead of trying to think of all the checks, because there's probably a check you forgot, just give it a shot.
43:20 Right.
43:20 Right.
43:20 And it's also, it's kind of the positive mindset.
43:24 Like usually it works.
43:25 And if it doesn't work, we deal with the consequences.
43:27 Whereas if you do all these, if checks that's code that always runs and might not be necessary. So it might even be faster.
43:33 Right, for sure. So more of the ideas that came to mind when I was reading your idiomatic code section was just one use that you did mention this a little bit, but I'll reinforce it is you said use a standard library. And one of the things I learned from working with you is there, you know, sort of, I would try to pull another other library, other things from PyPI or something, you're like, look, this is built in, right? Just, you know what, you're right. That seems pretty handy, right? Like, for example, counter is definitely one that I use all the time because of you.
44:00 - And a collection.
44:00 - So embrace, yeah, exactly.
44:02 So embracing what's there, instead of maybe getting another library or just trying to write that algorithm yourself, 'cause who knew it existed?
44:10 Didn't study the standard library enough.
44:12 I just came from C and this is how we do it in C, so I'm gonna try to do that here, right?
44:16 Like the enigmatic steps that you can take are really good there.
44:19 But also lambdas versus regular functions for like little inline things.
44:24 You mentioned some of the ones that take generators for like all and any and these other tests.
44:29 And there's just a lot of cool little language features.
44:31 Decorators.
44:32 Yeah.
44:33 Decorate.
44:34 Yeah.
44:34 Yeah.
44:34 Decorators.
44:34 Definitely.
44:35 I would put it in that, that space as well.
44:37 And the wit statement, context managers to automatically clean up resources, pretty important feature.
44:42 Another one that is near and dear to my heart, your number seven on this list.
44:48 And near to dear, because I have suffered badly.
44:52 The consequences of choosing wrongly is choose the right data structures.
44:56 Tell us about that.
44:56 Yeah, this is very important for performance, right?
44:59 So where you kind of need to understand just the fundamentals here, right?
45:03 Like list versus sets and dictionaries and how to perform, for example, if you're very big collection of a million items and you would leave that as a list yet you want to do searches or lookups, then how this works is they traverse them item by item.
45:16 Right.
45:16 Where if you'd make that a set, it's now in big O notation, O one, because there's hashing going on.
45:22 So it can immediately find the value because they're hashed.
45:26 Right.
45:26 So that that's an ex, a small, well, a pretty fundamental example, because we work with lists and sets and dictionaries all the time, but those are very important to know.
45:36 They are.
45:37 I, the dictionary and the set one can't emphasize that enough.
45:40 I mean, obviously we use dictionaries so much, but if you've got a list, but you think you might need to look up stuff in that list based on, say, if it's a list of objects, some field of that object, if you would make that a dictionary and make the key of the field, then it's not a little bit faster.
45:56 it's unbelievably faster.
45:58 My mind just never ceases to be amazed.
46:01 Like, you know what?
46:02 It really, it went through a million items that fast?
46:04 Like, instantly.
46:06 Right?
46:07 So different.
46:08 Or if you see people writing code, here's some more of the idiomatic stuff is, I'm looping over a thing, I want to build up this list of them, but I want duplicates.
46:15 You might see, if this thing is not, you know, look in the list for it, then you can add it.
46:20 Like, just don't do that.
46:21 Just make a set and just jam it in there and it'll reject the duplicates, right?
46:25 Like there's a lot of things where this ties back to your automatic one as well.
46:28 Yeah.
46:29 And another one I highlight there are decks, right?
46:31 So when you do inserts and leads on both ends of a sequence, that's also generally slow with a list for the same reason, because as the search example is that it's, it needs to remap her values, right?
46:44 Massively.
46:45 And a deck is designed to do that fast on both ends.
46:48 So anytime I need to, if it's a, it's a big collection and I need to insert stuff on at the start and the collection, a deck from the collections module is a better choice.
46:57 Yeah, absolutely.
46:58 Another thing, I guess, given a little quick shout out here is the default dict is really nice too.
47:03 Oh yeah.
47:04 For removing tests and checks and initialize.
47:07 Because you don't have to check if a key is in the dictionary.
47:09 You can just assume it's there and then that, yeah, that leads, you can ditch a whole bunch of if statements.
47:16 Yep, exactly.
47:17 It's the job of the dictionary, not your job now.
47:19 Yeah.
47:20 Nice.
47:21 All right.
47:22 Let me see.
47:23 So here's one that I wanted to sneak in before we get further beyond this, is to use guarding clauses.
47:32 And so often what I'll see, especially when people are new, is writing code that looks like this.
47:37 It's like, it looks to me like a saw, that a lumberjack would use or something, and it's just like, zip, zip, zip, out, and then back, and then out, and then back.
47:45 And you're like, whoa.
47:47 You know, it's usually like, it starts in the middle of the editor window and it goes to the right.
47:52 What is this?
47:53 And it's usually if some case, and then if some other case, maybe do a little step, then if another case.
48:00 And it's like assuming that it's got to build up a bunch of these steps before it can sort of go down the happy path.
48:07 So it's gonna check everything that could possibly go wrong before it tries to do that.
48:11 And I don't know how you feel about guarding clauses.
48:13 I love guarding clauses.
48:15 Yeah, you do?
48:16 So tell people about this sort of alternative.
48:18 It's so simple, but it makes such a difference.
48:20 Yeah, it goes back to the Zen of Python again, flat is better than nested.
48:24 So nested, deeply nested code is just inherently it's more complex, painful and it's harder to maintain.
48:32 Yeah.
48:32 So it's called the arrow arrow shape.
48:35 I learned from this article that it's also called the staircase, which is nice, but yeah, you can, if you look up our arrow shape, that explains more about it.
48:42 But yeah, it's, it's just more complex because yeah, it's, it's again, that cyclomatic complexity, right.
48:48 done, the amount of paths you have through your code.
48:50 And if you, in this case, reverse all these statements and early return, the code is now way flatter.
48:56 It's easier to read and less complex.
48:59 Right.
48:59 So this, this code looks really reasonable and, yeah, the other is, is just very hard to follow.
49:05 You don't even know.
49:07 And there's all these weird interplays of like, well, what if this is true, but that's false and could you actually get into that if case, whereas these guarding clauses just say, if this is wrong, bail out of this function.
49:18 If this other case is wrong, bail out.
49:20 We're just like return false or don't do the thing or whatever.
49:22 And then eventually once you've hit all those with very little indentation, then you have the thing you wanted to get to.
49:28 And that also kind of makes them a little bit unrelated to each other.
49:32 Unlike the, the super nested style.
49:34 Yeah, I guess.
49:34 Yeah.
49:35 The, the psychromatic complexity, right.
49:36 The amount of combinations you can have with this is less hands.
49:40 It's less complex hands.
49:41 It's, it's easier to maintain.
49:43 - And readability here is way better.
49:46 - Yeah, I totally agree.
49:47 So when you see things like flat is better than nested in the Zen of Python, you're like, cool, but I got to test some stuff, so it's getting nested.
49:55 Right, I guess, I just, there's nothing in that statement that it tells you, what are the mechanics that you consider to avoid that?
50:03 Right, it's just, well, it's complicated.
50:04 I'm sorry, it's going to be nested, right?
50:06 It would be better if it weren't, but it is.
50:08 Well, not actually, like guarding clauses almost always will lessen it, if not really, really decrease it.
50:13 Yeah, that's a good point.
50:14 Like this N tells you the what, but not necessarily the how and the guard clauses that this is a particular technique.
50:21 Another one in refactoring is the extract method refactoring.
50:24 And that's where we pull out a bunch of code into a new function, which also also often leads to less nested code.
50:30 Indeed.
50:31 Is that your next one?
50:32 Refactoring?
50:32 Could be.
50:33 I think that was a bonus.
50:34 It was a, that's right.
50:36 It was a bonus here at the end.
50:38 Absolutely.
50:38 So that's where you gave a shout out to the refactoring book, which also had those could, could smells in them.
50:44 Totally.
50:44 Yeah.
50:45 Do you still recommend people read that?
50:46 I mean, I, I read it with bated breath in 1999 or whenever that thing came out.
50:51 It was really an important book and I'm like, this has changed everything, but you know, that's 20, 25 years ago, something like that when it actually came out.
50:58 Yeah, that's a good point.
51:00 I think the tooling has improved, but still, I think conceptually, this is still an important book.
51:05 I was reading parts of it again.
51:06 Honestly, I do like the, the first edition a bit better sometimes because it uses Java where this uses JavaScript and I don't know the paper quality.
51:15 I'm not a fan of Java, but I think I would, I might actually prefer the Java or the JavaScript to be honest.
51:22 Yeah, but it's, it's very fundamental stuff.
51:24 And yeah, it still has, it's also nice.
51:28 That's a catalog, so you don't have to read it end to end.
51:31 You can just zoom into parts that are relevant for you at that moment.
51:34 So yeah, the collection of things.
51:36 Okay.
51:37 Very interesting.
51:38 Another one, I don't remember if it was directly in your article.
51:43 I don't think so.
51:43 It was just sort of implied throughout was, you know, embrace refactoring.
51:48 You opened the, with alluding to this at the beginning of this conversation is I see a lot of people who get stuck thinking about, I've got to get this right.
51:56 I know if I start wrong, I'm going to hate it.
51:59 It's going to be hard to work on.
52:00 It's going to be so bad.
52:01 So I got to sit here and think for like two weeks to get started on this project.
52:06 So I get it right.
52:07 You know what you could also do?
52:08 just pick one way that seems right at the time, spend a week on it, if it's not working, it'll refactor to the other, or even throw it away and go the other option.
52:17 Right, like there's a lot of people kind of stuck, and I think embracing the idea of refactoring, maybe not in the traditional by-hand effect of Martin Fowler's original book, but we have tools that do a lot of this automatically, or almost automatically, you know, right-click and ask it to.
52:32 So, what are your thoughts?
52:33 I definitely think people should at least catch the zen of it, if not go through the book, Cause the tools will take care of it.
52:39 Yeah.
52:40 You still want to know the concepts here.
52:42 The different techniques as I already highlighted the extract method, for example, grouping constants into enums we've mentioned before.
52:49 So a lot of actually what we've spoken about so far can be found in this book as well.
52:54 Right.
52:54 And I think overall the, the goal of refactoring is to keep that technical depth in check.
53:01 Right.
53:01 So avoid suffer rot.
53:03 They also say, and, yeah, as you start to add more code and.
53:08 cover more scenarios and the always changing requirements, inevitably what happens, you kind of need to learn.
53:16 You thought it was one thing, but as you got further into it, you're like, you know, that part is actually the, that's where the problem is.
53:21 I didn't think so, but now I know because that's where it's hard, right?
53:24 You wait, you only learn as you go, right?
53:26 Yeah, that's a good point.
53:27 Right.
53:27 But the people who are stuck, I don't think they internalize that.
53:30 Like the only way to know where you're going to get stuck further down the road is to go down the road.
53:34 You can't think about it very well.
53:36 There are a couple of interesting comments here.
53:39 With respect to guarding clauses, Alvaro says, you could go and put assert statements at the beginning of functions to indicate some of the assumptions that could be disabled in production and give nice errors on pytest.
53:53 Yeah, possible. What do you think, Bob?
53:55 Yeah, asserts are great, but yeah, I'm happy that you highlight the caveat that they can be disabled, so you cannot always rely on them.
54:01 I definitely see a place for asserts.
54:04 And of course, you use them in pytest all the time to assert different ways your function performs.
54:10 Yeah, I use most my asserts in pytest.
54:12 Yeah, but I can definitely see a place for the scenario that Alvaro highlights.
54:18 Yeah, and then Brendan at Lottie has a very realistic problem that people run into is, I worked a lot of places where they don't give you the time to do the refactoring and make things better. And so it just builds up until it becomes not ideal, as his second comment here implies.
54:34 - That's a good point.
54:34 - That is a real problem.
54:35 And I'll tell you how I solved it.
54:38 Bob, maybe we're getting short on time, but you can tell me real quick how you solved it.
54:42 For me, when my managers and folks would say, "Michael, what is your estimate for how long this is gonna take?" If I thought it was gonna take five hours, but maybe I would need to do an hour or two of refactoring, this feature took seven hours.
54:53 Like, you want it done, it's gonna take seven hours.
54:55 Because if I'm gonna do it, I'm gonna do it right, 'cause I'm gonna have to work on it afterwards.
54:59 And I know it's just gonna get worse if I...
55:02 So I wouldn't exactly lie to them, but I would, I would include the time to make it right.
55:07 Not just make it work as part of my estimate.
55:10 Yeah.
55:11 Maybe even double it.
55:12 Right.
55:13 Because there's the initial version and there's the testing, there's the documentation, there are bug fixes, there are inevitably are requirement changes in the sprint meetings.
55:24 So it might five hours might just be that initial draft almost, but maybe it should then be 10 hours.
55:30 Yeah.
55:31 But whatever it is, clean code should be part of the deliverable, in my opinion.
55:36 And it should be part of the estimate.
55:38 I love what you're saying.
55:38 Estimate times three plus two weeks.
55:41 That's a nice one.
55:42 Exactly.
55:43 That is a good one.
55:45 Very good.
55:45 All right.
55:46 Well, we're getting short on time here.
55:48 I, let me do some, I want to do something that's kind of different, Bob, because I think this is actually, let me, let me share a different thing.
55:54 I think that this is something people are going to have to start thinking about as well as maybe an option here.
56:00 So what about asking AI assistants, things like Google copilot or chat GP?
56:06 There might be some cases where people say, here's my code.
56:09 Hey thing, could you make it better?
56:11 So for example, let's, let's do this to close out the show.
56:13 Okay, Bob.
56:13 Sure.
56:14 How can we write clean code in Python?
56:17 I'm going to ask chat GP.
56:19 We have already, I've already covered this.
56:23 Are you sharing?
56:24 All right.
56:24 Oh, hold on.
56:25 I thought so.
56:26 I did.
56:26 I forgot to add a stream.
56:27 So I'm going to ask ChatGPT here.
56:29 And what else does it say?
56:30 So I gave it all of your tips and I said, we already discussed that.
56:33 What else can we do?
56:34 Does it fit based in the article as well?
56:35 No, I didn't give it the article, just, just the type, just the headings.
56:39 So it says you can document your code, keep it simple, which it didn't understand follow pep eight.
56:43 This is an interesting one.
56:45 Exceptions are for error handling.
56:46 So anyway, I just thought it might be kind of fun to think about, you know, can we ask these things to help us and improve our code?
56:53 Well, I don't know if you played with any of this, but it could be fun.
56:56 I have not yet.
56:57 That's Julian's department.
56:58 He's, he's a geek geeking out over the tool, but I'm, having on my high on my list to try it out because it's, it's, it's a really mind blowing what, what this tool is doing.
57:08 Yeah.
57:08 I had a program that I wrote and, let's just see if I can go to, I'll just go up some, grab some random just here on GitHub, let's see.
57:18 So you can take these things, this random code and say, you know, things like, here's a Python program.
57:25 How can I make it more readable?
57:29 You can give it this long thing, and it'll say, look, there are no doc strings.
57:33 There's some places where you had meaningful variables are missing, could have grouped the imports better, could add more type hints, and then it rewrites it.
57:41 So for example, this is my code to integrate with turnstile, and so you see it putting type hints and stuff as it goes. - So I think there, yeah.
57:50 - Yeah, yeah, yeah.
57:51 So anyway, I don't know.
57:52 I feel like this is as much as looking back 25 years and seeing refactoring go, that's seminal, you know, maybe, this kind of stuff is going to have to be something that we take into consideration going forward.
58:05 So, yeah.
58:06 Wow.
58:06 Pretty impressive.
58:07 Yeah.
58:08 We will see.
58:08 We'll see.
58:09 So anyway, that was fun, but let's, let's close it out.
58:12 Yeah.
58:12 Excellent work.
58:13 I mean, I feel like we only just touched the surface, right?
58:16 This could go on and on.
58:17 JF out in the audience asked about unit testing and we barely touched on that.
58:22 There's a bunch of, a bunch more areas, but I think, I think we're out of time.
58:25 Yeah.
58:25 The sun is coming up in your world.
58:27 I can see.
58:27 Thousand page or books have been written about this.
58:31 It's a whole field.
58:32 So, yeah, yeah.
58:33 We, we mentioned a couple of important things, I think.
58:36 Yeah, we sure did.
58:37 Well, it's certainly a lot of things that people can take and they do sound simple.
58:41 Like, well, write smaller programs and give them good names or write smaller parts of your program and give it good names.
58:46 But that it's like a never ending journey to do that right.
58:49 And to keep on it.
58:51 Right.
58:51 So there's a lot to nuance as well.
58:53 Right.
58:54 There is absolutely.
58:55 All right.
58:55 Well, quick, two final questions before you get out of here.
58:58 If you're going to write some clean Python code editor, would you use these days?
59:03 Well, the coaches, the team almost had me on VS Code, which is an amazing ID, but it was not as that super fast, of course, but it was for me or my workflow.
59:15 It was not as fast as VIM.
59:16 So I stuck with vim and I do everything.
59:19 It's just the setup, right.
59:21 With all the plugins and shortcuts and it makes me fast.
59:25 And then muscle memory.
59:26 Yeah.
59:27 It's one of those things where, you know, you could change to something potentially, but how long are you going to be in a degraded state of working before you gain enough experience and will it actually be that much better?
59:38 You know, it's, there's a lot to it.
59:39 Right.
59:39 And once a tool introduces a mouse and I have the other option of only using a keyboard, then it's still the key word.
59:45 Yeah.
59:46 Yeah.
59:46 Yeah, for sure.
59:47 Okay.
59:47 And then notable PyPI package come across lately.
59:50 Yeah, this is such a hard question.
59:53 I think I've mentioned requests in the past, these days there's HTTPX because we're using a lot of code to consume APIs and now you can do that to synchronously, but now maybe the shout is, is really for, Sebastian Ramirez, SQL model, FastAPI typer.
01:00:09 Apart from amazing tools, the documentation, it's so pleasant.
01:00:13 I keep going back and because I teach those tools as well, right.
01:00:16 To other developers, I can just teach them from the documentation.
01:00:20 That's it's really good with a solid code samples.
01:00:23 So FastAPI, I guess it is then.
01:00:25 Right on.
01:00:27 And all those things you named there, an important piece in the mix are Python types and Pydantic.
01:00:31 So yeah, exactly.
01:00:32 Exactly.
01:00:33 That, that, that really took the type hints to the next level.
01:00:36 It did.
01:00:37 Yeah.
01:00:37 Yeah, it definitely did.
01:00:38 All right.
01:00:39 Well, final call to action.
01:00:40 People are interested in this.
01:00:43 Maybe they're interested in getting to know some of your coaching stuff better as well.
01:00:46 What do you tell them?
01:00:47 Yeah.
01:00:47 Go to pybit.es our website, check out our PDM program and yeah, for anything, you can also join the pybyte slack hit me up there with any questions or commands on py Bob at py Bob there.
01:00:59 And, I think that's it.
01:01:01 Py Bob.
01:01:02 Yeah, that's good.
01:01:03 That's a good name.
01:01:03 I like it.
01:01:04 So my handle.
01:01:05 So yeah, perfect.
01:01:06 So people can, go check it out there and you've got the, you've got the cool aspect of the Spanish domain suffix. So, pybit.es. You can actually have just the word. That's very cool. Excellent. All right. Well, thank you for being on the show. Thanks for writing the article and good to catch up with you.
01:01:26 Yeah. Thanks for having me. This was so much fun. I'm happy to share.
01:01:29 You bet. Bye. Bye, everyone.
01:01:32 This has been another episode of Talk Python to Me. Thank you to our sponsors. Be sure to check out what they're offering. It really helps support the show. Taipy is here to to take on the challenge of rapidly transforming a bare algorithm in Python into a full-fledged decision support system for end users.
01:01:48 Get started with Taipy Core and GUI for free at talkpython.fm/taipy, T-A-I-P-Y.
01:01:55 Stay on top of technology and raise your value to employers or just learn something fun in STEM at brilliant.org.
01:02:03 Visit talkpython.fm/brilliant to get 20% off an annual premium subscription.
01:02:10 Want to level up your Python?
01:02:11 We have one of the largest catalogs of Python video courses over at Talk Python.
01:02:15 Our content ranges from true beginners to deeply advanced topics like memory and async.
01:02:20 And best of all, there's not a subscription in sight.
01:02:23 Check it out for yourself at training.talkpython.fm.
01:02:26 Be sure to subscribe to the show, open your favorite podcast app, and search for Python.
01:02:30 We should be right at the top.
01:02:32 You can also find the iTunes feed at /iTunes, the Google Play feed at /play, and the Direct rss feed at /rss on talkpython.fm.
01:02:42 We're live streaming most of our recordings these days.
01:02:44 If you want to be part of the show and have your comments featured on the air, be sure to subscribe to our YouTube channel at talkpython.fm/youtube.
01:02:53 This is your host, Michael Kennedy.
01:02:54 Thanks so much for listening.
01:02:55 I really appreciate it.
01:02:56 Now get out there and write some Python code.
01:02:59 (upbeat music)
01:03:01 [Music]
01:03:16 (upbeat music)
01:03:19 [BLANK_AUDIO]