#246: Practices of the Python Pro Transcript

00:00 Michael Kennedy: When can you call yourself a professional developer? Sure, getting paid to write code is probably part of the formula, but when is your skillset up to that level? Many folks in the industry suffer from imposter syndrome or other types of uncertainty. Yet, there are real techniques and skills you should know before you meet this bar. Dane Hillard is here to share his take on the practices of the Python pro. We'll discuss concrete steps as well as abstract design concepts to help your code make the jump to pro level. This is Talk Python To Me, Episode 246, recorded December 5th, 2019. Welcome to Talk Python To Me a weekly podcast on Python, the language, the libraries, the ecosystem, and the personalities. This is your host, Michael Kennedy. Follow me on Twitter, where I'm @mkennedy. Keep up with the show and listen to past episodes at talkpython.fm, and follow the show on Twitter via @talkpython. This episode is brought to you by Tidelift and Linode. Please check out what they're offering during their segments. It really helps support the show. Dane, welcome to Talk Python To Me.

01:13 Dane Hillard: Hey, thanks.

01:13 Michael Kennedy: It's great to have you here. And I'm really excited about this topic, actually. I think you've got something good going here. It'd be fun to talk about.

01:19 Dane Hillard: Yeah, thanks a lot, thanks for having me.

01:20 Michael Kennedy: Yeah, we're going to learn what a pro means. I mean, there's a lot of debate about titles and when are you no longer a beginner, or you're actually a professional programmer and all these things, so it's going to be a lot of fun to dig into, but before we get to that, let's start with your story. How'd you get into programming and Python?

01:35 Dane Hillard: For programming in general, I followed a fairly traditional path. I went to school for computer engineering, mostly learning C++ at the time, MATLAB, things like that. And then, computer engineering is kind of this mix of hardware and software disciplines, which I've mostly not made use of since school, so, I've stuck very much to the software side. But since I got out of school, I've been doing everything from Perl and PHP and C++ and C to Python and JavaScript as well.

02:09 Michael Kennedy: That sounds fun. The whole hardware side of computers, it's kind of a black box to me, and it's just something that I've never really learned. Obviously, I understand what RAM is, and a disk is, and bus is, and all that, but I couldn't build one or design one or even understand what the little parts mean when I take it apart other than this goes in the PCI slot. So, that's probably fun to know that level, right?

02:33 Dane Hillard: It is, and I've admittedly forgotten more than I would like, but I think, I still try to keep with it for things like Raspberry Pi and Arduino and things like that from time to time, so it's sort of fun to keep as a hobby.

02:47 Michael Kennedy: Yeah, it's a bit of a renaissance for that, right? I mean, it used to be you had to be Dell or somebody like this to make these things, and now, you could spend $5 and get a little microchip.

02:55 Dane Hillard: Yeah, absolutely. It's kind of cool. It's an empowering time in that regard.

02:59 Michael Kennedy: Yeah, absolutely. So, you got out of school and you're working with all these different languages. How'd you come to Python?

03:04 Dane Hillard: I want to say that there was a short precursor to my time with Python that was learning Ruby. And that was kind of at the behest of a friend of mine who was doing a hack-a-thon. He wanted to build some, I think like a domain-specific language of some kind on top of Ruby for something, and I had never used a dynamically typed language or anything like that, and Ruby was a refreshing experience from what I knew at the time. And I think around that time, I was also looking for a new language to write my personal website in. The current iteration was in PHP at the time. And I had tried Java Spring a bit and didn't like that too much. So I decided to try Django, and I think Django was my first major foray into Python 'cause I remember learning a lot of Python stuff at the same time I was learning Django.

03:53 Michael Kennedy: Right, right, right, that's cool. Well, I think a lot of people come into Python because of Django, right?

03:57 Dane Hillard: Yeah, I would agree.

03:59 Michael Kennedy: I want to do something on the Web. I heard Django is easy and cool. Let's try that. Oh, I guess I have to learn Python. Well, that wasn't too hard. It's kind of an easy language to learn. But that also touches a little bit on the subject of what we're going to cover. Some people say, "Oh, you can learn Python in a weekend. It's so easy." And yet, just weekly, I am learning new things on Python, and I spend all day diving into it, and looking at the edge cases, and you can't learn Python in a weekend, but you can sort of get it together. And I think there's a really interesting spectrum there from "Hey, I made this website and I kind of know the language." to "I'm really good at this language and the ecosystem." and so on. It's broader than it first seems, or harder than it first seems.

04:39 Dane Hillard: Yeah, absolutely. And I think there's plenty of people out there, too, who don't want or need to get to that depth for a given language, especially people who are polyglot, if you will, and need to know a bevy of languages. Sometimes just knowing enough of that language to solve the problems in those various domains is totally, totally good. So yeah, there's such a spectrum of situations and experiences there.

05:05 Michael Kennedy: Yeah, for sure. And then there's the folks who are doing something else, economics, biology, and they just need enough to get through their biology problem or to run that economic simulation. And they don't care about the nuances of generators. They just wanted to know whether they can get Pandas to spit out the answer they need or Matplotlib to show the graph that they need so they can carry on.

05:28 Dane Hillard: Yeah, exactly.

05:30 Michael Kennedy: For sure, alright, so, before we get to the main topic, maybe just tell us what you do day to day so people know where you're coming from.

05:35 Dane Hillard: Yeah, so, I'm still very much into the web side of things these days. I'm working on JSTOR.org. It's an academic research platform. Kind of a journal database. And been there for about five years now. Before that, I was actually in biology to some degree. I was doing some bioinformatics work, and I did do a little bit of Python scripting there, but a lot of database administration and things like that also. And before that, I was doing some radar telemetry, so.

06:06 Michael Kennedy: It's quite a spectrum of stuff. Yeah, I feel like bioinformatics would actually be really interesting.

06:11 Dane Hillard: Yeah, it really was. And I think to me, that's often like the joy of programming is that you're both solving problems with code directly, but also picking up these second-hand areas of knowledge about some different subject, so, people have a chance to explore beyond just knowing how to write code.

06:32 Michael Kennedy: Right, if you get bored of writing bank software, you can go write renewable energy software.

06:38 Dane Hillard: Exactly.

06:38 Michael Kennedy: Yeah, that's actually pretty awesome because unlike many jobs, developers are in demand across all the industries. Basically, there's this quote by Satya Nadella that every company is a software company these days. And I kind of feel like that's right.

06:53 Dane Hillard: Yup, that resonates with me, too. I think it's, you know, every company has some area, whether it's, I don't know, it ranges from mundane to interesting, but every company has this area where they can use software to augment whatever it is that they're doing. Whether it's to do directly what they need for their business, or if it's just to make their business better in some way, faster, more efficient.

07:16 Michael Kennedy: If they're thinking, "Well, we need to hire 10 people to manually do this thing because it's really slow and tedious." well, maybe people shouldn't be doing that thing. Maybe they should be overseeing the machines doing that thing or something to that effect. Yeah so, let's get into this topic. We're going to talk about your book, Practices of the Python Pro. Let's just start with that last word there. Like what makes a developer a pro? You've had a lot of experiences, a lot of different types of companies, what do you think? What's the indicator for you?

07:45 Dane Hillard: Yeah so, the title has almost got this double entendre to it. It gets at this idea that there are practices that can make you a more effective developer. There are things you can think about as you're developing code to improve the outcomes that you get. And that is stuff that, over time, sometimes decades, whole careers, you can build and learn and never run out of stuff to start learning. But at the same time, I think a lot of these things are things anybody writing code could start to pick up bits and pieces of right away. And so in that regard, like everyone who's paid to something is a professional, right?

08:28 Michael Kennedy: Right, right, right.

08:28 Dane Hillard: So like, if that's you, congrats, you've made it. And so, I think some of what the book is about also is just like, hey, if you're new and you just want to know what people in the industry are thinking about, maybe you haven't done software professionally before, maybe you've used it in your biology work or something like that, or you're switching fields altogether, it's kind of a way to get some insight into the kinds of things you might want to be thinking about.

08:57 Michael Kennedy: Yeah, absolutely, and I like the title to like how it sort of puts that out there is these are the things that you would expect somebody who really cares about their craft, either 'cause they love it or because they're paid enough that they're supposed to pay attention to that, whatever reason, these are the techniques and things that they do. You probably are not doing all of them. Probably you've come from some other industry or you're new to the industry or maybe even you've been a professional C++ developer for 20 years, and all of a sudden, somebody told you compilers are out, and you're going to go write this funny language with no semicolons, what do these people do in their pro side of things, right? There's experts here. I think there's a lot of to take away for a lot of folks. But yeah, this idea of just what is a pro, or maybe more broadly, so we're not hung up on the money side of it, just like an expert in a language or in Python. It's an interesting question of when you can call yourself a developer, when you get to call yourself a professional developer or something like that. I don't really know where that is. We can think a little bit more about that, but I do know that beginners get frustrated really easily when they're trying to work, and I'm thinking more beginning programmers, not beginner just in Python. They've got to solve some problem. They're like, "Oh my gosh, I just heard I have to use a database for this. I don't even know what a database is, not really, so what do I do now?" And it can be really frustrating. So I don't know, what are you thoughts for folks who are not at this level? But maybe things like this book would help them get there. What would you tell them?

10:26 Dane Hillard: Yeah, I think that a lot of the source of frustration as a developer, or as a beginning developer, is that sometimes, not only do you not know the answer, but sometimes, you don't even know the question. So you're often kind of trying to think about what it is you even are trying to know, whether it's because an error you saw is too obscure, or you just can't quite ask the right question. And so, like my general advice outside the book or sort of tangential to the book is to start figuring out how to search better, like for your problems.

11:01 Michael Kennedy: Yes!

11:01 Dane Hillard: So...

11:02 Michael Kennedy: Now, hold on, let me stop you for a second because I agree with that, but I feel like a lot of people who are new probably perceive searching for a fix as giving up or I failed to get this understanding correctly. But I think, and I don't mean this in a negative way, but I think that there's this whole sort of hidden realm that a lot of people don't see where it's Googling like crazy. As a professional software developer, you're like, okay, I got to use this library. I don't know anything about it. Let me take a shot. Nope, that didn't work. That's take that area, research the heck out of that. Oh no, I'm over here. Oh wait, this is this other thing that seems better. There's a lot of searching and really soon, at least in my world, when I'm like, if this is not working and I don't have an answer in two minutes, I'm on the Internet trying to find it because often, that works.

11:48 Dane Hillard: Yeah, I think that's a really great thing to dispel because it doesn't go away. And perhaps, perhaps only increases as your breadth of knowledge grows, or at least breadth of awareness of what's out there. And your search skills grow, you have a bigger surface area to explore and you have to figure out which piece to hone in on to maybe find your problem. And so, it's hard to say just go learn how to search better, I guess, but it's...

12:16 Michael Kennedy: Yeah, that's one of the elements, but I think it's part of it.

12:20 Dane Hillard: Yeah, and there's also that danger that you land on either the incorrect answer or something as a beginner that you wouldn't necessarily know how to assess the validity of.

12:31 Michael Kennedy: Yes.

12:32 Dane Hillard: And it could be from just plain wrong to malicious, or something, right?.

12:36 Michael Kennedy: Or even just it's a perfectly good answer, but it's not the same situation. But you haven't, you don't have the nuance to know that's not actually what I'm looking for. It's the next Stack Overflow one I got to go after.

12:50 Dane Hillard: Yeah, so unfortunately the answer there is sometimes is that there's a lot of trial and error, but, again, that sticks with you throughout your career, I think.

13:00 Michael Kennedy: Yeah, yeah, yeah, I did a show back on 203 with Ned Batchelder and some other folks about beginners and experts in this panel talking about, had a bunch of beginners, and I had Ned as the expert. And it was a really interesting conversation. And a few of the things that I condensed out of that conversation was there's a lot of things that beginners perceive as a this is me doing it wrong or this is me not good at it, but it's just that's how it feels. Like for example, being stuck and trying to get a library to work when it's not, or trying to get some, like the compiler to work if you're in a compiled language or whatever, just being stuck or feeling kind of lost is just part of the journey. And like, as a professional, as a pro, you're like, "Well, we're in this part of the journey. The next half hour's going to suck, and I'm going to be on Google a lot, but here we go." And then you just know if you just keep hitting that problem, eventually, you'll be gotten through it. Another one, I think knowing data structures is really, really good for people. And then, like you said, having these battle scars of yeah, it looks like they're talking about your problem, but that's not the problem, it's this other thing, because I've tried four times that way, and that was really a bad week, and that didn't work, so now we're over here. And some of those, you can learn. Like just knowing that well, if you're stuck, you just pound through it, and that's just part of the journey, it's fine. The building up the battle scars, that just takes time, I think.

14:21 Dane Hillard: Yeah, absolutely. And like you said, when you see a Stack Overflow answer and you think it might be the one to solve your problem, there's a similar thing as you're using like, third party libraries where...

14:30 Michael Kennedy: Yeah, that's true.

14:32 Dane Hillard: You can't get it to work. You can't get it to work, but you don't yet know if it's because of something you did or if it's really because there's a bug. And getting to uncovering that bug can sometimes be an exciting moment, even though it's frustrating getting there, because you're like, "Oh, I actually exorcized a problem that other people might be experiencing." and then you can sort of contribute back to that discussion. And if that bug's already been filed or something, you can say, "Hey, I ran into this, too." And it just kind of grows the community in that way.

15:07 Michael Kennedy: Yeah, it's interesting. I just got an email today from GitHub. I had filed an issue for PyInstaller. I was trying to combine it with something else, and it wasn't quite working right. It was doing something weird. Or no, it wasn't PyInstaller. It was Gooey. I was using those two together. And I got a message to say somebody else said, "I'm having exactly the same problem. What can we do about it?" I'm like, oh, I still can't solve it for you, but I started the conversation six months ago or whatever it was. So, let me ask you...

15:33 Dane Hillard: It's serendipitous...

15:34 Michael Kennedy: Yeah, go ahead.

15:34 Dane Hillard: Sometimes.

15:36 Michael Kennedy: Yeah, it definitely is. It definitely is.

15:37 Dane Hillard: It's aha moments.

15:39 Michael Kennedy: Yeah, it's funny. This portion of Talk Python To Me is brought to you by Tidelift. Tidelift is the first managed open source subscription, giving you commercial support and maintenance for the open source dependencies you use to build your applications. And with Tidelift, you not only get more dependable software, but you pay the maintainers of the exact packages you're using, which means your software will keep getting better. The Tidelift subscription covers millions of open source projects across Python, JavaScript, Java, PHP, Ruby, .NET, and more. And the subscription includes security updates, licensing verification and indemnification, maintenance and code improvements, package selection and version guidance, roadmap input, and tooling and Cloud integration. The bottom line is you get the capabilities you'd expect and require from commercial software, but now for all the key open source software you depend upon. Just visit talkpython.fm/tidelift to get started today. Another bad feeling is when you're searching for something that you don't quite understand how it works, and the best answer you can find is something you've written before. That's also bad. All hope is lost now. So, let me ask you, thinking back, I'm not sure if you can remember clearly to this, but when was the first time you felt like, "I'm actually a professional software developer?" Like I'm a pro now, I fit in with all these people who used to seem way smarter than me, and I couldn't figure out what they were talking about.

17:06 Dane Hillard: It's a great question. I don't know that I have this decisive moment, but so, I guess one place that I had a clear division line was for my first job. Everyone was a generation older than me at my office, and I mean, they were all great mentors, and I learned a ton of stuff, but I definitely realized my own shortcomings often. And then, at my next position, people were more my peers in age and experience, and I think when, mostly it was that kind of process of solving things together, and like collectively, we all knew about the same amount of stuff, and suddenly people are asking you to like, not only execute on a solution, but actually figure out what the solution is supposed to be. I think that's really where it starts to make you grow as a developer, so I would say that's, to me, that's kind of the hinging moment.

18:08 Michael Kennedy: Yeah, that's interesting. Like Dane, we've elected that you're going to write the eCommerce part of the website. You're like, "I've never done that one." "Neither have we, so someone's got to figure it out, so we've elected you. We got trust in you." I guess for me, I guess there's probably two experiences. One, the first time I got paid a good salary to sit and work on software, and create applications for this company. It was just so awesome. I was so excited, but I kept thinking, man, I better figure some of this stuff out before they realize I don't know what I'm doing, and they're going to kick me out of here, you know what I mean? So, that was the first step in the door, like wow, I'm really getting paid to write and create software, and it's fantastic. But I didn't really feel totally solid. And I guess when I really felt like okay, I guess I'm a pro, I guess, was when I was giving user group talks on subjects, design patterns or testing or whatever it was. And all these people are enthusiastic. They go to meetups and stuff. And gave a talk and they were like, "That was really great, I learned some stuff." and they didn't laugh at me. I'm like wait, that was great. Apparently, I can present ideas to people. There's a couple of those milestones I can think of, but yeah, it's really hard to put a clear demarkation on it.

19:24 Dane Hillard: Yeah, I mean, going back to your point about feeling like people are going to find you out, that is the epitome of imposter syndrome, which I think is, especially in knowledge work, and like, that kind of area, I think people experience that to a large degree, so that's another thing to call out as a beginning developer if you have this constant feeling that somebody is going to call you out for being a fake, like that's real. That's a thing, and that sometimes doesn't really go away either. I know I certainly feel it at least from time to time still.

19:57 Michael Kennedy: Yeah, for sure, it's hard for that to go away, but at the same time, if you're making steady progress and learning a little bit more and learning a little bit more, if you're delivering what people want from you, just 'cause you don't know how to factor that database into a third normal form, if that's not your job at the moment, you're probably fine, you know what I mean?

20:13 Dane Hillard: Yeah, exactly.

20:13 Michael Kennedy: Cool, so speaking of topics and things like third normal form you should know or design patterns or software design and all that, let's dig into some of the ideas behind your book 'cause I thought you did a good job laying out some of these things. And again, I don't think it's professional development, more like what do you really need to be an expert in software development, in general, and Python in particular, right? Find the right focus, huh?

20:35 Dane Hillard: Yup, I would say so.

20:37 Michael Kennedy: Yeah, so why'd your write this?

20:39 Dane Hillard: The impetus for this, I think, was seeing a few of my friends who were not in the traditional path to software switching into a career in software, and seeing how they thought about problems and the kinds of questions they were asking me at the time. That sort of insight into the kinds of things people were experiencing made me interested in helping other people who were in that same situation.

21:06 Michael Kennedy: Yeah, that's cool.

21:06 Dane Hillard: One of my friends was using Python to solve computational biology problems, computational chemistry problems, if I remember correctly, and was asking me to review his code from time to time, and asked like what he was doing wrong or what he could do better. And it's sometimes hard to answer that question, right? If it's someone who isn't in software, you don't necessarily want to dump the entire world of, you know, you should format your code this way, or you should optimize for readability, or you should comment your code for these reasons. Like it can overload people, so, I figured a book could be a more digestible thing that people can jump in and out of as needed. And I tried to organize it in a way that they could pick areas that they want to get better at and read about that topic without having to know everything.

22:01 Michael Kennedy: Right, it does seem like it's a little bit of this cookbook feel, like you can drop in, and alright, I need to work on design, I need to work on testing, or whatever it happens to be. Nice, so, let's start at the beginning, I guess. And you talk a lot about design and planning and software architecture to get started. And I guess one of the challenges, I guess probably two challenges, that are really tricky for people right away is first, like what the heck is software design? Like you know when you see that it's not design. If there's no functions, there's no comments, like you know this is not design and this is not professional, and probably this other thing if I go to GitHub and pull down Flask and I look at its code, it probably is design. But that's a little bit tricky. But knowing how much energy to put into it, what are the first steps that you take, what do you think about that?

22:50 Dane Hillard: Yeah I mean, I definitely emphasize in the book to the point about energy. Like things are always trade-offs as you develop software. You only have so much time in your day, and realistically, you do need to ship software sometime. There's always a trade-off between how much time you spend upfront designing things versus how much time you spend actually implementing things and getting them out the door, so that is worth stating, I think, explicitly here. And then, generally, I think about design often, the way you think about solving problems in general. Like the problem is often too big to solve in one go. And so we typically break problems down into smaller chunks that we can solve. And that is a lot of what software design is too, this idea of separation of concerns and things like that, encapsulation, abstraction. They're all about not doing the whole world at one time.

23:48 Michael Kennedy: Sure, I've tried to boil the ocean before. It's not so fun. Like I remember there's this project that a whole team of us worked on for years, at least a year. And I somehow discovered this cool linting tool that would come along and tell you all the issues, even security issues, performance issues. And ran it against that, and it found like a thousand problems. And I started to work on a couple, and it took like, 10 minutes per one. Like yup, we're not doing this. Just throw that away. We're going to just keep going the way it's going 'cause you can't take the next half year off, and just make it the same but slightly better. So, there is this real tension of I know this could be better. Maybe I should refactor this to use not use deep inheritance, but I should use composition. Or I should refactor this so this is a separate package, and then we can all use that, and things along those lines. But one of the challenges with software design, I think, is it's almost got this creativity, this art aspect to it, where you could just keep polishing or keep sculpting indefinitely. I could continually refactor this code or restructure it basically indefinitely. And so, you need to know maybe some reasons or triggers that say it's time now to go put a little effort into this, or it's time to stop. Yeah, I probably could refactor that as well, but you know what? Nobody touches that part. It doesn't really matter.

25:11 Dane Hillard: Yeah, that's a great point, and I think a lot of newcomers, as they learn new tricks if you will, tend to try to use them as much as they can, which it's a great way to continue learning that subject and understand more of the nuances around it, but sometimes, if a nuance around, like you said, when to stop or when things are good enough is missed, you can certainly continue much too far down that track.

25:35 Michael Kennedy: Yeah, so let me throw in a couple of ideas on what I might think of as clues there, and you can comment or add to it. So if I'm trying to add a feature, and it's super hard to add the feature without causing a bug, I need to make a change, and I got to touch a bunch of places, or something like that, that seems like I need to redesign something so that it doesn't happen too much. Or there's a part that bugs keep coming back to or something like that. If it's really problematic, and every time you need to make a change, I got to spend 10, 15 minutes on this section just to make sure it doesn't break, or to make sure it's in sync, over and over and over again, I guess if you feel annoyed at your code, like this is really, ugh. You're like wait a minute, maybe that's a signal that I should take that frustration and fix, rather than just endure this code.

26:23 Dane Hillard: Yeah, that's exactly right. I think the squeaky wheel gets the grease in that sense, right? Like as you go about your day to day work, if you find an area of code that's commonly the thing causing you grief, that is a pretty strong signal. And to the point about going too far, I think if it is giving you grief at that time, and you choose to attack it, like it's very easy to get tunnel vision and want to solve it to its extremity, but try to focus on solving the thing that was causing you pain. And you know, if you need to change some peripheral things to solve that direct problem, that's okay, but try not to fix every problem until they really jump out at you.

27:04 Michael Kennedy: Yeah, and you probably also need some guide rails or maybe safety nets is a better analogy here. If you're going to start tearing stuff up, maybe you want to have tests to make sure that it still works, or something like that.

27:15 Dane Hillard: Exactly.

27:15 Michael Kennedy: Like not every bit of code needs testing, but if you're going to start really, really pounding on it, might be worth putting some basic, rough, does this kind of still sort of work tests in place before you do that.

27:27 Dane Hillard: Right. And these are like these characterization tests. Like you want to make sure that the most important aspects of your code continue to stay...

27:36 Michael Kennedy: Right, if this breaks, it's going to be really bad.

27:39 Dane Hillard: In line with your expectations.

27:41 Michael Kennedy: Right, if it ends up being the core trading engine that decides buy or sell, that part can't break, so we better...

27:46 Dane Hillard: Yeah, exactly.

27:49 Michael Kennedy: I'm sure I'm going to hear of it if I switch a minus to a plus the wrong way.

27:55 Dane Hillard: That's not one you want to hear about with production error count going up.

28:01 Michael Kennedy: No, no, it's definitely not. I guess when it comes to this design and like restructuring your code stuff, I kind of try to follow what I guess I'd call the Boy Scout rule. I was never a Boy Scout, so maybe I'm mischaracterizing them, but just leave the code slightly nicer than I found it. If I come in here, instead of making it worse, what could I do a little bit here and there to make it better than when I, if I got to touch it anyway, how do I make it a little bit better?

28:26 Dane Hillard: Yeah, and I think the corollary of that is that that's a great place to start incrementally improving your code if you're not sure where to start. Like as you're working on something already, whether it's to add a feature or to fix a bug, why not use that place where you're already working as the place to improve? So, if it feels like there's too much, and you need to fix everything, start by fixing where you're already going.

28:51 Michael Kennedy: Right, yeah, that's a good point. So you call out some interesting things, some considerations of making software good. As a user and as a developer. And I think these can also guide this design thinking and this refactoring story. So, some of the considerations as a user of how the software's designed. I mean, as a user, hopefully you don't see the code. You see the buttons and the widgets and whatnot, but you still get a sense of, like I could tell you there's certain times where I've worked with certain websites, I'm like, whoa, this is one shady, crappy website right here. Or other ones you're like, this thing is super slick, and maybe it's crummy underneath, but it feels like at least pretty well put together. And some of those for the user are like, speed. How quickly does it respond to me? Or data integrity. Does the data get corrupted? And is it accurate? In terms of resources, maybe if it's a website, you don't see these, but certainly if it's a desktop app or something, you would. And does it feel secure? So, these are certainly things that people can think about as well.

29:53 Dane Hillard: I talk a little bit in the book about when is good enough? To the point of speed, things are often down to your user's perception. You don't necessarily need to continue making something faster from 100 milliseconds to one millisecond if it's, for instance, some sort of UI interaction. Like human perception doesn't go much faster than about 100 milliseconds. So, there's heuristics, if you will, that can help guide where you should shoot for.

30:22 Michael Kennedy: Yeah, and this is interesting. Certainly, there's the single user speed, and then there's the scalability speed. On Friday, nobody can use the website because everyone's entering their time sheet or something like that. But on Monday, it's really fast. That's also another one to consider. But at least in the Web world, maybe even in apps for APIs and stuff, there's a lot of stuff that doesn't, it seems like it's some weird side case. Yeah, I don't really care about this. For example, if I'm working on a website, and I want to make it fast and the page loads in 150 milliseconds, I could optimize indices, I could refactor databases, I could do cluster, I could do all sorts of stuff, but if there's a bunch of JavaScript and a bunch of CSS and a bunch of images that don't have the right cache setting, and the page appears to me as a user in 1.5 seconds, whether I shave 10, 20 milliseconds off it, I don't care, the thing is super, super slow. I don't like calling companies out, but there's so many professional sites out there of big companies, like why is this so bad? It's inconceivable to me that it is this bad. And I actually have a recommendation for people. I don't know if you're familiar with this, but you know PageSpeed Insights from Google?

31:38 Dane Hillard: Yup.

31:38 Michael Kennedy: Yeah, so that's a really great place, and you could just put your domain, or actually, put any particular page in there, and it'll tell you how fast your site is. So for example, the training site that I run, the Talk Python Training, its response time is super fast. I've got that dialed in so well, and it was responding in like, 15 milliseconds, out the other side of the web server. I'm like, well, I can't do much better than that, and no one's going to perceive that last five milliseconds. I throw it into PageSpeed Insight, and it says, "Your page is average to slow." I'm like, wait, wait, wait, wait, wait. Like it's 15 milliseconds, what're you talking about? And then there's all these other things, like well, this type of caching does this. These images are like, 1200 pixels, but when you put them into a phone, the phone has to resize it, and that makes it render more slowly and stuff. So I did actually a ton of work, and now I've got it down to 95 on mobile and 99, out of 100, on, its speed, higher is better, right? So, there's actually different levels. Like you can get really focused in on well, let me make the database query fast and it should be fast, but there might be bigger problems that are actually easier to solve.

32:43 Dane Hillard: That's a really good point. That process that you described sounds very familiar to me.

32:48 Michael Kennedy: Did you put something into PageSpeed Insight, and you're like, "I thought this was good. What is the problem here? This is not what I expected."

32:53 Dane Hillard: Yeah, exactly, and like, Google Webmaster Tools start to incorporate some of that data, too, so you can see how often you're terrible.

33:02 Michael Kennedy: Yeah, well, and you know, to put a little more fire under people to try this out, Google is starting to use that for rank.

33:10 Dane Hillard: Absolutely.

33:10 Michael Kennedy: Right, so. It's an advantage.

33:13 Dane Hillard: Yeah, they're including things like on mobile, if you have an icon-based navigation menu or something, if the icons are too close together, they'll ding you because it's difficult to touch the icon you meant to, especially for people with mobility issues or things like that. So they're really starting to, like you said, incorporate that back into how well you're actually ranked.

33:37 Michael Kennedy: Yeah, yeah, so, I guess to wrap it up is like, you definitely want a fast website, fast app, whatever, but make sure you're viewing it from the user perspective. What does fast really mean? If something is running quick, but another part is really slow, make sure that the way the user feels it is actually the way you want them to feel it, I guess.

33:58 Dane Hillard: Yeah, well said.

34:01 Michael Kennedy: This portion of Talk Python To Me is brought to you by Linode. Whether you're working on a personal project or managing your enterprise's infrastructure, Linode has the pricing, support, and scale that you need to take your project to the next level. With 11 data centers worldwide, including their newest data center in Sydney, Australia, enterprise-grade hardware, S3-compatible storage, and the next generation network, Linode delivers the performance that you expect at a price that you don't. Get started on Linode today with a $20 credit, and you get access to native SSD storage, a 40 gigabit network, industry leading processors, their revamped Cloud manager, cloud.linode.com, root access to your server, along with their newest API and a Python CLI. Just visit talkpython.fm/linode when creating a new Linode account, and you'll automatically get $20 credit for your next project. Oh, and one last thing, they're hiring. Go to linode.com/careers to find out more, and let them know that we sent you. So, that was considerations as a user, speed, integrity, resources, security. But as a developer who's working on this project, be it a library or an application or whatever, there's a different set of considerations, right?

35:13 Dane Hillard: Yeah, I mean, from the developer perspective, in the book, sort of a famous idiom. It's like make it work, make it right, make it fast. So, often we get to the make it work. Like that's the software we often ship or the software we often write to start.

35:30 Michael Kennedy: Yeah, and I think part of the problem is also the project managers. They're like, "Oh, I see it working. Our sprint is almost over. You can pick another item out of the sprint." You're like no, no, no, no, you don't understand the technical depth that is deep within this beast. I am nowhere near done. I actually ended up, at some point, not lying, but just when somebody says, "How long is this going to take?" it's not like how to minimal, viable feature. It is how long to feature plus unit test plus refactoring. That's what I'm going to tell them the estimate is because otherwise, I'm going to hate life. And six months is like, that builds up.

36:05 Dane Hillard: Yeah, in my experience, the best time to incorporate those things is as you're writing the code because at least then, you have some leverage because things aren't shipped yet. And six months from now, if you didn't address the tech debt back then, it's really often difficult. And it's understandable, right? 'Cause at the end of the day, most of us work for some sort of business, and they do have to have value to keep the lights on, and so, that's why I encourage people to bring this in as an iterative part of their day to day programming process because again, that's kind of the best time to do it, especially because you have all the context and everything that's easy to lose after even a couple hours after leaving some code, so.

36:52 Michael Kennedy: Yeah, it's hard to get that back. Like it's much easier to keep your credit card paid off than it is to deal with it when it's maxed out.

36:59 Dane Hillard: Yeah, exactly.

37:00 Michael Kennedy: I mean, you have the same amount of money over the same period of time, but it doesn't feel the same.

37:05 Dane Hillard: Yeah, and so like, as you start shipping stuff, you also then, as a developer, you've hopefully already worked on the ideas of speed and security and things like that as part of the feature you've developed. But under all of that is the actual code that you wrote. If ever you need to return to it to add another feature or fix a bug or performance, the way you've designed your code is when it becomes really important because I often find, if I haven't put a lot of thought into some code, when I come back to it later, I'm kind of viewing it for the first time, even if I was the one who wrote it.

37:44 Michael Kennedy: What is that? That saying like, code as if the person inheriting this code in six months is a psychopath and they know where you live, or something like that?

37:50 Dane Hillard: Right.

37:51 Michael Kennedy: Yeah, I mean, but that applies to you as well because you'll forget. You'll forget and it seems so clear and simple at the time when it was all perfectly balanced in your mind. But then when you haven't picked it up for a long time, you're like why is this still not working? It's like a whack-a-mole game.

38:08 Dane Hillard: Yeah, the best is when you git blame on the line, and you're like, oh, I did that.

38:14 Michael Kennedy: Yeah, like this is wrong. Who wrote this? Oh yeah, alright.

38:19 Dane Hillard: I mean, the worst to me is when I'm like, trying to understand some stanza of code, and I'm like, I need to go ask someone what this does. It's like, who should I ask? Well, I guess I need a time machine.

38:33 Michael Kennedy: Yes, exactly. If it says you wrote it then well, you're kind of out of luck 'cause... The buck stops there. Yeah, so I think these are all really interesting ideas. I do feel like one of the areas you should focus the most on is make it easy to add new features. And some of the things that come out of that are maybe loose coupling and extensibility. Because if one little change means I have to change five other parts of the application, well then, you do have to hold a lot of those things in your mind, but if you've got a very clear structure, like here's all of the strategies we have for this part of our app, and I don't know, then when you just clone this thing and derive from that and register it there, and now there's another option in the UI or whatever. You can definitely help yourself or hurt yourself in that regard.

39:21 Dane Hillard: Yeah, I mean, we often talk about software complexity as sort of like this inherently bad thing, which I wouldn't necessarily agree is true. I mean, you can argue the semantics of it, I suppose, but to me, like the danger of complexity is that people, not software and code, but people have trouble understanding it and keeping it all in their head. And they will inevitably not have the full context when they make some change. And so, bugs become an inevitability, and security failures become an inevitability, and that's where things really break down. So the value in breaking down complexity is often for people's benefit. It's nothing about the code itself, if that makes sense.

40:10 Michael Kennedy: Right, absolutely, and with the right design, you can control how much of that complexity has to be cached in your brain. Like the software may be super complex, but you just got to work with this little part. And you know, if I'm working this part, it interacts with this other thing through this clear way. And as long as I have that in my mind, I can't mess up the rest of it. Or you design it poorly, and you change it in 10 places 'cause somebody's hard-coded a string, duplicated in four places that means something, and now you better not forget that fourth place, or you're in trouble.

40:40 Dane Hillard: Yeah, and so, I think new people often pretty readily understand the value of writing a function. Like here, you need this six times, and instead of writing it six times, you can write it one time. And so, they get the value of that kind of statement. And then, this book tries to bring that concept along to layers of abstraction and encapsulation as things as concepts that you can further use to get that same benefit at a larger scale on more complex systems.

41:10 Michael Kennedy: Yeah, that's a good point. Obviously, functions are probably the lowest level that people generally have that thought process. But yeah, you have a whole section where you talk about separation of concerns, and talk about functions, but there's also classes, and there's modules, and there's packages. And probably the whole step from different modules, I feel like not everyone, but there's a lot of folks in the Python space that are hesitant to have many files that make up their application. They'd prefer to have one 2000 line file. Even if it's got a bunch of functions and classes in it. And I don't know, there's just this reticence to do that. And so, certainly focus in on modules, and then even maybe packages for broader reuse makes a lot of sense.

41:56 Dane Hillard: Yeah, and I think it's easy to go crazy with those as well. You could go to the other extreme and have like, one function per module, and have 1000 modules. Again, it's this balance that you have to strike, and it's whatever, at the end of the day, the goal is always whatever helps people understand what they need to understand, when they need to understand it, and no more, no less.

42:23 Michael Kennedy: Right, if I could say group five actions, and a few pieces of data into a module, and then if all the people have to think about is I use this module, it solves this problem, it takes this input, it has this output, they no longer have to worry about the details or what changes there or anything. And while that's useful on one level, if you can break your whole way of thinking your program about that, you can just think at a different level, and the building blocks become much more easy to think about how you position them and how you use them and so on. And I think changing the way people think about problems is a really important part of this separation of concerns, and this design thinking. If you can change the building blocks as people perceive it to be large and clear rather than many, many small pieces, that's huge.

43:13 Dane Hillard: Even through the course of this book, my sense of that concept has definitely grown. It's very easy to think of the value of refactoring or pulling out functions and things as strictly for reuse purposes.

43:28 Michael Kennedy: Or testing purposes or something, yeah.

43:29 Dane Hillard: Yeah, and so, I've come to kind of feel that instead though, the real value definitely is that sort of cognitive load aspect.

43:38 Michael Kennedy: Yeah, I find a similar value with design patterns. Instead of thinking of these 10 lines of code, I can think of it as a strategy or a something else, right? Even though design patterns are not used that often in the Python space, it's just a little bit simpler. So many of the design patterns are about solving the inflexibility of static languages. And so, you don't have to really think about them as much. But there's certainly many cases where design patterns are super valuable. Even just things like passing a lambda function to a list to sort it, rather than getting in there and trying to deal with it yourself or whatever. It's pretty nice.

44:15 Dane Hillard: My experience is that it just takes a lot of trial and error to get there.

44:20 Michael Kennedy: I don't know, I think there's a lot of design patterns applied from things like Java and C++ and C# that are just, they just don't belong. Like really complicated singleton patterns, where you know what, that could just be a global variable in a module, and alright, what's next? You know what I mean?

44:33 Dane Hillard: Yup.

44:34 Michael Kennedy: That's the same thing.

44:36 Dane Hillard: Yeah, and so, I kind of, in the book, I deliberately kind of skirt around this idea of design patterns explicitly. I do kind of mention it at the end as something to learn more about for sure, but I think, like the value they provide is taking some concept and naming it. Like once you have a name for a thing and everyone agrees what that name is, you can start to have conversations about it, and whether it's the right thing or the wrong thing.

45:03 Michael Kennedy: Right, and it has clear trade-offs. Like for a singleton, it has the trade-off that you have shared data, and it could be really tricky to test it or other things, but it also is really clear how you use it. So, I think it's not just the name. It's that the name plus known trade-off. Like these are the positives, these are the pros, and these are the cons. Do we use that here? And that's really valuable to bring in quickly.

45:26 Dane Hillard: Yeah, and so, if you didn't have a name for that, you couldn't even start to apply those traits and trade-offs, so even if you aren't using design patterns, per se, in your code, it's good to at least understand. I mean, there will be patterns of some kind, whether they're design patterns that have a name or not.

45:45 Michael Kennedy: Right, right, are they organic or are they well known ones?

45:48 Dane Hillard: Yeah, right, so the name just gives you the hook around which you can explore more.

45:54 Michael Kennedy: Yeah, yeah, the nomenclature, that's cool. Yeah, so another thing you talk about in the book is designing for performance. You want to touch on some of the highlights from that?

46:03 Dane Hillard: Yeah, I think that goes back what you said earlier about knowing and using the right data types for jobs, and understanding a little bit about complexity and about how to do things as iterative chunks of work rather than trying to read large files into memory and things like this. That in part is where you do become, so called, more professional in a language at least, is when you, I think, have a good handle on those data types and when they're useful or expensive.

46:35 Michael Kennedy: You know, I think there's a lot of people who, in the industry these days, who are just kind of, they don't know how to address this kind of broader conversation in hiring people or assessing skills. And they kind of fall back to testing algorithms. A lot of the technical interviews seem to be like, well, implement quicksort on a whiteboard. Like, you know what? Nobody needs to implement quicksort on a whiteboard. But one thing I do think really is an important skill and is an indicator of how effective you're going to be is do you know what a dictionary is? Do you know what a set is? Do you know what a list is? Do you know when to use a dictionary over a set? When should I use a list over a dictionary? Like those types of things can just dramatically change the complexity of the code you got to write, the speed that you write it in, or speed that it runs in, maybe both, but these simple common data structures seem just so valuable.

47:31 Dane Hillard: Yeah, and I think that's a thing you can translate to most any language. You can carry that with you wherever you go, so most times, if you have a background in one language with those, in another language, it's really just learning the syntax to produce those. I mean, sometimes, they're varied names, whether it's a hash or a map or a dictionary, that kind of thing, but largely, largely the same, so.

47:54 Michael Kennedy: Yeah, I definitely remember an experience where we were storing some data in a list, and we had to look up some feature about it. So we would iterate through the list, and then go find it, and then work with it. And that was happening over and over and over. And the code was supposed to be something kind of like real-time, and it was just too slow. The other thing you talk about in there is profiling. So I actually threw it into a profiler, and I found, oh my gosh, it's spending 80% of its time in this list finding the item. 80%, it was this huge complicated thing, and 80% of it was one line trying to find the item in the list to go do something with it. And I switched it to a dictionary, and that 80% went to like, 1% or something because you know what, dictionaries are awesome looking up by feature and getting the real thing back. And it's just like, ever since that, I'm just like, oh boy, I'm going to pay so much better attention to data structures. I think that was 20 years ago or something, but it really still stuck with me because wow. Yeah, pretty interesting.

48:51 Dane Hillard: I emphasize the benefits, I guess, of profiling in the book because I do think the situation you describe is sometimes difficult to find, even if you're familiar with your code. And certainly, as someone maybe less familiar with data types and things like that, it would sort of be a needle in the haystack kind of search. And so, profiling really gives you that.

49:12 Michael Kennedy: People are really bad at estimating what, if you show them code, like where is it slow? Even if you were pretty experienced, you could still get it quite wrong.

49:20 Dane Hillard: Yeah, and we build up these very subjective views of these things in our minds, I think. And a profiler is a way to kind of strip away all of that. And it's nice, too, because it's much easier to take feedback from a computer, I find.

49:35 Michael Kennedy: That's right.

49:35 Dane Hillard: It removes a lot of the guesswork, and so, I emphasize definitely like, shoot for, like always profile and gather some kind of evidence for your changes that are performance-related at least. And as you're changing things and looking at the profiling data, shoot for things that are like order of magnitude changes, and not just beating it with a hammer bit by bit until it gets roughly where you want.

50:02 Michael Kennedy: Yeah, because there's certainly different things that you can do, and knowing, I guess the first thing is if you've got, let's say, 1000 lines of code. And you look and you say, "I think this five lines of this function is where it's slow." If it turns out that that's only 2% of the time, it doesn't matter if you can make it infinitely faster. You're only going to make it 2% faster. That's the upper bound. If you can make it go to zero, that's as good as it gets. So knowing where to put that energy, that's what I find the value of the profiling is. So like, maybe not saying this algorithm is bad or whatever, but just like, we should focus here 'cause 80% of the time is on this little bit, and I don't know what the fix is. Maybe it's to use a different library. Maybe it's to rewrite this part in Cython. Maybe it's to just use a different data structure. But this is where the energy and effort should go because we can optimize the rest all day. Maybe introduce a bunch of bugs, and it won't get much faster.

50:59 Dane Hillard: Yeah, it's often true, too, that trying to eke performance out of something necessarily makes the code more difficult because you are either going closer to the middle or doing these kinds of hoop-jumping that doesn't really result in clearer code.

51:16 Michael Kennedy: Right, you're like well, this library that we could just call and get the answer to, well now, we've got to rewrite it because we need to have more control over it. But now, all of a sudden, what was one or two lines is now many, many lines that is your baby to take care of. Cool, so definitely profiling, I think it's really good. Another thing to think about, I guess, is test data versus real data. How many times have there been website launches where oh, it seemed like it was fine, but now it's down because there are actually more than 10 rows in the database, and they forgot to put indexes. I don't know how people can forget to put indexes because they're like magic to databases. They're like speed magic. You can just give them and it's like a thousand times faster. So as soon as you think it's slow, put an index on it. Now let's go worry about what's next. But still, this happens all the time.

52:05 Dane Hillard: Yeah, and I don't know, indexes are sort of notoriously, notoriously hard even when you are thinking about them.

52:12 Michael Kennedy: You can have situations where you're like, you know, this index didn't cover it right because we were sorting this way. It actually used a different index. We had to give it a hint. I'm not talking about that. I'm talking about we've got a million rows that we're doing a query with no index, which I think happens more than people want to admit.

52:27 Dane Hillard: Yeah, definitely. And I guess to the point about real data, also. Sort of at the product and user and social level, there's a bunch of videos where people of color can't use a product because they didn't test it on people with that skin tone.

52:41 Michael Kennedy: Like those automatic hand soap dispenser things and all sorts of stuff like that.

52:46 Dane Hillard: Yeah, exactly.

52:46 Michael Kennedy: It's got to be super frustrating. Those things already frustrate me, and I mean, I'm...

52:53 Dane Hillard: It not only manifests as a cost to you as the business, but absolutely impacts people in the real world in very negative ways.

53:01 Michael Kennedy: Yeah, well, I can't speak to a hard wire testing of that sort, but one thing that I found, Bob Belderbos introduced me to this, is this place called Mockaroo. You know Mockaroo?

53:12 Dane Hillard: I'm not familiar.

53:13 Michael Kennedy: Mockaroo, make sure I spell it right for people, yeah Mock-A-R-O-O, yeah. And what you can do is they have all these different types of data and you can say go generate me a certain amount. So you can ask for like, first name, last name, email. But you can also ask for all different kinds of stuff. There's so many, so I could ask for like, cars, VIN numbers, I could ask for, let's see, for commerce, I could ask for currency codes or stock names or, you know, and it'll generate as many of those as you want. Say I want a million of those in a JSON format like this, give it to me. And it looks real and it feels real, so you can definitely generate some quick data with Mockaroo and it's pretty awesome.

53:56 Dane Hillard: Yeah, that is really cool. There's a couple of plugins I know for Django that do something similar, meant exactly for that same reason, like testing. Testing often is, people espouse, like testing in isolation and testing just this part or that part, and there's like unit testing and integration testing and all sorts of varieties of that stuff. But I don't think this point is talked about enough, which is test it in isolation, but also use values that make sense.

54:26 Michael Kennedy: Right, maybe don't just test with a first name and a last name. Some names, it's like a three-part name, or something like that. And if you're splitting on the space, that's probably not going to come out okay, right? So yeah, yeah, definitely a lot of testing.

54:40 Dane Hillard: Yeah, or if you're always using like a string as your test variable, maybe you won't uncover your bugs that are throughout your code when there's unicode characters and things like that.

54:52 Michael Kennedy: True, test with emojis, test with emojis as well, I guess. That's pretty awesome. So, we're kind of getting near the end of our time to talk about all this. Let me throw out a quick rule of thumb that I use a lot of times around this, and I just want to hear your thoughts on it. You can disagree with it if you want. But one of the rules of thumbs that I have is that if a function or a class or some part of code needs a comment, it's probably broken.

55:18 Dane Hillard: Yeah, that's interesting.

55:20 Michael Kennedy: I'm not talking to docstring. Like if you wanted to document it, that's a different thing. But if you're like, well, we've got to do this weird thing because, or the function is named so badly there's a comment saying what the function name should've really just been, you know, something like that.

55:33 Dane Hillard: Yup, so, I would agree that like, comments that are redundant to what the code around it could've been or could've said are definitely an anti-pattern. I don't mind when a comment says the why of the code. So, code often expresses simply the how or the what, but if you can't get all of that just by your variable names or your class names and stuff, like comments giving you that extra context are okay.

56:05 Michael Kennedy: Yeah, maybe you're calling some API, and it returns a dictionary, and you're like, "This looks weird, but we have to do this thing because sometimes, it does this, and sometimes, and we don't control it. We got to live with it, but just please leave this check here, 'cause I know it looks like it's not necessary, but on the first day of the month, it always goes weird, so we're going to just need it." That's cool.

56:25 Dane Hillard: Postel's Law is like this idea that you should be liberal in what you accept and conservative in what you send. Sometimes, being liberal in what you accept means exactly that. We have this API we have no control over, but we want to use it because it provides this other value, and so we're going to accept this reality and we wrote this code to deal with it, and here's the situation.

56:48 Michael Kennedy: Yeah, yeah, pretty cool. And those comments make perfect sense. Like please don't take this out. I'm going to hide that away in a function so hopefully you don't look at it, but if you got to get in there, here's what we're doing, so look out. Yeah, pretty cool. Alright, well, it's been really fun to think about these design concepts. And I think a lot of what we talked about is actually pretty broadly applicable, although we did talk about some stuff that very much is Pythonic. We talked about comprehensions and whatnot a little bit. So, great.

57:15 Dane Hillard: Yeah, I think this major chunk of the book, I would say, is very translatable to other languages. I use Python as sort of the vehicle of teaching the concepts, if you will.

57:26 Michael Kennedy: Yeah, yeah, sure. Not that people shouldn't use Python. It's an awesome language. But you know, I think these are just, I guess the point I'm trying to make is these ideas transcend individual languages. Knowing this kind of stuff will help you across, if now you're a Python developer, and later you're going to go learn C#, or you're going to learn Java, or you're going to do C++, knowing a lot of this stuff is really applicable.

57:51 Dane Hillard: Yeah, absolutely.

57:52 Michael Kennedy: I think that's also maybe one of the pro things. When you're a beginner, you're very much focused on the syntax of the language and just making the tools do their thing, and not worried about these higher-order, general ways of thinking. So, that's probably another little indicator there, like we started at the beginning.

58:09 Dane Hillard: Yeah, and I think parallel to design patterns, there's just these patterns of languages and patterns of ways of thinking about systems that only come with time. And as you learn the names for those concepts, you can, again, have those conversations.

58:23 Michael Kennedy: Yeah, absolutely. Alright, cool. Let me ask you the two last questions before you get out of here. If you're going to write some Python code, what editor do you use?

58:32 Dane Hillard: So, I would say I split my time fairly evenly between PyCharm and Vi. I guess when I'm working in a larger context and I need to be flipping between a lot of files and jumping through code a little more often, or if I want to attach a debugger, I usually will use PyCharm. And then if I'm just doing some quick things, I prefer Vi 'cause I like moving around the text that way. There are like Vi bindings for PyCharm, and I might be happy with that, but...

58:59 Michael Kennedy: Can you bring those worlds together?

59:01 Dane Hillard: Yeah, I don't know.

59:01 Michael Kennedy: I don't know. You know, speaking of debuggers, I suspect, I don't know if you cover it anywhere in the book, I don't recall, but consider a debugger is probably not a bad piece of recommendation.

59:13 Dane Hillard: Yeah, that's really great advice. If you can understand what it's doing and why you would want to step through your individual lines of code, like it's a great way to solve problems.

59:24 Michael Kennedy: There is a tendency for some people to use it as a crutch almost, to just not really think it through, to just like, well, the debugger will show me. But I think there's a bigger tendency of people to put 20 print statements, not because really has had some timing issue and you really have to have the print statements. It's just like well, I'm not in a thing that debugs easily. I didn't set it up right in VS Code, so there's no button to press even though like, you could just go and set up a run configuration, whatever they call it. So yeah, I think definitely that's probably something in there as well.

59:54 Dane Hillard: Yeah, and it can help you bisect where the problem is. Like if you have a suspicion about where the problem is, but don't know exactly what's going wrong, picking a couple of starting points, and then slowing moving your breakpoints until you really find the line that was mucking things up is a really nice way to solve problems.

01:00:14 Michael Kennedy: It comes back to just having less in your head at once. You're like, well, it's down to these three lines. What do I have to think about for these three lines?

01:00:20 Dane Hillard: For sure.

01:00:21 Michael Kennedy: Cool, and then notable PyPI package. Not necessarily something super popular, but something you came across. You're like, "Oh this thing was so sweet. "I can't believe I found it."

01:00:28 Dane Hillard: We've developed a couple of services in FastAPI recently.

01:00:32 Michael Kennedy: Okay, yeah, I've heard tons of good things about FastAPI. It seems like there's a lot of momentum around it.

01:00:36 Dane Hillard: It's really kind of a joy to work with. There's sort of some underlying packages that it uses, Starlette and things like that, but there's also some that it works in tandem with. Pydantic is one.

01:00:49 Michael Kennedy: Mmhmm, Pydantic is really cool, yeah.

01:00:50 Dane Hillard: Yeah, it's like a data modeling-related library, and just love them all.

01:00:54 Michael Kennedy: Yeah, you can put little Pydantic constraints on decorators and then, decorators, I think, right?

01:01:00 Dane Hillard: Right, it's a little bit like data classes with validators, if you will.

01:01:03 Michael Kennedy: Yes, exactly, it looks super cool. I haven't had a chance to use it, but I just really kind of got the zen of it a while ago and I want to know.

01:01:09 Dane Hillard: Especially if you're working in microservices and things like that. Asynchronous microservices, kind of where it shines.

01:01:16 Michael Kennedy: Yeah, that's super cool. Awesome, well, those are some good recommendations as well. Alright, so final call to action. People are inspired about this design stuff. What's your advice to them? What do they do?

01:01:25 Dane Hillard: I mean, there's a couple of other seminal works that I would say to read. There's the Gang of Four book and other things like that. If you're interested in Python, I would certainly hope you'll take a look at this book.

01:01:36 Michael Kennedy: Yeah, and I'll put a link to your book in the show notes so people can get to it easily.

01:01:40 Dane Hillard: Yeah, fantastic, and I think once you get beyond that or maybe even in parallel, this idea about testing and refactoring are both super valuable, super valuable bits, so Martin Fowler has a good book about refactoring.

01:01:54 Michael Kennedy: It's got a pretty straightforward title, Refactoring. Yeah, from 1999. It's still totally valid, though. It's a great book.

01:01:59 Dane Hillard: Yeah, and there's Working Effectively with Legacy Code. I think Michael Feathers, maybe, is...

01:02:05 Michael Kennedy: Yup, yeah, Michael Feathers wrote that. That's a really good book. If you've got a large code base, and you're like, you know that problem I described where there's a thousand issues, and we're just like well, we're not fixing these. Like you were in that situation, but you want to carve out a part that behaves better, his book is beautiful for that.

01:02:21 Dane Hillard: Yup, so, those are even the ones I'm reading to some degree now, so, super.

01:02:25 Michael Kennedy: Alright, well, Dane, thanks for being on the show. It was really great to have this chat with you.

01:02:28 Dane Hillard: Yeah, I really appreciate it. It was a lot of fun.

01:02:30 Michael Kennedy: Yup, you bet, bye.

01:02:30 Dane Hillard: Yeah, take care.

01:02:32 Michael Kennedy: This has been another episode of Talk Python To Me. Our guest in this episode is Dane Hillard, and it's been brought to you by Tidelift and Linode. If you run an open source project, Tidelift wants to help you get paid for keeping it going strong. Just visit talkpython.fm/tidelift, search for your package, and get started today. Start your next Python project on Lindode's state of the art Cloud service. Just visit talkpython.fm/linode, L-I-N-O-D-E. You'll automatically get a $20 credit when you create a new account. Want to level up your Python? If you're just getting started, try my Python Jumpstart by Building 10 Apps course. Or if you're looking for something more advanced, check out our new Async course that digs into all the different types of async programming you can do in Python. And of course, if you're interested in more than one of these, be sure to check out our Everything Bundle. It's like a subscription that never expires. Be sure to subscribe to the show. Open your favorite podcatcher and search for Python. We should be right at the top. You can also find the iTunes feed at /itunes, the Google play feed at /play, and the direct RSS feed at /rss on talkpython.fm. This is your host, Michael Kennedy. Thanks so much for listening. I really appreciate it. Now get out there and write some Python code.