#332: Robust Python Transcript

00:00 Does it seem like your Python projects are getting bigger and bigger?

00:02 Are you feeling the pain as your code base expands and gets tougher to debug and maintain?

00:07 Patrick viafore is here to help us write more maintainable, longer lived and more enjoyable Python Code. This is Talk Python to Me Episode 332, recorded August 2021.

00:31 Welcome to Talk Python to Me.

00:33 A weekly podcast on Python.

00:35 This is your host, Michael Kennedy. Follow me on Twitter where I'm @mkennedy and keep up with a show and listen to past episodes at 'talkpython.fm' and follow the show on Twitter via @talkpython. We've started streaming most of our episodes live on YouTube. Subscribe to our YouTube channel over at 'talkpython.fm/youtube' to get notified about upcoming shows and be part of that episode. This episode is brought to you by 'Clubhouse', soon to be known as Shortcut by 'masterworks.io', and the transcripts are brought to you by 'AssemblyAI'.

01:07 Please check out what they're offering during their segments. It really helps support the show Pat welcome to Talk Python to Me, thank you.

01:14 I'm so honored to be here. Long time listener.

01:16 That's fantastic. I really, really appreciate that. It's an honor to have you here. We're talking about one of these subjects I really enjoy. I feel like it's one of these more evergreen type of topics. You know, it's super fun to talk about the new features and flasks 2.0, but that's only relevant for so long and for so many people. But writing software that's reliable that can be changed over time, doing so in a Pythonic way, good stuff to learn, no matter where you are in your career.

01:45 Oh, I absolutely agree.

01:46 You've written a book called Robust Python, which caught my interest. As you can imagine, get a couple of books and ideas said to me periodically per day, and so they don't usually appeal to me.

02:01 But this one really does for the reasons that I exactly stated in the opening. And we're just going to sort of riff on this more broad idea of what is Robust Python. What is clean code? How do you do this in Python?

02:12 Make it maintainable for both you in the future and other people, and so on.

02:17 Yeah, sounds good.

02:17 Before we get to that, though, let's start with you. What's your story?

02:21 So I been the programming like many males my age, I love video games as a kid and I was 12 or 13 and some ad in one of my video game magazines caught my eye for video game creator studio.

02:34 There was this C++, just engine, but it provided easy ways to do sprites and animations and particle effects, and they give you step by step instructions. Here's how you build, Paul. Here's how you build Frogger. And here the assets I started ripping up. I really came to love it.

02:53 I then took a programming class in high school, True Basic, which I don't remember much of other than I don't think I'd ever go back to it now, but it interested me enough that I decided to go to computer science. I learned more about the video game industry, especially some of the hours they work and said, you know what? Maybe the video game industry is not for me, but I still loved programming at its heart. So I continued on and just found a few jobs after college and just went from there. As far as Python one. So work in telecom for a little bit. Early 2010, we had some test software was all written in the Tickle programming language, and that is an interesting programming language. As far as types go, everything is treated as a string, and it led to some very unmaintainable code.

03:39 So a few sharp engineers myself on the side of you. What would it look like if we wrote this in Python? And a whole test automation was born from there, and we just went, wow, Python makes everything nicer. Like, this is so much possible to work with. I have done like C++ and Java before then. So Python a breath of fresh air and I just fell in love with the language.

03:59 Those are both very syntax symbol heavy languages. I've done a ton of C++.

04:05 I did professional C++ development for a while. I've never done much Java, but yeah, they're really interesting. I remember when I came to Python. I came from C#, but also from C++.

04:16 And just in my mind, those symbols, all the parentheses on the if statements and the semicolon at the end of the line, and all that business that was just required. That's just what languages needed to be.

04:29 And then they weren't there. And I'm like, I just. I'm having a hard time with us. Right? And I thought it was weird. But then when I went back just to work on a project that was still ongoing, it seemed more weird. I'm like, wait a minute. Like, the role has been taken off my eyes. I know now these are not required. And still here I am continuing to write them, and it just drove me crazy once I realized I don't have to put these things here. It's just this language that makes me. And so, yeah, it was really nice.

04:55 Yeah. Python kind of capture. That magic for me. And it's funny because going back to C++ every once in a while, my C++ got a lot better because of Python. You start focusing on the simple. That's nice.

05:07 Yeah. Yeah, exactly. You're not trying to flex with your double pointers and all that kind of stuff.

05:12 We don't need this. Let's not do this. Come on. Very cool. So how about now? What are you doing?

05:16 So right now I am a software engineer. Canonical, I'm on the Public Cloud team. So we're building the bunch of operating system for public clouds. Aws, Azure, GCP a couple of others, and so we customize images for Clouds, work with a lot of other canonical teams to make sure that their changes are reflected in our software. We maintain the CI/CD pipelines to deliver those as often as we can.

05:41 Most of the tooling are written in Python, so I get to use Python day to day makes me happy.

05:45 That's really cool. What kind of features are we talking about here? General aspects.

05:50 Yeah. General hardware enablement. So Clouds may have new instances, new processor types working to make sure that that's enabled. Do a lot of close work with our kernel team features that the cloud themselves might request.

06:05 Maybe in the cloud needs to set up and maintain the VM aspect more on the infrastructure side, maybe.

06:14 How do they hibernate their VMs?

06:16 Okay.

06:16 How do they improve boot speed, those sort of changes, or how do they improve the security profile? That's something I'm working on right now with some of our clouds.

06:23 Super important. That's part of the promise I go into the cloud is often there's better security and better durability. But if it fails, it fails. For potentially all of some big chunks of the internet, the consequence of failure is also higher.

06:39 So you have to be very aware of how many people are using your software when you deploy to the cloud.

06:44 Does it make you nervous to work on that kind of stuff?

06:47 Oh, absolutely. Yeah. I'd be lying if I said it didn't. But you learn to be careful. You learn to really focus on making everyone have a good communication between your teams, making sure there's no surprises for anyone, that sort of thing.

07:01 I remember that as well, just working on certain things, like the first e commerce system I wrote, this thing's going to be charging thousands of dollars per transaction, and I might screw this up. And I really we deployed it for the whole company. There's like, big purchase, as people were doing, and it made me really nervous. But at the same time, one of the things I've come to learn over my career programming, it's one of the things that will really put a sad look on your face is if you spend a lot of time and create beautiful software, no one uses it. That's right. Even though it may be stressful that it's getting used a lot is really central. That's also amazing, right.

07:37 But it's also made me a better programmer because you just start thinking about explicit error cases more. And my last job, I mentioned a telecom and 911 calls get routed through your equipment. You don't want to fail that. And yeah, it is scary at first, but you learn to develop disciplines over time of. Okay, how do we make sure we're not making mistakes along the way. Absolutely

07:58 You know speaking of that, I think that's maybe a good segue.

08:01 And to talk about this idea of Robust Python.

08:04 First of all, let's just talk about your book for just a minute, and then we'll just get into the general idea of what motivated you to write this book.

08:10 So I mentioned I've done C++ development in the past.

08:14 I started diving heavy into modern C++ and how it uses a lot more type safety features. And some earlier versions of C++ did a quick dabble on to hassle just for learning perspective and really start to love static type systems.

08:29 But my day to day language is Python, so I really, really dove deep on how do I make my Python safer moving forward. So I pitched to a few publishers. I want to write a book about type system best practices in Python, type, checkers lets talk about design types and so on and so forth. O'reilly, publish my book.

08:49 They bit and they said, oh, that sounds interesting. Can we expand the scope a little bit more? And I thought, okay, well, why do I write types in Python? Why do I advocate type of steps to make things more, maintainable, make things live longer, to make things clearer.

09:04 And the idea of Robust Python came out of that.

09:06 Well, I think it's really a pretty interesting idea. I do think when you come from these languages like C++ or Java or something, there is something to be said for. It literally is verified to be clicked together, just like a perfect jigsaw puzzle. Yeah, right.

09:23 It won't catch every error, but it can catch more errors. And that's a good thing.

09:28 Yeah. Well, I remember how excited I was when I got my first non trivial C++ thing to compile, and I'm like, yes, I've done it. Little did I know I was in for it. I mean, it was out of compiler errors and into, like real errors at that point. Right? Yeah.

09:43 The fun errors, right?

09:45 Yeah.

09:46 Exactly. What is it? What is it? Seg fault? Whatever.

09:50 I don't know. We're going to work on that. So anyway, that was really interesting. I do think that it's not required.

09:58 Right. I do think that there is really interesting code being written in Python, and you can see that how it's being adopted in all these ways.

10:06 The biggest, maybe example I have in mind of these two things side by side is YouTube versus Google Video and how Google Video was in Python or 20 engineers. Google Video was 100 C++ engineers. And eventually Google just said, I'm going put to that project aside and by YouTube because they just keep out running us in features.

10:25 But I think that's the beauty of Python is that you can build things fast, you can see them working faster, and you don't have to fight the compiler as much of the way. So it's almost like these two things are at odds in the answer is just Python gradual type. You just type as you need to. You add typing where it makes sense, and I won't be prescriptive and say you need types everywhere. There are certain places where it makes a lot of sense and saving money and time.

10:50 I do find it pretty interesting how they've implemented types in Python. And to be clear, this idea of Robust Python that you're focused on types just one part of it. But it sounds like it was the Genesis of it.

11:01 It was absolutely the Genesis.

11:02 Yeah.

11:03 So we have sort of two other realities that we might look at. We could look at C++ or C# or Java like they're all the same in that regard. They're like static compiled languages. And then we've got something that's much closer to Python with Type Script, where they said, we have the same problem. We have a JavaScript, which is even less type because the runtime types are all just weird dictionary prototype things anyway.

11:28 And we want to make that reliable for larger scale software and integrations and stuff. So we create TypeScript. But the real fork in the road, I think that was interesting is the TypeScript. People said, we're going to apply the ideas of absolute strict typing. Once you start down that path, it's all down that path and they have to match up, whereas in Python, it's like, let's help you down this path, but not give up the Zen of Python, where you can easily put things together and you're not restricted by this type system.

11:59 But if you run the right tools, be that PyCharm, VS Code or mypy or something like that, it will be there to help you make it better in lots of ways. I think that's a really creative and interesting aspect of it.

12:12 Yeah. I absolutely agree with that.

12:14 A lot of things that I discussed throughout the book is the idea that these are tools near toolbox. And so I talk about types. I talk about API design. I talked about sensibility. They're all tools and they shouldn't be applied everywhere. I really want to start focusing on first principles. Why do we do the things we do in software engineering and what are the most appropriate for them?

12:38 You had a really interesting quote in your book about software engineering, and I've always sort of struggled to have this conversation with people like, oh, are you a programmer? Are you a coder? Are you an engineer?

12:50 Where are you on? Is that even a meaningful distinction? And to a large degree, I've always felt like it was just sort of whatever sort of culture you're in, if you're in a start up, it's one thing. If you're in, like a giant enterprise, they may value a different title in a different way. It's all kind of the same. But you had this cool quote that said something like, Software engineering. Is programming integrated over time? Yes.

13:12 And I wish I could claim credit for that, but that actually came from Title, winners of Google and C++ con CPP Con talk, and it's just really resonated with me. We program the software engineering is the efforts of that programming over years or decades, your code will live decades.

13:32 The more valuable it is, the longer live it probably will be. I've worked on code that's been 15 years old when I started on it. And there's code that I wrote twelve years ago that's still running operationally. And that scares me to some degree, because, like, did I know what I was doing back then? But it just re emphasize why it's so important to think towards the future of what your audience you're going to have maintainers that come after you. And do you want them to curse your name, or do you want them to be like, thank goodness this person wrote something that I can use. It's really easy. And I always prefer the ladder on that.

14:06 Like, the nicer code that you write in, the more durable and long live code that you write, the more it can continue to have a life even after you've stepped away from the project. Right? It's like, this is the thing is still working. Well, we can keep growing it rather than it turned into a complete pile of junk that we got to throw away and start over. Right. So that's a goal that you might want. And then also that long term maintainer might be you.

14:32 Yep.

14:32 You might be working out five years from now. And go, why did I make the decisions that I did? I don't remember this. Maybe you jumped to a different project and came back. That absolutely can be you.

14:44 This portion of Talk Python to Me is brought to you by Clubhouse. IO. Happy with your project management tool. Most tools are either too simple for a growing engineering team to manage everything or way too complex for anyone to want to use them without constant prodding. Clubhouse.IO, which soon will be changing their name to shortcut. is different, though, because it's worse. No, wait, no. I mean it's better. Clubhouse is project management built specifically for software teams. It's fast, intuitive, flexible, powerful, and many other nice positive adjectives. Key features include team based workflows. Individual teams can use Clubhouses default workflows, or customize them to match the way they work. Org wide goals and roadmaps. The work in these workflows is automatically tied into larger company goals. It takes one click to move from a roadmap to a team's work to individual updates and back type version control integration.

15:35 Whether you use GitHub.

15:36 GitLab, or Bitbucket, Club House ties directly into them so you can update progress from the command line keyboard friendly interface. The rest of Club House is just as friendly as their power bar, allowing you to do virtually anything without touching your mouse. Throw that thing in the trash iteration planning, set weekly priorities, and then let Clubhouse run the schedule for you with accompanying burndown charts and other reporting. So give it a try over at 'talkpython.fm/clubhouse' again, that's 'talkpython.fm/clubhouse' choose club house again, soon to be known as shortcut, because you shouldn't have to project manage your project management.

16:17 Let's talk about some of the core ideas that you have about making software maintainable reliable. What are the things you talk about is the separation of time and how code has to communicate, or the artifacts that we all produce around. Code has to communicate with people both, maybe almost immediately. Like, I'm working on this project with a couple of other developers, and we need to keep it going forward. The other one is way down in the future, someone comes back and they're like, oh, I'm new here. The person who created it left maybe talk about some of those ideas that you.

16:52 So it feels a little weird because what we're talking about is asynchronous communication, and it's you to talk about that on a Python podcast and not talk about async way.

17:01 Yeah, exactly.

17:02 There's asynchronous communication in real life, which I actually think is much harder. You have to think about your time traveling to some degree. You have to think about the future, and you have to communicate to them. You probably will never meet them. You'll never talk to them. The only thing that lives on are the artifacts you create. So your code, your documentation, your commit messages.

17:21 That's what people in the future are going to construct. This mental model of your work from. They're going to be doing archeology when things go wrong. Why is this so the way it is? Is it safe to change what were the original intentions?

17:33 And so the more you can embed that into code you write and the surrounding commit messages, documentation, the more robust your code base going to be, the more you're communicating intent to the future.

17:44 That's one of the things that was so important for 2020, 2021 for all of us, right? We didn't know how much we were going to need it, because there's always been this kind of attention. Well, there's the open source world and these other projects, and there's those weird remote teams.

18:00 But we come to our cubicles and we all sit down and we have our stand up in the morning and we write our software together and we use perforce or something internal, where we lock the file. No one else can edit the file. I unlock the like, there's these different ways, and we've been moving more and more towards this sort of.

18:18 Even if the person is right next to you. The way we work is if they were across the world and it's been really, I guess, lucky for us as an industry that that was mostly in place, that actually became true.

18:33 I think you touched on an interesting point of a lot of developers think if we're close in space, we can collaborate.

18:41 I don't have to worry about my remote teams, and I don't have to worry about someone global.

18:46 But by thinking of that, your collaboration. In that terms, it sets you up to think about the future as well, because you could be asynchronous in space or asynchronous time.

18:56 And basically the same tools are there to address it.

18:59 The same strategies help with both.

19:01 Yeah.

19:02 So one of the things that you talked about was this principle of least surprise.

19:08 Yeah, there about that principle.

19:10 The principle, at least surprise also knows a principal least astonishment. I feel like it's safe to say most developers have gone through a code base and been legitimately surprised that function does that.

19:22 Oh, my goodness. I would never have thought that once worked on a nasty bug where the get event function was setting an event. And I kept overlooking it because I'm like, what's a get even is just a get.

19:33 I can ignore this to side effects. This will be fine.

19:37 Two days later, I'm like, Let me go step through this. And I was floored. I'm like, of course, this explains my bug.

19:43 So your goal when developing software is you don't want to surprise your future readers.

19:48 And I think this is why people say avoid clever code favourite clear code over clever code.

19:54 You don't want to surprise readers.

19:56 Many, many people may not be as well versed in the languages. Maybe they're coming from a different language, maybe from type script to Python. The more you rely on clever tricks or poor naming or just wrong patterns, you're throwing people off for a beat. And there's an added cognitive burden that they must then carry to understand what you've read.

20:16 One of the beautiful things about different abstractions are, here's a function. I read the function name. That's all I need to know. Here's a class. I understand what the class does. I don't need to go and look into the details. And that lets you build these more larger building blocks of conceptual models.

20:33 But if that's not true, right? Like if a getter is a setter, well, then all of a sudden those things are all out the window and that's bad.

20:40 I mean, we work by building mental models, and you need trust to build those mental models. And as soon as that trust is violated, it just starts taking time to do everyday tasks.

20:51 You say I want to implement a single API. Endpoint, well, if I have to go dig through ten different files just to make sure I'm doing everything right, it's gonna slow me down. If I can trust that my mental model is correct and it has been shown to be correct time and time again, I could put more faith in that code, and I can feel safer to start changing that code.

21:09 Yeah, for sure. Another thing that I really liked about some of your ideas was that you talked about people having good intentions. Even if they write bad code, they're trying their best. And for the most part, I mean, there might be people who are just lazy or whatever but for the most part, they tried to write this well, even if it came out bad, they probably tried to write it well, and it just didn't turn out as good as they hoped.

21:32 Or even they wrote it well, and it's just changed over time, and those original assumptions got lost.

21:38 It it made sense in the early days, and the assumptions or the context change now it's no longer accurate or it's inflexible.

21:46 And it's really hard to use to extend to your current use case. I see that all the time.

21:50 One of the ideas that I thought was interesting is this idea of legacy code. And I've always been kind of fascinated with what does legacy code mean?

21:57 Because legacy code for one person could be Cobalt.

22:01 Legacy code for another person could be Python2.6. That's pretty old. It could even just be Python3.6. That's been around for a while. Right. And there's different people have different definitions. Michael Feathers has a cool book called Working Effectively or Effectively Working with Legacy Code, something like that. It's an interesting book. I believe it comes from maybe a slightly different time, but still, some of the ideas will make you think.

22:25 And I was gonna quote, that book actually consists legacy codebase doesn't have tests. I used to think that for a long time. I love tests, but I've come to evolve my understanding of legacy code. And here's where the definition I settled on. Legacy code. Legacy code base. It's a code base, which you can no longer communicate to the original authors or maintainers.

22:47 So the length of time doesn't matter as much if you've lost contact from the original authors.

22:53 All you have is the code base and the surrounding documentation to understand why it's student does. So that's become my favorite definition for legacy code as a place. Yeah.

23:02 I like that one, too. I don't really like the test one.

23:05 I see where it comes from, but I feel like it's judging the world from too strict of a place.

23:13 Right.

23:14 Because not every piece of code is written in a way that it has to be absolutely correct.

23:20 Yeah.

23:21 Right. So I think this is also a good way to sort of scope this conversation, because a lot of times people here, I have to use protocols and have to use mypy. I have to use X, Y, and Z, and I have to do all these fancy things because Pat and Michael said so because they were awesome, and it's a big hassle. I don't think I need it, but I'm trying to be a good software developer. Right. So I'll give you an example. Like, years ago, I switched all the talk Python stuff, especially the Talk Python Training stuff from relational databases and SQLAlchemy over to MongoDB and Mongo engine. And so I had to write a whole ton of code that would take a couple of tables and then put them into a structure and then put them in Mongo. And that was all fine and good. But here's the thing. The moment that code ran successfully, I never wanted to run it again. It only had to run once. It had to move. The one time when it was done, there was no scenario where I cared about its typing or I cared about its continuous integrate. I could have deleted it. I just kept it because, hey, source control. But there's these scenarios. On the other hand, you talk about if you're an online reservation system, like the reservation booking engine, that part needs an entirely different bit of attention and mindset than my little migration script.

24:38 Right? Yeah.

24:39 And what I see is what value are the things delivering some things have one off value, and that's perfectly okay.

24:45 Your migration script, it's service value, but there will be much value derived from it in the future.

24:51 Maybe from an archaeology standpoint. How do I go do this?

24:57 Maybe with something that's core to a business, a reservation booking engine. It delivers value when you built it, but you want it to keep delivering value throughout its lifetime.

25:07 And Furthermore, the people were working on it want to deliver value just as fast as you did in the beginning. So you don't want to slow down the future. You start digging into where you get product manager saying, Why is this taking so long? This is super easy. Why do you have to spend three weeks just adding this one little field? The answer is often, you know, we didn't think about how to enable value faster when we built it. And there's a tricky line there because you can't just go plate everything and say, I'm going to make everything super flexible. That often has the reverse effect. It makes things too flexible, and that becomes unmaintainable. Well, there's a fine line between saying, okay, I'm going to think for the future and deliver value.

25:46 That if every dependency can be replaced and everything can be configured from a file, some settings file, and eventually that becomes a nightmare. It sounds cool. It's not cool. I worked on some of those. Yeah, it's just painful.

26:00 And so here's the advice I give to people who want to think about how to make their code base more. Maintainable, target your money makers, the things that produce value, because those are the things you want to protect and whatever value means to you. Target things with high turn so you can look in your get story. You see what files change the most. Chances are those are the files that are being read the most. They are the files that people are working in the most. Putting more safeguards in those files, making them more extensible will pay off just because more people are using them. Look for areas where you do large swath of changes. It's called shotgun surgery, where if you want to add a single thing, you have to touch 20 files. The same 20 files. Keep getting it changed again and again in a grouping that tells you that if I were someone coming in to the project, how do I know it's 20 and not 19 or 21?

26:47 That's a place we can see.

26:48 Those are the things that are super easy to forget. A case.

26:51 Oh, yeah.

26:52 Oh, we added this feature and every if statement had an elde if that covered the new thing.

26:58 Except for that one where we did the auditing, or except for that one where we checked if they were an admin. Oh, that one. Now everyone's an admin. oops.

27:06 So that's the place where I think type, hinting and other strategies super useful because you could start encoding those ideas of I want to catch this when I miss a case, you can start encoding that into your checks, into Linters, type trackers, static analysis, and so on and so forth. Important.

27:25 I think people are pretty familiar with the typing system these days. I think it's really cool. The new type system is coming along with more things like 39. Now lets you write lowercase set bracket integers or whatever, rather than from typing import capital set, and then you can say it in parallel. Right. That's really nice. With the pipe Union, your definition, you can do none pipe a thing instead of optional.

27:53 Yeah.

27:53 All of those are really nice and so on. I suspect that a lot of people are using types for their editor, but are not going any further than that with anything like mypy or continuous integration or any of those you want to maybe speak to the use case of both of those.

28:10 Yeah. So the use of Editors, alone is valuable. You get auto complete it locally if you do it wrong.

28:18 That's right. Auto complete alone is so good. And I was just thinking when you were talking about that getter that was actually a setter, there's a really good chance that whoever wrote that code knew that was bad and their tooling was such that it was so error prone for them to change the name that they were willing to live with a getter that change the values because they're like, I could break so much stuff in ways I don't understand if I don't have a proper set of tools like a proper editor that'll do multi file refactoring and continuous integration and all of those things. Right. And so this is sort of like in that vein of your tools, now do more for you.

28:59 So if you take a look at how costly errors are there, the customer is incredibly cost when you factor in support and testing and field engineering, whatever you need to resolve that customer context, not to mention loss of customer faith. It's expensive for tests in the later stages of QA to catch an error too. Because now our development hasn't been planning to go fix this test or this code, we actually stop what we're doing and go back and fix this test that maybe worked on three weeks, though. The best time to catch an error is immediately after you write it. And that's where that tooling comes in with your editors as you're typing. If you can find an error, great, you is the least amount of cost you could have spent. Then the second least amount, in my opinion, is letting some sort of stack analysis tool catch it. Something like mypy. So using types you can say I want to encode some assumptions to my code base.

29:52 This value will never be none. That may be not. This value may be an inter string or this string.

29:59 If you're right, that it's never none. You never have to check it for none.

30:03 But if you're wrong.

30:04 You always have to check it for none.

30:06 Which is it? Yeah, we know you got a new defensive programming of checking is not on every single variable we create in every function invocation that would just it wouldn't be fun, but your alternative if you don't have that tooling is alright. Does this function return on? Let me go look at its source code. Oh, it calls five other functions. Let me go look at the source code. This is call something to the database. Is that a nullable feel?

30:28 And anytime you're making someone troll through your code base to try to answer a question of can this value be none? You're wasting their time. And they're either going to delay delivering features that just adds up over time or they're going to make something correct assumptions and that's going to cause mistakes which will lead to time.

30:47 And wasted money, you've already talked about focusing your attention to put things like type annotations on the parts that matter and not necessarily stress about the parts that don't, especially for code that's being retrofitted to code. I think that's not right. The thing that logs it's going to turn whatever to a string. And if it comes out as like some object, hit some address, we'll catch it later and figure it out. But it's the core thing that you want that stuff to be right. But one of the things that can be challenging is the interest and the buying and the love for this idea might not be uniform across your team, no no.

31:23 And I've seen the same thing for testing and I've seen the same thing for continuous integration. Not necessarily the same person in the same thing. But you know, it's like if there's a person on your team that just doesn't care about the continuous integration and turn off all the notifications that the continuous integration fails and then they keep checking in stuff and failing the build.

31:42 Like not again. I got to go fix this because this person doesn't bother to check their thing and it just gets super frustrated. And I probably typing has a similar analogy that, it does.

31:53 And that's why, if you're in a legacy code base or even a maintain code base that doesn't have a whole lot of typing in it, there's an alternative you can have just beyond being strategic, where you pick, there's some fantastic tooling like Monkey Type, which can annotate your code base for you.

32:10 There's Google Type Checker, PyType. It could do type checking without type annotations, and it does a little bit different philosophy than mypy which I think is kind of cool. It tries to infer based on just the values throughout your function bodies. The type should be without type annotations, so there might be ways to get the benefits without actually making the full commitment to those type checkers for just type annotations. In general, there's also tie your value to it. Look through your bug reports if you find out that, hey, you know we've had twelve references of none in the past month.

32:49 No, it does not have such attribute. Whatever, right?

32:52 And it's X amount of dollars I can now go to someone say, look, this is costing us real money and like our time, sometimes data speaks volumes.

33:03 If you're having a tough time convincing people, I often say find the data to back it up. Look through your bug reports. What would we have called? How much faster could we go out? Even an informal survey around your developer based? How much more confident do you feel working in your code and use that data to decide? Yes, this is working or no, we need to look at alternative strategies.

33:26 How much auto complete do you get? I mean, that might be a winner, right? That would for me.

33:31 Actually, this portion of Talk Python to Me is brought to you by Masterworks.IO.

33:37 You have an investment portfolio worth more than $100,000. Then this message is for you. There's a $6 trillion asset class that's in almost every billionaires portfolio. In fact, on average, they allocate more than 10% of their overall portfolios to it. It's outperformed S&P gold and real estate by nearly two fold over the last 25 years. And no, it's not cryptocurrency, which many experts don't believe is a real asset class. We're talking about contemporary art.

34:06 Thanks to a startup revolutionizing, fine art investing. Rather than chilling out $20 million to buy an entire Picasso painting yourself, you can now invest in a fraction of it. You realize just how lucrative it can be. Contemporary art pieces returned 14% on average per year between 1995 and 2020, beating the S&P by 174%. Master Works was founded by a serial tech entrepreneur and top 100 art collector after he made millions on art investing. Personally, he set out to democratize the asset class for everyone including you. Master Works has been featured in places like The Wall Street Journal, The New York Times, and Bloomberg. With more than 2000 members. Demand is exploding. But lucky for you, Master Works has hooked me up with 23 passes to skip their extensive wait list. Just head over to our link and secure your spot. Visit 'talkpython.fm/masterworks' or just click the link in your podcast players shownotes and be sure to check out their important disclosures at Masterworks IO/disclaimer. I do want to move on to some other ideas because it's not all about typing.

35:13 But I think typing unlocks a lot of these sort of durability ideas that you're covering them.

35:21 Another one that you talked a lot about, I think is really interesting in this context.

35:26 Has to do with collection types and knowing the right data type and think that matters so much. Somebody might use a dictionary where they should have used a set or something. You're. Well, you use a dictionary. So you mean me to look this up by value? Like, no, no, I just want to have one of everything. Like, okay, why do we use a dictionary? But people when they're new, they don't necessarily know that they find the first thing that works like a dictionary work. For this, we're using dictionaries, but beyond that, it means something for certain container types and other things.

35:57 Right? Yeah. I think if you look at the one Python of, there should only be one way to do it, most people say, but there's multiple ways to do that. Like, I can use a dictionary, I can use a set, I can use a list and just search for unique values.

36:08 It's code in a string, and you can parse it every time.

36:11 Yeah, the choices you make, the abstractions you choose communicate a certain intent. When you choose to use a set that tells me I can, iterate over it, there won't be any duplicates, and I won't be looking things up by key.

36:27 When I think of a dictionary, I think of a mapping from key to value. The keys must be unique. But if all I care about was the keys and no values there's added cognitive, just dissonant. Why do I have values for the dictionary there alone?

36:41 It doesn't make sense.

36:42 Yeah, it goes back to the principle of the Stochman. You get surprised, like, this dictionary is being used as a set. I get it now. And if you're not addressing that as you find it. You're just kicking the can on to a future maintainer. We then have to add that to the 20 other things he's trying to keep track of throughout the maintenance of the program.

37:02 Yeah, you mentioned that one of my big pet peaks. So additional essay is a mapping from key to valve. That's typically a homogeneous mapping, every key, same type and every value of the same type. But we really, really, really love dictionaries for things like JSON responses or relationships of data, and that can be so detrimental to maintain a really of code basis. If you're not careful.

37:27 The problem is a type checker isn't the greatest. It saying, oh, this is a Dict.

37:33 Some of the keys are strings, others are ints, others are decimals, and the values are all over the type. You can use a type stick to try to get around that. But really, what you're talking about is a relationship of data.

37:45 When you talk about dictionaries, you're getting into the. Well, if I see, let's say I'm code review not only but this in concrete terms. I'm code reviewing some code. I see someone accessing dictionary, and the key is it. I have to go look at all of where that dictionary was created and modified to make sure that food is actually a valid fuel in that dictionary. I have no guarantees. Just if I see Dict and a type checker. I'm sorry, type annotation. So again, it's that trolling through the code base. This is actually coming from an API. Now I have to go read the API, and I can effectively code review code or maintain code if I'm just reading that through without doing that every time to make sure something hasn't changed.

38:29 So in this case, I'd say user data class.

38:32 You had explicit fields. You can clean up your tooling. If you mess up the field, you aren't expecting to see new fields get created and 95 of the time.

38:44 Like, I would prefer a data class to a dictionary. If I have heterogeneous data almost all the time.

38:51 Maybe get it back like a flask API call called JSON, and then just jam that star, star that thing into the data class. Something like that.

39:00 Yep. And I know you've talked about Pydantic on the show a lot. I love Pydantic.

39:06 Just to find a model, let it parse that JSON response and just build that data class forming throw an error if it's Invalid. And I really like that model of attacking programs.

39:17 I love Pydantic. I love how it tries to sort of be flexible. Like if I think we can fix this. If you had a a thing that is a string, but in the string, it really Parsable to a number, and it's supposed to be a number. I'll just go ahead and do that type of conversion for you.

39:32 If not give you a decent error message. It's really lovely.

39:36 So if I'm working with APIs, I love Pydantic for that reason that you just outlined. But I'll often convert it to a data class or Pydantic data class so that I can say this is a relationship of data, and I can kind of shape how a user uses that relationship of data throughout the lifetime of the code.

39:51 Yeah, it's an interesting tension on how much those models get used throughout all the tiers of your app and how much you want to keep them separate. Have you seen SQL model?

40:01 I believe it's called I do not. I have not.

40:04 Okay, here we go. You probably heard a FastAPI, right? Yeah, of course. Obviously that's where Pydantic got. I think it's a big boost. So, Sebastian, Ramiro has came out a few days ago with a this thing called SQL model, and already I think it's less than a week old. It has 4000 GitHub start, which is amazing, but basically it's a merging of Pydantic and SQLAlchemy.

40:26 Fantastic.

40:27 It has a SQLAlchemy using the work model and underlying has all the SQLAlchemy stuff, but its models are actually Pydantic models. Yeah.

40:36 And I think this is just another illustration of why thinking about what type things are, even if you're not doing type annotations, why that's so important? It goes back to how do we build mental models?

40:46 How do we build these abstractions in our brain that we can rely upon as we work through a code base?

40:52 I do think this is interesting in that you could use the same model at the data level, internal to your app, and then you could even use an API level. But there's also people are saying, but maybe that's not a good idea. Maybe you want to separate those in interesting ways for one can change and the other doesn't have to change. But I see a lot of value to the thing. It looks.

41:12 I mean, the answer to that, I would probably be it depends if your application needs some to be the same.

41:18 Willy surprised make them the same if they have different reasons to change. This is something I see a lot too. We all get the dry principle. Don't repeat yourself and grind in our head and we think, oh, source code looks the same. We must deduplicate it. But if that source code can change for different reasons, you're going to add more headaches by deduplicating it. You're going to start adding special cases. Well, this thing needs to change, but the other thing doesn't. How do I reconcile that special cases and soon you've become the very thing you've sworn to destroy. As you build that out?

41:51 It's just littered with special cases as you're trying to tie together two things that have two separate contacts. So if you want to use something like SQL Model for an API level and your internal data model, ask yourself, do these have different reasons for change?

42:07 Maybe in the future. And a lot of this is just empathy for the future. When I talk about maintainable code robust code, you have to have empathy for those future maintainers purchase it from their shoes. Are they going to want to migrate some time and maybe change the API that they present to customers or users or other developers?

42:26 Do you want to change your database at the same time? If so, keeping together. If you want to keep that separate so that you have certain migration paths, maybe you keep them separate. And so it's a dependent case by case. But again, everything comes down to those first principles of what are you communicating with your intent when you make that decision in your code?

42:44 Like we've touched on you're building different things at different times under different constraints. Are you building Instagram API, which millions of people and apps are using, or are you building something really quick so you can get that app to work for marketing for the next thing for the week?

43:00 Right?

43:00 Yeah.

43:01 If it's the second one, then you don't want to worry about too much abstraction, and you just want to go. I just need these to be in sync.

43:07 The data goes here.

43:08 We're going to send over the we're gonna be good, right? This is going to fly. On the other hand, if you're building something incredibly consumed and long lived, then maybe you have different design patterns and care about it.

43:18 And I think it's important to think about organizational boundaries.

43:21 Is the users, the consumers, the actors, and your use cases?

43:26 Is there an organizational boundary separating in the Instagram case, the people we're going to be using My API I have no control over in my organization is general public as far as I'm concerned. If it's just the one on my team using it, we can work through. Okay, I'm changing this API. Let me help you through that. But if if someone outside your team or outside your organization, you're going to have a backwards compatibility to think about, you're going to have all these vast amount of things that people are going to complain about, and you need to think about that and say, okay, can I make these changes? And this is good for my user base. Even though I can't control them, the best I can do is entice them. You'll never be able to force someone to use something they don't want to if you don't have that control over them.

44:09 That's why we have open source forks. One of the ideas that I think comes up in this whole story. And you talk a lot about, like, inheritance, both inheritance in terms of class hierarchies and even multiple inheritance, but also maybe the more traditional interface style with protocols, just so that you can express type stuff separately in interesting ways without coming up with inheritance beasts and whatnot So one thing I did want to give people a quick shout out. So we've all heard of SOLID principles, and I really enjoyed the SOLID principles that I thought they were super interesting. I still do. I think they're pretty great. I recently came across a presentation and I counted over on Richard Campbel s show .Net Rocks, which I know people probably don't care about that Net, but they actually talked about this thing called Cupid, which is an alternative to Solid.

45:03 I have to go check that out.

45:04 And it was super. It has nothing to do with .Net. It's just like software in general patterns. Like, what do we know now? 20 years later? That doesn't necessarily make so much sense for Solid. And what is the alternative? And he has this really cute name. So as people are thinking about maybe those kind of levels like that's pretty interesting to check out. Yeah.

45:23 I like thinking of SOLID again, going back to first principles. The things that I like about SOLID are there things I also think get a bad rap because they're associated with heavy, heavy O code bases that have class hierarchies that are just unmanageable.

45:40 Probably multiple templates in there somehow.

45:43 And we kind of grouped these things together. So an example, one of the most misunderstood ones, in my opinion, is the list of substitution principle of the solid principles, and it applies when you're doing duck typing in Python, which has nothing to do with class hierarchies. It doesn't have anything to do with interfaces or things like that. It's a turnout substitutability. And so when we talk about duck typing in Python, it multiple types can represent some behavior. Maybe this or an addition method and a subtraction method. Can they be substituted into this function that's expecting addition and subtraction?

46:18 So my I do addition and subtraction on dates and times. Another might do it on imaginary numbers. But the concept of adding and subtracting things you can read that right. Something like that.

46:28 And so things like list of substitutable principal. So how do you think about substitutability from your requirements, from your behaviors, and you can get value from that without ever touching a class. So I think all too often we lump those solid principles to a strict OO of 90s the or the the knots and think that they aren't as useful. I'm going to have to check out, keep it, and see how that's changed as well. I would expect that a lot of the same first principles are true, but they kind of reframed in a way that is more applicable to help you program today.

47:03 I don't remember all the details exactly, but something roughly along those lines. I think that makes sense. So one thing I do also want to just sort of get your thoughts on before we leave it completely in the dust is you talked about a lot of times you've got a dictionary and it's supposed to represent some stuff.

47:20 Maybe it's a response from an API where it says here was what you requested, here was the status code. Here's what the cloud cost of that is. And here's something that looks like that.

47:31 Maybe it doesn't make sense as a dictionary, maybe gets moved into a pydantic model or something like that.

47:36 Right.

47:37 But the other one is I have 10,000 users, and I want to put their email address as the key and their user object as the value. And given an email address, I want to know super fast, which user is that, or do I have that user at all? In that case, the dictionary makes a lot of sense.

47:51 Absolutely.

47:52 There's a super big difference to say, the keys represent the same thing across different results, whereas the same data structure a dictionary represents this heterogeneous, like, not really a class, but kind of a class.

48:07 How do you position those so that beginners understand that those are completely unrelated things that need to be thought about separately, but they kind of appear the same encode, because when I'm talking to people about dictionaries and I'm teaching, they're like, well, why would you use a dictionary here? Because over here you were doing it this other way, but now it's like a database. Like, what are you doing with it? This is weird. Why are these not by the same, but different.

48:28 So here's how. So for the key mapping value, I would say there's a few use spaces you're going to have. You're going to either iterating over the dictionary to do something on every element, or you're going to be looking at a specific element.

48:41 And that element that lookup is dynamic, typically passing in some value. Maybe it's a variable that contains the email address in your example, or something like an API response, something that might be more heterogeneous, something that should be a data class or pydantic model.

48:56 What have you you typically aren't. Iterating over every key. You're looking up specific keys but

49:02 It's static to the index, meaning you're passing in string literals between the square brackets.

49:08 This is the key I want on this circumstance, the name, the age, the date of birth.

49:14 Right.

49:14 And so you're looking at specific fields, and you're building a relationship between different key value pairs. A name, age, and date of birth is a relationship called person with your dictionary you're doing, or with your mapping, your key value mapping. There's no relationship between email one, user object one and email two user object two. The relationship is just from key to value, and that's typically what I explain to people trying to discern the difference.

49:39 That's a really interesting way to think about it, because, yeah, you will almost always use static literal string when it's the heterogeneous API response style, and you will almost never do that.

49:50 Why would you say quote type in the email address? You would just have that object you wouldn't need.

49:56 You would just never create that structure in the first place, which I think is really interesting. So, yeah, if you're dynamically passing in the of keys, it's probably all the same object, but different ones of them, as opposed to an API response.

50:10 And I think that's the heart of why I really wanted to write about Robust Pythons. So I've been programming for a while now and I make a lot of decisions. As a senior engineer, I have already started to step back and ask myself, Why do I make these decisions? Why do I choose a dictionary over a data class? Why do I choose a class over a data class? Why do I choose an acceptance test over a unit test?

50:30 Why do I choose to do a plug in here, but a dependency injection somewhere else, and I just wanted to start documenting. Here's why I make decisions I do here's. Why? What intent? I'm trying to communicate through the hopes that we start normalizing that conversation more in our field. Why are we doing the things we do as a beginner? It's frustrating. You use a dictionary here why?

50:54 How did you ever know to do that?

50:55 And so I wanted to really capture again. I said it a lot. The first principle, why am I doing the things I'm doing? What are the things that I often give his advice to junior programmers, intermediate programmers, too, and senior programmers. It's just it was really enlightening for me to really step back and try to dissect why I do the things I do in Python and come up with a few of the valid engineering reasons behind it and try to frame that in terms of maintainability.

51:23 Yeah, I think that's very valuable. I think it'll help people learn because it's not enough to see it by example. Well, these both kind of work like, yeah, but they mean something different. They totally communicate something different. A couple of thoughts from the livestream owner Share Kim Benwick out there says, fantastic advice about having too much configurability. Far too often I prematurely de dupe similar blocks of code and then find myself adding special cases I wouldn't need it sounds like what you are saying as well.

51:53 And so what I would say is the there's a difference between policies and mechanism. The policies are your business logic and the mechanisms or how you go through some 'logging module' in Python is fantastic. You're logging the logger module doesn't care what you're logging, how you're logging in when you're logging it. It's just data mechanism.

52:10 Devan, who cares?

52:12 It's all the same, but what you're logging is your policy. And so very often I try to find a way to say, okay, how do I make my mechanism deep duplicate is those are what have reuse. My business rules are going to change. They're going to change for different reasons. I want to make that simple to define business rules, but keep the mechanisms reusable. You want to be able to compose those mechanisms.

52:33 Get also on the live stream.

52:35 Mr Hyper Magnetic says, Is it faster to look up a user or whatever from dictionary rather than iterate over list? And I think, boy, oh, boy.

52:45 If you've been down that path, that's different.

52:48 This is coming from my own advice from a C++ programmer who really care performance, measure it. Use cases can be surprising if there are times where a contiguous block of memory with a binary search is faster than a dictionary lookup, but you will not know until you measure.

53:04 There are also things that need to be fast in our program, and there are many things that don't be fast in your program. I think support being aware of which one of those are.

53:13 You don't want not to sound cliche. The whole premature optimization is the root of all evil breaking that down of the bits. You don't want to just optimize everything because it takes time to optimize sense and it can opt you skate your code optimized. Why am I entering over a list when a dictionary lookup can be just as useful? If you measure the parts that need to be fast and you say this is slow and this way is faster, as long as you have a comment and a good commit message that says why you're doing the slightly obfuscated way or breaking that log these surprise.

53:48 As long as you have those bread crumbs for someone to go, oh, it's first speed. This is critical performance group of my application. Maybe it's the web request handler or something in a database engine. You want to give those breadcrumbs so people can learn why you made the choices you did the last one. You want someone in five years to say

54:05 Oh, they were entering over list because they don't know that a dictionary exist. Let me go change that and slow down your app. But without that measurement, without that data, it's a fruitless endeavor in my opinion.

54:16 For a lot of data in general, a dictionary look up would be a lot faster. But if what you have to do is every time create the dictionary and then do the lookup, probably that act of creating the dictionary erases the speed up because you you're kind of looping it over anyway, right? It certainly depends.

54:35 Yeah. Don't take what I said is prescriptive measure, measure, measure.

54:40 Yeah, absolutely.

54:41 And the invoking just your domain line application with -MC profile is super illuminating. How long have I spent in a function? How long time did I call this function? Where is most of my execution time spending? And it might be that the bottom neck if you think you have is somewhere else complete?

55:00 Well, yeah, that's actually a really important point as well. If you do that C profile and you find you're spending 5% of your time in that part you're trying to optimize. If you could make it go to zero, it's still going to be 95 fast.

55:14 Right? So it might be able to be sped up significantly, but it might not actually be German to making your program feel any faster because it's like optimizing nothing, basically.

55:25 And I don't want to misrepresent go back to the question on the live stream. Typically, a dictionary lookup will be faster than iterating over a list. I expect that most of the time I would not rely on that.

55:36 All of the time we have the same measuring, measuring, measuring, measuring. I've definitely been in the place where I'm like, oh, this 6000 lines super complicated thing that is hard to understand and work on. This is where it's slow.

55:49 And no, it was just some other wrong data structure and some other very simple part. But if I didn't measure I would have gone and tried to rewrite the part I was, like, barely understanding. I would have probably broken it and still would have been slow.

56:01 Yeah, I forget who mentioned the order of writing things is you make it correct and you have test around it so that you know that it stays correct. You then work on making it clear, and then you make it fast if it needs to be. And that last part is optional based on is this slowing actually slowing me down? Is their business value on losing?

56:19 There are certain things I work with.

56:22 We're publishing something once a day. Something takes another five minutes. I don't really care. Like my time is spent. Better focus on parallelization things of that nature. And so it's just really understanding why things are slowing down and where there's a little bit a tangent of robust Python, but it's an interesting question to bring these.

56:42 Is that five minutes? Are you waiting in that five minutes?

56:46 You kick it out, you go, yeah, exactly. Then you probably don't care care at all. But speaking of time, I'm sure there are a ton of people listening that are like, I would love to take some of these ideas, maybe using mypy or putting more types or looking at stuff like Pydantic and so on. But at the same time, there's a lot of pressure on me to just from the time I received the request to get some feature done until the feature is done and the people making that request managers, business owners, clients, whatever. They don't care that much about robust Python. They want minimum input of money and time for output of features.

57:29 But anyone who has done that for any extended period of time knows that that is a net negative. How do you address that? I'll give you some thoughts on what I've done, but how do you address that?

57:40 For people who the way I am receiving that I want to frame the conversation for self, it is your duty to deliver value as quickly as you can. It is also your duty to make it so that your maintainers can deliver value at the same speed.

57:54 So if you're making if you're delivering value today but hindering your ability to deliver a month, three months from now, that's a problem. Now, I'm not saying jeoparadize your MVP or go to market, but making sure that the business I understand I can do this. I'm not going to be able to do this again. The other part that I say that is just practice, practice with these sort of concepts and apply them in small places. You can do things increment. Maybe there's a couple hundred lines of a library. You have just great type annotations for that or make it more extensible. Find small little winds here and there.

58:29 And as you keep working out, just as Boy Scout rule, leave the code base cleaner than you found it so if I'm going into something, you know what? These assumptions don't hold up anymore. I can't fix it all right now. But I can do one incremental step that makes it easier. The next time I'm in here, I take those little bites over time and slowly morph into something. It's like, okay, as we start slowing down delivery, I'm starting to make those improvements, and then we start speeding up back again.

58:55 Because there is a real tension between deliver it now and how making sure that you are going to be able to deliver hastily in the future. And by no means am I advocating gold plating where make everything maintainable. Now, there's typically a business behind this, and you don't want the business to fail. You need to be able to be able to ship early and often. Is that often part that is often the tricky part. So practice, practice, practice and target. Where you're applying these ideas.

59:27 You don't have to put all that flexibility there first, right? You write it one way, you get it out, but then you have refactoring tools. You have continuous integration. You have things that allow you to make those changes as long as you keep using them. And as long as the code doesn't get too bad.

59:41 And here's one of the things I've always like seeing. I forget who said this. There's a great code like to make a hard change. First make the hard change easy. That's the hard part. And then go make the easy chain.

59:53 When you see a lot of teams do some sort of estimation of their stories, how big will this effort take? If you see consistently high estimates, this is always a large. We're always doing large and extra large.

01:00:07 Ask yourself, what would it take to make this small? Sometimes you Fibonacci numbers, like, if this is an eight, what would it take for me to make this a three? Some of your listers, maybe. Like, what are you talking about? But estimation, we're estimating effort, right?

01:00:22 Yeah. Traditionally, way, way back in the day, people used to estimate in hours. How many hours is this going to take? You like, that is a granularity that we don't have. I can't tell you, is it 17 or 18 hours? I can tell you that 2 hours or two days, but I can't tell you whether its 17 hours.

01:00:39 So when you get large estimates again and again, ask yourself, Are these estimates because of necessary complexity? Complexity that's inherent to my domain, or is it accidental complexity? This built up over time as necessary complexity?

01:00:54 The deep workings of a neural net or flight controller software like these are inherently hard problems.

01:01:00 I can't make that simple, not without making a whole lot of money revamping an industry.

01:01:05 But I maybe a tax preparation. Right? There's a minimum set of complexity to just figure out what that answer is, because the rules are complex.

01:01:15 But let's take that tax if I have to maybe add a new rule, and it's just a simple multiplication somewhere.

01:01:23 I shouldn't have to add 20 to 30 files of changes just moment. That one thing, because it's going to happen again and again and again. So what are these common cases? You know, we keep saying this is a large effort. I don't make that smaller. And then you say, oh, if we just restructured things here, you often find the same amount of effort to go make that hard change easy will pay off in one to two cycles. I was like, okay, now that we're doing this again and again and again, like, yes, I took an extra day longer or a week longer and again, be mindful of your schedule. I'm not saying skip schedules, intention, but find that wiggle room where you can band together with your team and find those areas. This is a place that we can get some change back. I often like to set junior engineers on the team at these sort of tasks. They get more familiar with the code. They often don't have as many responsibilities as a senior engineer. And then just get that understanding. And they feel like I've done something that really, really matters.

01:02:21 I really, like, not plan up job.

01:02:24 And that has worked wonders for me as well. It's just it's a win win.

01:02:29 And then you're delivering those features the same things you're doing again and again at a much greater speed, and that saves you time to go to the next thing we have to do.

01:02:37 Then my approach to this, I mean, it only works with certain groups of stakeholders, I suppose.

01:02:44 But when somebody would ask me, how long does this feature take? It wasn't like, Well, here's the essence of the feature.

01:02:50 Like, it will work, and you can click the button. Here's how much it'll take to add error handling. Here's how much it'll take to add logging. Here's how much it'll add to make that work in continuous integration and have tests. And here's how much we got to do to not add technical debt while adding this feature. Right? Instead of presenting, like, well, here's your menu.

01:03:10 All I want to feature, just give me the feature. Right.

01:03:12 I just say, okay.

01:03:13 Well, if the feature is 3 hours, the other or 2 hours, like, 5 hours, it's gonna take me 5 hours or whatever metric used. That is what delivering it means in a professional setting is that it is done at its minimum path.

01:03:28 Zero errors and zero bad data will execute, right? They're not the same, but it's easy to get sucked into the minimum. How quick can I do this? I bet I can do it an hour if I really fly. Right.

01:03:38 I think that's a trap we often fall into we're as engineers.

01:03:43 It's our responsibility to provide accurate estimates for what's best for the business.

01:03:48 And if you say I can do this without tests, what you're really telling people are I can do it, but I can't guarantee it will keep working after I deploy it, because I have no way of knowing I've removed that visibility of tests.

01:04:00 Tell me this important thing that you really wanted now. No idea if it's working now. Sorry. And once you start framing it like that, you go, okay. Yeah.

01:04:10 Okay. Let's see what we can do. And you might need to have some hard conversations with the people who manages it. Might need more head count. We don't we can't take as many projects we can't.

01:04:22 We can't take as much complexity, maybe maybe a worldwide pandemic is happening, and we've been severely impacted. And I mean, you can't some degree you can't fix poor management of something, but you can coach your way up to try to get out of some people very receptive to that. Some people are not. And in the places that aren't. I mean, you have to ask yourself, is this something that I'm going to live with, or is this something that bothers me? And I need to think about other things?

01:04:49 Well, I think you're probably in a pretty good space if you're making that clear to people. So I think if you're proposing a feature or how long will it take you to this feature, then you just include what that means for a feature to be done.

01:05:01 It's not like zero bad data, and it works sometimes it works for real.

01:05:08 The harder story is we've gotten into a bad place. We need to not add features and just refactor improvement. That is a different story than what does that mean?

01:05:18 And that's a really tough. If I'm a business person, I'm never going to say yes. Please stop feature delivery and making money for us to go build the same thing we already have. That's a tough sell to a business. And I really think just an incremental approach. And again, I mentioned earlier targeting strategic. What are your most used libraries or parts of code. What are the areas with the highest bug count? What data to that? Hey, we've had five customer complaints about a crashing their system. They're linked to this one piece. We start doing this, we save X amount of money, and we would have kept that one cost. That's the sort of thing that speaks. When you're getting against those somewhat unrealistic business pressures, you have to speak on their level. And what I'm saying is a little wrong because it's not an us versus them type mentality. You're in this together, the ushers them at some time, but you need to work together, make them understand. Here are the business implications of the choices we're making. Exactly.

01:06:15 So you're the engineer and they're the business people. But you got to just put the situation in terms that they appreciate and they can decide if it makes sense for them. I would also add people out there listening that if the answer is almost never can we clean up technical debt and make this better? You probably will end up with the team over time who is full of people who don't care about erasing technical debt and making it better. And it's only just going to get worse and worse in Combinatorial Wave.

01:06:45 It's not where the top engineers want to be, and they don't get excited about coming to work, where they have to, like, sneak in something that's unreliable and cram new features into broken, ugly code.

01:06:57 And something I've learned over a long time. Programming is almost every technical solution.

01:07:02 There's a people or process problem at the root of it. There's a lot of things that we think, oh, code can solve this. If I just refactor this, everything will be great. But there's a people aspect of this, and I'm not putting down those people, like everyone around you is living, breathing human being. They all have their own hopes, their own dreams, their own obstacles, their own places in life.

01:07:21 And we have to recognize that sometimes you have to work with the people in order to solve that technical problem.

01:07:28 So much for the mythos of the software engineer in the basement by themselves.

01:07:32 I think that stereotype is long dead for the most part.

01:07:36 I agree. I agree.

01:07:38 All right. Well, we can go on and on, but I think we're out of time, so thanks for all your thoughts on this and books. Interesting. We'll talk more about in a second, but before we do, if you're going to write some of this code, work on some of these ideas. What editor are you up to?

01:07:52 Yeah. I use VS Code because I still jump between languages quite a bit. I like the flexibility of that for one to 2.

01:08:00 Cool. Then notable PyPI package.

01:08:03 I'm going to call up Stevedore package used for plugins doing plug in development. I think it's really neat how it uses the packages entry points so that you can deploy plugins as separate pip installable packages.

01:08:17 Just something I wish more people knew about.

01:08:19 That's.

01:08:20 Awesome.

01:08:20 One of the things we didn't cover, but is also part of this robust story that you tell is about extensibility, about plugins, about all those kinds of things. Right.

01:08:29 Yeah. And that just goes back to making the common case of simple. If you're doing something again and again and again, make it easy for people to do that an extensible codes way to do that.

01:08:39 Like that tax example, if you could plug in the formulas that are applied in this place, and then you just add it in and maybe it just picks it up and goes, that would be ideal. Yeah. All right.

01:08:48 Awesome.

01:08:48 Good recommendation. One that I had not heard of. Right. So people are interested in this idea. I assume that they can check out your book, maybe tell people about that real quick. And then what else would you provide as resources or places to get started.

01:09:01 Yeah, I think just talking to senior members on your team, talking to junior members on your team, learning how people use the code around you, keeping an ear for your stakeholders, your co developers.

01:09:16 Think about the code you write. Think about the implications it has.

01:09:20 Just listen.

01:09:21 Most of this is empathy. I thought I was writing a technical book. I wrote a book about empathy instead. Just didn't know it. Put yourself in other people's shoes and think about how they're going to be recieve code a month from now, a year from now, five years from now.

01:09:34 And again, a lot of talk you'll see at PyCon or the other Python conferences, you'll see these nuggets of truth throughout them.

01:09:44 Ask yourself, why are they making the decisions they do? And see if you can really understand the kind of the root cause of why they're using this feature, especially as new features come out with in a lot of great ideas in here.

01:09:57 And I had a fun time talking about that.

01:09:59 Me too.

01:09:59 Yeah. Thanks for being here, Pat.

01:10:01 Been an absolute pleasure. Thank you so much.

01:10:03 You're welcome.

01:10:04 Bye bye.

01:10:05 This has been another episode of Talk Python to Me. Our guest in this episode was Patrick Viafore, and it's been brought to you by Clubhouse.

01:10:14 Master Works and Assembly AI.

01:10:16 Choose Clubhouse.IO for tracking all of your work because you shouldn't have to project manage your project management. Visit 'talkpython.fm/clubhouse.

01:10:24 Clubhouse make contemporary your investment portfolios unfair advantage. With Master Works, you can invest in fractional works of fine art. Visit 'talkpython.fm/masterworks'. Do you need a great automatic speech-t-otext API? Get human level accuracy in just a few lines of code.

01:10:42 Visit 'talkpython.fm/assemblyai'. Want to level up your Python.

01:10:47 We have one of the largest catalogs of Python video courses over at Talk Python.

01:10:51 Our content ranges from true beginners to deeply advanced topics like memory and async. And best of all.

01:10:57 There'S not a subscription in sight.

01:10:59 Check it out for yourself at 'training.talkpython.fm'.

01:11:02 Be sure to subscribe to the show, open your favorite podcast app, and search for Python. We should be right at the top. You can also find the itunes feed at /itunes, the Google Play feed at /Play and the Direct RSS feed at /RSS on talkpython.fm.

01:11:17 We're live streaming most of our recordings these days. If you want to be part of the show and have your comments featured on the air.

01:11:24 Be sure to subscribe to our YouTube channel at 'talkpython.fm/youtube.

01:11:28 This is your host, Michael Kennedy.

01:11:30 Thanks so much for listening.

01:11:31 I really appreciate it. Now get out there and write some Python code.