#411: Things I Wish Someone Had Explained To Me Sooner About Python Transcript

00:00 What advice would you give someone just getting into Python?

00:02 What did you learn over time through hard work and a few tears that would have really helped you?

00:07 It's a fun game to play, and we have Jason McDonald on the podcast to give us his take.

00:12 This is Talk Python to Me, episode 411, recorded March 2nd, 2023.

00:17 Welcome to Talk Python to Me, a weekly podcast on Python.

00:34 This is your host, Michael Kennedy.

00:36 Follow me on Mastodon, where I'm @mkennedy, and follow the podcast using @talkpython, both on fosstodon.org.

00:43 Be careful with impersonating accounts on other instances.

00:46 There are many.

00:47 Keep up with the show and listen to over seven years of past episodes at talkpython.fm.

00:52 We've started streaming most of our episodes live on YouTube.

00:56 Subscribe to our YouTube channel over at talkpython.fm/youtube to get notified about upcoming shows and be part of that episode.

01:03 This episode is brought to you by Sentry and us over at Talk Python Training.

01:08 Please check out what we're both offering during our segments.

01:11 It really helps support the show.

01:13 Jason, welcome to Talk Python to Me.

01:16 Thank you for having me.

01:17 Yeah, it's great to have you here on the show.

01:19 I'm looking forward to talking about a bunch of things I wish people taught you about Python sooner.

01:25 I think there are many, many things that fall into that category.

01:29 And part of that, I guess it's true with all languages, but I think it's maybe a little extra true in Python because getting started is so easy.

01:37 But getting really good at it takes a lifetime sort of thing, you know, and so there's like this big mismatch of how accessible it feels.

01:44 But then what you can get out of it if you go further.

01:47 Right.

01:47 So I think that we'll touch on that going further part, maybe.

01:49 Absolutely.

01:50 Yeah, for sure.

01:51 Now, before we get into all those things, let's start with your story.

01:55 How did you get into programming in Python?

01:56 Well, I got into programming by a route that I don't recommend to anybody.

02:00 I fell down a staircase, hit my head on the banister and ended all hopes of ever becoming a doctor.

02:05 Oh, so went from 4.0 straight A student, college level reading in high school to family pre-game material.

02:13 Two extra years to climb back up where I was academically a bit longer after that for other things.

02:19 But somewhere along the way, I discovered that I had a natural inclination towards an attack for coding and got into that because I wanted to make games.

02:28 I wanted to make educational games.

02:29 They had been part of my recovery.

02:30 So I'm like, I want to make games.

02:31 So I learned how to code.

02:33 And so I taught myself BB.net and then felt very constrained by the .NET ecosystem and broke out into Python.

02:41 Found my home, stayed put until I wound up going into a bunch of other languages.

02:46 I used Flash for quite some time.

02:49 I used C++.

02:50 I'm not coming out in whatever, Sandy, but Python is definitely near to dear to my heart.

02:55 Yeah, absolutely.

02:55 Well, if you started out in BB.net, I think it's a little less true these days.

03:00 But in the earlier days, that was a really different ecosystem than Python, right?

03:05 There was like, well, what is the framework for this thing that Microsoft recommends?

03:09 Like, how do I do the web?

03:10 I do ASP.net because that's what they provide me.

03:12 How do I do database?

03:13 I do this because that's what they provide me.

03:15 And you come into Python, you're like, wait, there's a thousand different ways to do this.

03:18 And I have to pick and decide and I can mix and match.

03:21 And it's both amazing and like.

03:23 Terrifying.

03:23 Yeah, it's terrifying.

03:24 Like, what do I make the wrong choice?

03:26 I've how do I decide?

03:27 I have no basis, right?

03:29 I'm both new.

03:29 So I have no basis for deciding.

03:31 But I also, I've got to decide to get started.

03:33 So where do we go, right?

03:34 Like, it's a different world.

03:35 It is where I first started running into trouble with the paradox that I wound up addressing in my book on back of that.

03:43 But the paradox that a lot of people find themselves in is like, I know all this stuff, but I don't know enough to do this thing.

03:50 And so you wind up with this weird little gap in a lot of training material because you have like the absolute basic elementary.

03:56 And the examples there are almost insultingly simplistic.

03:58 And it's like, oh, you just have to type this stuff into this file and you run the file and look at that.

04:04 You have a hello world.

04:05 Aren't you smart?

04:06 And then you go all the way to the extreme and it's something like super complicated.

04:09 You don't understand any of it.

04:10 And nobody sat and wrote down.

04:13 Here's how you structure your multi-Python project in a way that's going to, at least when I started, that wasn't documented anywhere.

04:18 Like no one had ever written it down.

04:20 So it's just like, what do I do now?

04:22 Yeah, it's really tricky.

04:23 A lot of the tutorials and stuff are like, you just put all the stuff into one file and then it's magic.

04:29 But you should never, ever do that.

04:30 Well, maybe in simple cases, but in realistic apps, like no realistic app belongs in one file.

04:36 Just like you wouldn't have one function or just like no functions, right?

04:39 You're like, not really.

04:40 That's not how you do it.

04:41 But so there is this, even when you're sort of hunting through tutorials, there is this mismatch of the simple way to show it, the simplest way to show it.

04:50 And then like maybe how you should.

04:52 Yeah.

04:52 Yeah, exactly.

04:53 It's the example I've given is that there's two types of book slash article slash tutorial.

04:58 out there.

04:59 There's here's basics of variables, voice and curls.

05:01 And then the next level after that is here's how to program a natural language recognition, OCR, AI, machine learning on a Raspberry Pi, your grandmother's base or running on a potato battery.

05:11 And there's nothing in between.

05:13 Yeah.

05:14 Yeah, that's true.

05:16 It does make it quite challenging.

05:18 All right.

05:18 How about now?

05:19 What are you up to?

05:20 These days?

05:20 Oh, gosh.

05:21 I'm actually kind of veering these days more into actually more into kind of the facilitation business analyst side of things more.

05:28 Interestingly, the last last couple of years, that's become a point of particular interest.

05:32 But, you know, in terms of coding, I'm working on I'm working on some interesting projects that, you know, at work building, you know, an API for an internal project.

05:39 And that's been that's been fun getting to get my hands dirty with FastAPI and SQLAlchemy.

05:45 And that's been that's been fairly enjoyable.

05:47 So that's been the majority of my programming bandwidth lately.

05:50 But it's fun.

05:51 It's it's the sort of project you hope you're going to get at work because it's an interesting topic.

05:55 You have enough love leeway.

05:59 That's where we have enough leeway to do things right and do things to some degree your way if you can justify it well enough.

06:05 And, you know, other tech lead on it.

06:07 So I'm able to make sure, like, you know, we're making good decisions early and paying down technical debt and building something that's really, really cool.

06:14 So it's it's a fun project.

06:15 I'm enjoying it.

06:16 Yeah.

06:16 Those kind of projects, they're they're fun, but they're also important.

06:20 Right.

06:20 Like if your work, if you work, if you work at a place where they say, you know, you need to use 10 year old technology and there's no opportunity to try new things like FastAPI and other others.

06:31 What you end up with is a whole bunch of people who that work there who don't want to learn any new technologies and don't are not deeply passionate about programming.

06:39 And it's kind of the self-fulfilling prophecy.

06:42 And I think it's cool that the place you're working at allows that.

06:45 Right.

06:45 Because I think it's it's one of the signs of a healthy tech, not necessarily a startup or whatever, but all these companies have like small software teams within it.

06:54 It's a sign of a healthy small software team, even if it's just building internal software.

06:58 Oh, definitely.

06:59 Yeah.

06:59 And, you know, and there's a place for older technologies.

07:01 Well, it's good to be well-versed in both and be able to move between it.

07:04 But I mean, yeah, if you're if you're just stuck in the past, then you either, like you said, get a lot of people that are afraid of change or are really wanting that change.

07:13 They're dissatisfied with the fact that you can't get it.

07:15 But on the other hand, if you just if you're only new technology, you'll just spontaneously combust because you can't.

07:20 You know, there's just there's so much constant invention and reinvention and research and unexpected bugs and blockers and whatever.

07:28 It's like sometimes the nice thing about a 10 year old library is that there's less bugs because it's years old.

07:33 Or you can find documentation about issues or whatever.

07:36 You know, they have a term.

07:38 It's like a term in the software industry about this constant churn that's like going too quick and especially not appreciating stuff that's been around.

07:46 It's called JavaScript.

07:47 You're not wrong.

07:51 JavaScript is the embodiment of shiny object syndrome.

07:53 It is.

07:54 I mean, I'm not bashing on JavaScript per se, but the life cycle of some of these frameworks is just so brief.

08:00 You know, it's nuts.

08:01 Everybody wanted to use Dino for about 10 minutes.

08:04 I remember that.

08:05 There was a thousand articles a minute on Dino.

08:07 And now it's now we've moved on to other things.

08:10 And it's funny because you get a new technology like, OK, the topic of the minute, ChatGPT.

08:17 And everybody on LinkedIn is an expert in ChatGPT.

08:19 You know, have you noticed that?

08:20 Every other post is about ChatGPT.

08:22 And it's like, well, it's interesting, but it's just a bit early.

08:26 And the early adopters have a lot of egg.

08:28 They're still trying to wash off their face in the last six fans.

08:30 So it's like, let's see what happens.

08:33 But chances are, this is just an early hiccup.

08:36 Yeah, for sure.

08:37 So you mentioned FastAPI and SQLAlchemy.

08:39 And Yasser out there says SQLAlchemy is the best library.

08:42 SQLAlchemy is pretty great.

08:44 And it's cool to see Mike Bayer and crew updated to 2.0 and add async and await to it and stuff.

08:50 But there's also, especially when you're in Pydantic world, say with FastAPI,

08:55 there's a SQL model also from Sebastian Ramirez that says, hey, we are already doing Pydantic for the API or front-end exchange.

09:06 Like, what about making that your database?

09:07 Did you all evaluate or consider a SQL model?

09:10 I did.

09:11 It's one of those things like, this would be great.

09:13 And I think we can move towards that later.

09:15 But sometimes when you're paying down technical debt, you have to make a distinction between,

09:19 do I want to make the monthly payment as it were more than a monthly payment?

09:24 Or do I want to try and pay it all off in one thing?

09:27 And sometimes when you're having to budget that time, you know, for us, it made the most sense to migrate from the 1.0 that we were working with earlier on to using, you know, SQLAlchemy 2.

09:38 And so there was a relatively smaller set of changes that we needed to make.

09:41 I see.

09:42 Moving to something like a SQL model would be great, but it would also require us to rewrite some major things.

09:49 And so that's one of those things is like, well, this might be a good option.

09:53 I mean, it looks really cool, but let's defer that for right now so that we can actually get some stuff done.

09:58 It's a constant balancing act.

09:59 Like, do we pay down technical debt or build new features?

10:02 And the answer is yes.

10:02 You have to do both, but you have to keep it in balance.

10:06 Pay down the credit card, but also make sure you're buying your groceries, you know?

10:09 Yeah, absolutely.

10:10 I feel like if you wanted to go that way, the change is pretty straightforward.

10:14 But if you already have it all set up in the other way, again, it's like, eh, what is the value?

10:18 Just one thing that kind of timely as of a couple days ago, let's see, as of two days ago, the Pydantic folks, Samuel Colvin and Terrence Dorsey,

10:27 posted, hey, we're excited to announce the first alpha of Pydantic v2.

10:32 And I had Samuel on like a year ago to talk about what they were doing there.

10:36 Wow.

10:36 But the big news is, the headline news in here is, they rewrote much of the core of Pydantic in Rust.

10:44 And now Pydantic version 2 is 5 to 50 times faster.

10:48 Nice.

10:49 That's cool, right?

10:50 That's gorgeous.

10:51 Thank you.

10:52 This has just put more work on my plate.

10:54 That's okay.

10:54 This will be good.

10:57 I'll have to bring this up at work.

10:58 It's like, hey, we should look at this later.

11:00 Yeah.

11:00 I mean, I don't think that there's any, I don't think anything will, obviously nothing will change for your SQLAlchemy side.

11:06 But even for FastAPI, I'm sure that this is going to get rolled in in a smooth way.

11:10 But it does mean that all that data exchange should just go way, way faster, which for basically no effort on users of things like FastAPI and SQL Model and Beanie and all those.

11:21 So that's cool news.

11:22 Yeah, definitely.

11:23 That is neat.

11:24 Other than the fact that if you do a lot of work directly in Pydantic models, there might be a few small breaking changes.

11:30 But let's talk about our things.

11:33 The things we wish someone had explained to us sooner.

11:36 Yes.

11:37 And I guess we can start with this.

11:38 Like these things were motivated.

11:40 We'll come back and talk a bit more about this at the end.

11:42 But they were motivated.

11:43 This whole conversation was motivated.

11:44 This big, long book that you wrote here, Dead Simple Python.

11:48 Give people the elevator pitch and then we'll get to all things.

11:50 Yeah, exactly.

11:50 So Dead Simple Python is the book I wish I had.

11:53 And yeah, it's, I get teased a lot.

11:55 It's like dead simple and it's that thick.

11:57 But it's dead simple in the same way that Python is obvious.

12:01 It's obvious retrospective.

12:03 Things are dead simple.

12:04 Once you understand it, you look back and go, oh, okay.

12:07 That clicked.

12:07 That just makes sense.

12:08 A lot of things in Python are like that.

12:10 You look into it.

12:10 It's like, that looks really complicated.

12:12 How is this ASIC supposed to work?

12:14 And then you look back and go, oh, that's all?

12:16 Okay.

12:17 No, that's.

12:17 Yeah, yeah, exactly.

12:18 Why was I afraid?

12:19 Why did I put learning, put off learning this for six months?

12:21 Exactly.

12:22 It's, you know, things look more intimidating than they are in practice.

12:28 But basically this is the book for those persons who already know another programming language,

12:34 or maybe they've even been working with Python for a while.

12:37 Like they're familiar with the basics.

12:39 They do not want to have their hand held through the, you know, this is what a variable is.

12:44 And this is what a function is.

12:45 They've been through this probably more times than they want to admit to.

12:50 And they want to get into what's different about Python and how to use Python as Python.

12:56 I think that's kind of the core of the book is there's a big difference between Python code

13:00 that works and Python code that makes the most of the language and its patterns, what we would

13:06 refer to as Pythonic code.

13:08 And there's a big difference between the two because it's very tempting to come in and write Python as

13:12 BB dot that in my case, because that's where I started or write Python as C or write Python as Java or write Python as Ruby.

13:19 And we wind up holding ourselves back and having really convoluted and hard to maintain code because we don't understand the patterns.

13:25 So the approach I took with the book is let me explain why we do things this way.

13:30 Let's go all the way down into the internals and how this actually works under the hood.

13:37 So by the time you get to the forward with statements, you know, the meta class or whatever, you're like, oh, I understand what this is, why it's there, how it's used and when not to use it.

13:46 Yeah, that's a great service.

13:48 So you're saying if I created a class called like user factory and it implemented I user factory and then I would use dependency injection.

13:58 I might be doing that wrong.

13:59 You might be writing C# and Python.

14:01 Yeah, indeed.

14:03 Yeah, indeed.

14:03 This portion of Talk Python to Me is brought to you by Sentry.

14:08 Is your Python application fast or does it sometimes suffer from slowdowns and unexpected latency?

14:15 Does this usually only happen in production?

14:18 It's really tough to track down the problems at that point, isn't it?

14:21 If you've looked at APM application performance monitoring products before, they may have felt out of place for software teams.

14:28 Many of them are more focused on legacy problems made for ops and infrastructure teams to keep their infrastructure and services up and running.

14:36 Sentry has just launched their new APM service.

14:40 And Sentry's approach to application monitoring is focused on being actionable, affordable and actually built for developers.

14:47 Whether it's a slow running query or latent payment endpoint that's at risk of timing out and causing sales to tank,

14:53 Sentry removes the complexity and does the analysis for you, surfacing the most critical performance issues so you can address them immediately.

15:00 Most legacy APM tools focus on an ingest everything approach, resulting in high storage costs, noisy environments,

15:08 and an enormous amount of telemetry data most developers will never need to analyze.

15:13 Sentry has taken a different approach, building the most affordable APM solution in the market.

15:19 They've removed the noise and extract the maximum value out of your performance data while passing the savings directly onto you,

15:25 especially for Talk Python listeners who use the code Talk Python.

15:29 So get started at talkpython.fm/sentry and be sure to use their code Talk Python all lowercase

15:36 so you let them know that you heard about them from us.

15:39 My thanks to Sentry for keeping this podcast going strong.

15:45 So I like the idea of it.

15:47 Let's talk.

15:48 Let's go through the topics.

15:50 We both contributed a little bit.

15:52 Mostly, I'll throw in a few if we got extra time.

15:54 Yeah, yeah.

15:55 So the first one has to do with this, I guess, like really early stage writing code, like how do I create variables?

16:01 What are the deals with types and variables, but also types and objects?

16:05 Tell us about that.

16:05 Yeah, exactly.

16:06 How names and values actually work.

16:08 And this is something that I had to learn early, but I didn't fully understand.

16:13 So when I first logged into the Python IRC room back when I learned this language, you know, back in the Cretaceous period, I logged in and I, you know, I'm coming from a strongly typed background.

16:23 So I'm saying, okay, so how do I declare a data type on a variable?

16:27 Because I'm used to VB.

16:28 I'm like, yeah, dim in the name of the variable, the type equals whatever.

16:31 How do I declare a type?

16:32 It was before type hints existed.

16:34 And one of the guys in the room goes, you're a data type.

16:37 And I'm like, sorry, what?

16:38 And he goes on to explain, well, it doesn't quite work this way in Python, but I didn't realize just how different it was.

16:44 So because I just sort of had this vague, like, oh, you just have the name equals the value.

16:48 Cool.

16:49 Type still seems kind of important.

16:50 Maybe I should start doing Hungarian notation.

16:52 Don't do that.

16:53 By the way.

16:54 S Z username.

16:56 No.

16:57 Exactly.

16:58 Exactly.

16:59 Int age or whatever.

17:00 Yeah.

17:01 And I thought I was being super clever and I wasn't.

17:03 But eventually someone explained to me, I think it was actually Ned Batchelder.

17:07 Someone gave me the link to Ned Batchel.

17:08 And Batchelder's talk on this is that Python's variables work very differently than other languages.

17:14 So much so that some smart elics like to say, well, Python doesn't really have variables.

17:18 Yes, it does.

17:19 But they're variables in a different sense.

17:21 You have names, which are just like they're like labels and you can call it whatever you want.

17:25 But a name has a scope.

17:27 You know, so the name exists in the context of your whole program or in the context of your function or whatever.

17:32 And a name can be bound to whatever you want.

17:34 Imagine drawing, you know, basically having this label on a corkboard, thing like a conspiracy theorist corkboard.

17:41 You know, you have the tack in a label and you put a little string around it and you tie that string to this other thing over here.

17:48 And that's your value.

17:49 Your value has a tie.

17:50 It has.

17:51 It is one thing.

17:53 It's a Boolean or it is an integer or it is a string or it is a list.

17:56 We've all seen the error.

17:57 You cannot combine integer and string with plus operation.

18:01 Like they know what they are, right?

18:02 Exactly.

18:03 Those values know what they are.

18:04 And that's something that throws people off coming from JavaScript where everything's weakly typed.

18:08 It'll just try to coerce them together.

18:09 Python says, hey, these two values are different.

18:11 I'm not going to try to figure this out.

18:12 That's because it's working with the values.

18:14 The values have type, but the values don't have scope.

18:17 And this is where people get thrown off because if you say, you know, let's say I'm going to use the example here.

18:25 X equals spam.

18:26 If you were to say Z equals X, you're binding Z to the same value as X.

18:31 And then if you decide to change Z, that works.

18:34 X is unchanged.

18:35 Z is changed.

18:36 You might think, oh, cool.

18:37 Okay.

18:37 This is just, it just made a copy.

18:39 Not exactly.

18:40 Because we have this whole thing about mutable and immutable types.

18:44 Some things can't be changed.

18:46 Boolean, an integer, a string, a tuple.

18:48 They're not changed.

18:49 The value is never modified in memory.

18:51 A new thing is made instead.

18:53 But then some, some, and by some, I mean most.

18:57 Sorry about my clock.

18:58 Most types in Python, like lists and dictionaries and the objects that you're creating with your fancy little classes are mutable.

19:05 So if you create a list or a dictionary and you bind that to two different names, you modify, you add a value to that list.

19:14 It doesn't matter which of the names you use, you're modifying the value in memory, in place.

19:19 And now that is accessible from either of those names.

19:22 Yeah.

19:22 And that can be confusing, especially when you start to understand that functions don't pass by value.

19:26 They don't pass by reference.

19:27 They just bind the value to the name of that argument.

19:31 Pass by assignment is what we usually call it.

19:34 So you're just having one more handle, one more alias pointing to that value in memory.

19:38 So if you change on that argument, you're just changing your spot number.

19:42 Many languages have values that are like local and copied types of values.

19:49 So you mentioned VB.net, which like the integer and float numbers and things like that.

19:56 If you assign one, it's a copy of a thing.

19:58 Whereas Python, literally everything is a pointer, right?

20:02 This corkboard conspiracy theory thing, right?

20:07 There's no concept of, oh, I created this on the stack.

20:09 And when the function returns, it's gone.

20:11 It's everything is a, you know, even like a, just an integer is a pi long star down in the runtime.

20:17 Right.

20:18 That's a bit of a mental thing people got to get used to just like how much, because when

20:24 you come from a language like C++ or something, pointers and these reference type type of ideas

20:29 are like right in your face, right?

20:31 Like you see this, like I just said, pi long star, like the star means it points at things

20:36 without the star.

20:37 It's on the stack, right?

20:38 It's a, you are way down in those details.

20:41 Whereas in Python, you never see pointers.

20:43 Like you could imagine, like there's no such, like we don't even have pointers in Python.

20:46 Right.

20:47 But the irony is like literally everything is basically managed in that on the heap reference

20:52 type style of work.

20:54 So that's certainly something I wish I knew.

20:56 Yeah.

20:57 And, and of course, then people come from C++ and say like, oh, cool, that's easy.

21:01 So it's all pointers.

21:02 Well, once again, Python being Python, not quite because it's actually technically called a

21:07 reference, not a pointer.

21:08 And that's important because a Python reference or a Python value has two things.

21:15 It has, yes, it does have a pointer, at least we're talking about CPython, but it also has

21:19 a reference counter, which is a self and integer.

21:21 And that's adding overhead.

21:22 So if you ever do actually go down and look at the sizes of things, you will notice that

21:28 your base value has this additional overhead.

21:31 Yeah.

21:32 two numbers.

21:33 One of them is your pointer and the other one is your, the other one is your, your reference

21:38 counter because it has a reference counter garbage collection.

21:40 Yeah.

21:41 Often a lot more.

21:41 I think trying to, I guess I could check, but either an integer or a single character.

21:48 I can't remember which is 28 bytes.

21:50 Right.

21:51 And that's in C, the single character would be one or two bytes and the integer would be

21:56 two, four or eight.

21:57 Right.

21:58 And so there's just a big difference in the way you got a sort of like the implications

22:02 of that.

22:02 So related to that, kind of moving on to the next, before we move on, the other thing that

22:07 you mentioned, and I think it's also a really big deal in this area is the scope of a variable.

22:13 And so in some languages, you have very fine grained control of the lifetime of a variable

22:19 by where you declare it.

22:21 So for example, if you declare a variable in a C++ or VB.net or whatever in a if block,

22:29 when you leave the if block, the thing goes away.

22:31 Like if there's curly braces around it in a lot of languages, like that is the lifetime of

22:36 the variable innermost bracket of curly braces, like outside of a loop, the thing goes away

22:42 in Python.

22:43 We don't have that.

22:44 And that's a, you know, kind of the variable persists and lives after afterwards.

22:48 Right.

22:49 Which is interesting.

22:50 Yeah.

22:51 difference.

22:51 Yeah.

22:51 Because the garbage collection is something you're never supposed to have to deal with.

22:55 So even if you call delete on something, it's not necessarily guaranteed to be cleaned up.

23:00 And when you expect it, like the language is going to handle things when it, when it sees

23:05 fit for whatever reason.

23:06 So yeah, there's only to my knowledge, there's only three, there's only actually three scopes

23:11 in Python.

23:12 There's global level.

23:13 There's module level and there's, I don't even, that gets a little pedantic and then you

23:20 have function level classes are not their own skull.

23:23 Classes actually do not, a class exists in the global level.

23:28 And whenever you have an attribute sitting on a class or sitting on an instance is actually

23:32 in a dictionary, unless using slots.

23:34 I'm not going into that.

23:35 Actually sitting in a dictionary that is a part of that object in memory.

23:41 Yep.

23:41 It's special, but it works out.

23:43 It's just something you got to kind of get used to.

23:45 Definitely.

23:45 All right.

23:45 Next one.

23:46 Ducks.

23:47 Quank.

23:47 Duck typing.

23:49 Yeah.

23:49 And, and it's, this is where I went.

23:52 Oh, haywire.

23:53 When I, when I was doing that, that Hungarian notation, I mentioned it.

23:56 I'm thinking, I'm being so clever.

23:57 I'm just putting the, the name of the, or the type in the name.

24:01 And then I know it.

24:02 But unfortunately you're coming from a stronger type language.

24:05 You're actually, it's easy to underestimate just how much strong typing can actually hold

24:10 us back in some cases, because we started thinking about what a thing is rather than

24:14 what it can do.

24:15 And we see this in Java.

24:16 Like you have to have these complex inheritance patterns because you want to be able to say,

24:19 well, this thing is, it is this matches this interface.

24:23 You know, it, it does this particular thing.

24:25 Python's take is I really don't care what it is.

24:29 As long as I can interact with it, according to a particular kind of set of assumptions,

24:34 then I don't care what it's really doing under God.

24:37 And this can be really beneficial because it allows you to create new things that act like

24:41 integers, even though they're not integers or new things that act like Booleans that aren't

24:45 Booleans, new things that act like strings that aren't strings.

24:47 Python doesn't care.

24:48 It's just interfacing with those objects on the basis of how it can use them and what certain

24:57 things are going to return.

24:58 That's all it cares about.

24:59 So the concept of the duck typing being, it doesn't care whether it's a duck or a picture

25:03 of a duck or a statue of a duck or a moose in a duck suit.

25:05 As long as it can interact with it in the way it needs, it's good.

25:08 Yeah.

25:09 And that's very free.

25:10 And talks like a duck and quacks like a duck.

25:12 And it's probably a duck.

25:13 Yeah, exactly.

25:14 And it's very freeing once you get that because you don't have to muck about with all this weird

25:18 inheritance anymore.

25:19 It's like, well, it, it does what I need.

25:22 And that's the important part.

25:23 There is certainly less inheritance and less complex hierarchies in Python.

25:27 Not, not that there's not an important place for object-oriented programming and classes.

25:31 They're just often simpler.

25:33 A lot more straightforward, definitely.

25:35 And it frees us up to then the saying I coined a couple of years ago during a conference talk was,

25:41 was that you need to define objects by their constituent data.

25:44 So when you're writing a class or you're designing an object, you're thinking about what is it storing?

25:50 Like what, what sort of state, if you think in a functional way, or what sort of data am I really putting in here?

25:57 And you can design based off of that.

26:01 And then you can think about, well, how do I want it?

26:03 How does the system need to interact with that data?

26:07 But you don't really have to sit there and think about encapsulating behavior because that's not really the point.

26:11 The point is you can think about, like with the example I use is, we always see the example of,

26:16 I'm going to create a class called cat and a class called dog and they inherit from animal.

26:19 And that's actually a really lousy example we need to stop using because that's starting from the concept of behavior.

26:26 Well, it barks, it meows, it eats, whatever.

26:29 And in real systems, what we think about is I have a, you know, someone designing a game.

26:35 I have a monster.

26:36 Okay.

26:37 What is a monster?

26:38 Well, it's, it's a bad guy.

26:40 No, no, no, no, not what it does.

26:42 What is it?

26:42 Well, it's, that's a collection of statistics about the character, you know, like the, how, how much strength it has and how much health it has.

26:49 Okay.

26:50 That is not a monster.

26:52 That's an, that's an entity.

26:53 That's a, you know, that you don't have to put what it's doing into the, into that code.

26:58 It is this collection of information.

27:01 And when you start, when you shift your thinking to a data first mindset about object oriented, then you find that you can actually use it with functional paradigms that are sometimes a lot easier to reason about.

27:12 Yeah.

27:12 The, the grouping together of data is an important part.

27:14 I would say the most extreme example of duck typing in Python, maybe a couple of areas.

27:22 And they all seem to involve, revolve around the, the magic methods or the Python data types, like the Dunder methods.

27:28 Right.

27:28 I love those.

27:29 I could.

27:29 Yeah, I know.

27:30 So I could have a function, which I could call, or I could have a class that implements Dunder call is a callable, but it's actually an instance of an object.

27:40 Right.

27:40 And, but as far as Python is concerned, it doesn't matter.

27:43 It can call it and pass parameters to the thing it called.

27:46 Right.

27:47 And there's no real type that disambiguates those two things.

27:50 Like one is a function type.

27:51 One is a, you know, a class with a particular method on it.

27:54 Right.

27:55 But they work just fine.

27:56 And that's a pretty extreme example.

27:58 You know, another one would be, I have a list I can iterate, or I have something that implements the iterator pattern through the Dunder methods.

28:07 Right.

28:07 Like the Dunder iter and so on.

28:09 Right.

28:09 Well, yeah.

28:10 One of the classes I, one of the examples of the book I gave was like a, a cafe queue.

28:15 So a queue of customers in a line.

28:17 And yeah, I wrote it as inevitable, but there was all this additional stuff in there.

28:22 It wasn't just a basic list.

28:23 There was more to it than that.

28:25 And, but because I implemented as an iterable, I didn't care.

28:28 I could use it like I could use a list.

28:30 Yeah.

28:30 But it was able to do this additional work with data behind the scenes that made it just easier to use the, to reason about this line of customers.

28:41 And then you get generator methods with the yield keyword and yield from, and like all sorts of stuff.

28:46 You, you get pie test fixtures that allow you to set up and tear down without actually writing.

28:53 Try finally blocks and all these things.

28:55 A lot, a lot of interesting places.

28:57 While I was thinking about this conversation and this topic in particular, I ran across an article called static duck typing in Python with protocols.

29:05 How's that to like crash two ideas together?

29:08 Yeah.

29:09 It's well, when we look at data types, it's like the way Python even does its data types is yeah.

29:16 You can say that you want to say, okay, this is a list and you can type it as a list, but you could also just actually type it as an iterable.

29:24 And basically tell Python what you care about.

29:27 Do you really care that it's a list?

29:30 Probably not.

29:31 Yeah.

29:31 Do you care that it's an iterable that you can, that you can work with similar to a list?

29:36 They're like, how are you going to use it?

29:37 That's what you ask yourself.

29:38 Then you base your annotations around that.

29:40 And so you can say, well, I care measurement like I want it to, I want it to be usable as a measurement.

29:47 I don't really care what it is.

29:48 It doesn't need to be this particular type of class.

29:51 And so that's moving from like we're talking about moving from Java thinking into Python thinking is, well, it's not what it is.

29:57 It's what it does.

29:58 Yeah.

29:58 A lot of the types people put down are like, I need to treat it like some duck typing situation.

30:04 So it needs to have these attributes about it.

30:08 Let's try to put that aspect of it into the formal static typing rather than earlier.

30:13 Yeah.

30:13 Okay.

30:14 Maybe a quick audience question or a bit of feedback here before we move on to the next one.

30:18 It's relevant.

30:19 Nick says, early in my Python journey, I wish I had gotten better understanding of all the intricacies, differences, and use cases for various data structures like set versus list.

30:28 Right.

30:28 Yeah.

30:29 Versus dictionary.

30:30 And those are certainly, even if you've done programming before, maybe that environment language didn't deeply embrace these data structures.

30:37 So certainly those are table stakes for being good at Python.

30:40 Absolutely.

30:41 Use cases hits the nail on the head because it's like, what's the difference between them?

30:45 Well, you know, it's a collection.

30:46 And so maybe that's all you care about.

30:49 But I find myself using all of them in different cases because it's like, all right, I never need to add or remove things from it.

30:54 I'd probably a tuple.

30:55 Or I need to, or I need to be able to hash it, use it as an immutable.

30:59 That's a tuple.

31:00 Or I need to be able to see what's in group A and not group B.

31:03 That's set.

31:04 Or I care about the order.

31:05 That's a list.

31:06 Or I want to be able to associate two things.

31:08 That's a dictionary.

31:08 So knowing your different data structures is helpful.

31:12 And, you know, it's tricky because Python's data structures aren't really all that unique, but they have different names than maybe are used to whatever languages.

31:19 You know, you come across lists and what is that?

31:22 Well, it's a C++ vector or a JavaScript array.

31:24 You know, it's like it's, it's, which is where it's helpful to come back to Python kind of with a fresh mind.

31:30 It's like, what is this?

31:32 Not assuming that things have the same names, but being prepared to get a different vocabulary.

31:36 This portion of Talk Python to Me is brought to you by us over at Talk Python Training with our courses.

31:43 And I want to tell you about a brand new one that I'm super excited about.

31:47 Python web apps that fly with CDNs.

31:50 If you have a Python web app, you want it to go super fast.

31:54 Static resources turn out to be a huge portion of that equation.

31:58 Leveraging a CDN could save you up to 75% of your server load and make your app way faster for users.

32:05 And this course is a step-by-step guide on how to do it.

32:09 And using a CDN to make your Python apps faster is way easier than you think.

32:14 So if you've got a Python web app and you would like to have it scaled out globally,

32:19 if you'd like to have your users have a much better experience and maybe even save some money on server hosting and bandwidth,

32:26 check out this course over at talkpython.fm/courses.

32:30 It'll be right up there at the top.

32:31 And of course, the link will be in your show notes.

32:34 Thank you to everyone who's taken one of our courses.

32:36 It really helps support the podcast.

32:37 Now back to the show.

32:43 Iteration, iterables, iterators, living.

32:45 Yes.

32:46 I was working in some code last year and I came across something that just, I was, I cringed and then I laughed out loud because this was written.

32:55 So this code was written by who I best guess university data scientists.

33:01 So, you know, to their favor, this is not a software engineer.

33:05 What they had written and they could tell it had been refactored a few times, but I am not kidding.

33:10 They had said, all right, so I have this thing that it's iterable.

33:13 It was already something that we could use in a loop.

33:15 And we'll come back to what that is for anyone who's curious.

33:17 But we had this iterable.

33:18 I'm going to convert that into a list.

33:20 By the way, we're talking about, in this case, tens of thousands of items in that list.

33:25 I'm going to convert that into a list.

33:26 Then on the next line, they say for index, comma, value in enumerate.

33:33 Enumerate's a thing that lets you get the indexes as well.

33:36 Enumerate that list.

33:37 And then it says, you know, and then it accessed the value again inside the loop by going to the original list and accessing using the index.

33:46 And I just sat there and I kind of half laugh, half cry.

33:50 It's just like there is so much.

33:52 This is so convoluted.

33:54 And all I had to do was take all that logic out and just say for item in original iterable.

34:00 And then item was the thing.

34:02 But what had happened was that they didn't understand what was really happening in a for loop.

34:06 They were still thinking C or C++.

34:08 And the thing is that Python has these two magic methods called iter and next.

34:13 Itter just says, give me the thing I can use that has the next operator that knows where it is in the list.

34:19 I'm going to leave it at that.

34:20 And the next just says, give me the next item.

34:22 Yeah.

34:22 That's all it is.

34:23 Just give me the next item.

34:24 All a for loop ever does is it grabs that, is it calls that next.

34:28 It's like, give me the next thing.

34:29 All right, good.

34:30 I'm done.

34:30 Give me the next thing.

34:31 Give me the next thing.

34:31 Give me the next thing.

34:32 You can write entire classes that behave like this, even generate the value at that moment.

34:38 It's called lazy iteration.

34:40 You can actually generate the next value when it's wanted, not a moment sooner, which means you only ever have one value in memory sometimes.

34:47 Yeah.

34:48 And then you can just put that in the for loop.

34:49 You come up with any arbitrary name for when you access it for each thing in iterable, name a thing, whatever you want, and then use it directly in your code.

34:58 And I wish someone had told me that a lot sooner, discovered that about three years into my journey.

35:03 I'm like, oh my gosh, I've been making lists of everything and I've been doing it.

35:06 It's like, what a waste.

35:07 Yeah.

35:07 And, you know, the yield keyword, obviously, is a super easy way to do that lazy iteration that you talk about.

35:12 But it also is really important for things like databases, right?

35:16 Yeah.

35:16 If you talk SQLAlchemy, right?

35:18 I'm going to do a query.

35:19 I could call the all method and turn it into an in-memory list, or I could, if I don't need that, I could just loop over it, right?

35:25 And kind of read it cursor style, potentially stop talking to the database halfway through.

35:29 Exactly.

35:29 If I want to need it, I don't need the rest of the data.

35:31 Exactly.

35:32 Yeah, exactly.

35:32 Yeah, interesting.

35:33 Cool.

35:34 Yeah, I think iteration is quite, it's settled because there's only two types of loops in Python.

35:41 And usually we're using the for loops.

35:43 But the variations, there's just so many of them, right?

35:47 You can have your for loop that just goes over a thing.

35:50 But what if you need the index like you described?

35:52 Not for this improper use.

35:53 But sometimes you actually still do need the index.

35:55 So you could do enumerate of that thing and get a tuple.

35:59 But you could take it to another level, do tuple unpacking and say like index comma value in enumerate of things.

36:04 Right.

36:05 And you get it back.

36:06 And you can unpack a dictionary too.

36:08 You can unpack a tuple with that in the dictionary.

36:10 There's this whole library and it built into Python called iter tools.

36:14 And if you ever have, you know, 10 minutes over coffee, just pull up the documentation for iter tools and just kind of scroll through.

36:19 There's some neat stuff.

36:21 Like you can, you can like find every possible combination of three different lists.

36:25 I just give it three different lists.

36:27 And this is one that will take all three of them.

36:28 Just give you every single possible combination of it.

36:31 I actually use it in the book.

36:32 And it's just, there's all these neat things you can do with iterables.

36:35 Some of them don't even have to stop.

36:36 There are infinite iterables where you can just say, I want to get this sequence repeating infinitely.

36:42 One, two, three.

36:42 One, two, three.

36:43 One, two, three.

36:43 One, two, three.

36:44 One, two, three.

36:44 One, two, three.

36:44 And two lines of code.

36:46 And you can have that repeating forever.

36:47 And if you're using an old fashioned while loop or an old fashioned C++ style for loop, that would be multiple lines of code.

36:53 Like, you know, you have to keep track of the state and all that stuff.

36:56 Yeah.

36:56 I think it's two lines.

36:57 Yeah.

36:57 It's amazing.

36:58 Yeah.

36:59 So there's, there's a lot of depth, even though it seems so simple when you first look at it there.

37:02 Closely related to that are comprehensions.

37:04 Yeah.

37:05 Like a list comprehension, right?

37:07 People often see a list comprehension or a set comprehension and go, especially a list comprehension, they're like, oh, that's the same thing as a for loop.

37:15 Where I create, before the for loop, I create a list.

37:17 And then in the for loop, I add the thing to the list.

37:19 And these are the same.

37:20 May have the same outcome, but they're not actually the same thing.

37:24 Like the runtime performance is not the same.

37:26 The bytecode implementation of those two things is not the same.

37:30 This is like a nonstop debate I have on YouTube whenever I put a video and mention something about list comprehension.

37:35 They're like, that's just a for loop.

37:37 It's not actually just a for loop.

37:38 It's like a kind of accomplishing the same thing.

37:41 But we now have for potentially 312, Carl Meyer put out a PEP 709 called inlined list comprehensions, which are supposed to take some of the mostly invisible behind the scene implementation and make comprehensions two times faster.

37:59 I believe they're already faster than this looping story I talked about, but this would make them faster still, which is pretty cool.

38:06 Wow.

38:06 How does this how does this syntax work?

38:09 I'm curious.

38:09 It's just like it's just an internal change.

38:11 Okay.

38:12 So the syntax is unchanged.

38:14 Yeah.

38:15 So the way here's the magic of list comprehensions.

38:17 That's funky that people maybe aren't aware of is it.

38:20 It used to actually create a list comprehension object that implemented the loop in a function.

38:26 And then it called that function as if you'd like nested a function to create the isolation of the list comprehension.

38:32 So that created a stack frame.

38:34 I created an object.

38:35 Then it did function in direction and all those things made it slower than it needs to be.

38:40 So now they create a new load fast and clear byte code operation that like stores the state of the function, then does the list comprehension and then pops it back off the stack to unwind it.

38:52 So anyway, comprehensions are interesting.

38:55 I just wanted to this is like kind of a brand new thing.

38:57 That's cool.

38:57 That's coming along.

38:59 That might be interesting for people.

39:00 Well, I think it's worth mentioning then that what a list comprehension really is.

39:05 So like if I unpack it because people think, oh, people use that term very loosely.

39:09 A lot of times they want the list comprehension.

39:11 But what is a list comprehension?

39:12 And if I unpack that, what it really is, is it is a generator expression that is being unpacked into a list.

39:23 So what's a generator expression?

39:25 Generator expression is a type of generator.

39:27 What's a generator?

39:27 A generator is a function or type of function that is itself iterable.

39:34 So that's where this is.

39:36 That's where this is critical is that a generator you can you can write a generator that what when call it that it has.

39:43 You mentioned that earlier, this yield keyword.

39:45 And so you can do whatever you want in this function.

39:49 You can write it however much logic you want.

39:51 Every time you hit the yield statement, it will return that value.

39:55 And then it's going to wait for the next call.

39:56 Then the next iteration of that, like the next call to next on that generator.

40:01 And then it's going to run through again until it hits a yield.

40:03 And then it's going to stop, turn that value and wait again.

40:06 And you can do some incredible things with these.

40:08 But once you understand the generator, which are incredibly powerful tools,

40:13 then you can write a generator expression, which is I discovered later.

40:17 A generator expression is to a generator as a lambda is to a function.

40:21 It's an anonymous generator.

40:23 That's all it is.

40:24 A lambda is just a very small inline anonymous function.

40:29 Has the logic.

40:30 It can take in some values, return some values.

40:32 It's a lot simpler, a lot shorter.

40:33 And you do it right in place.

40:35 So you're not having to go off somewhere and write a different function and come back.

40:38 That's why we have anonymous functions is so that we can just do it like,

40:42 like you see there at the bottom, reduce and just some some rules right there instead of writing a separate function.

40:47 Generate expression, same sort of thing.

40:49 We're doing writing some generator logic.

40:51 It's shorter.

40:52 It's creating a generator, but it's just a lot smaller.

40:56 And if you just wrap it in in parentheses, you can use it as a generator.

40:59 But when you change those parentheses to be square brackets, what happens is you're running that generator and then you're loading all the values into the list.

41:08 You do it in in curly braces.

41:10 You're creating a set.

41:12 Or if you're if you're doing the same thing, but you are the return value in each step is a key value pair.

41:18 Then you're going to create a dictionary.

41:19 But all you're really doing is unpacking that generator into a data structure.

41:23 So sometimes you want to think when you're mucking about with these, do you really want the data structure when you're done or do you just want to go over the values?

41:32 Maybe you just write a generator expression.

41:34 Yeah.

41:34 The difference is super subtle.

41:35 Like, do you have square brackets or do you have parentheses around the expression?

41:39 But the effect or consequence of that is very significant at runtime, right?

41:44 It's kind of like that example you talked about putting everything in a list and looping over it or just looping over it.

41:49 Right.

41:49 Exactly.

41:50 And because these are these are these one liners and we as coders tend to get a text of these one liners.

41:55 It's like, oh, look at all the cool things I could do.

41:58 I have a couple of examples in the book.

41:59 Some real list comprehensions I've come across or some colleagues of mine have come across where it's like it's multiple lines.

42:05 And that's a bad list comprehension.

42:08 That's a bad generator expression.

42:09 Just like you don't want to have a multi-line lambda at that point.

42:12 Write a function.

42:13 Generator expressions and lambdas are supposed to be these really small, short, in place bits of logic.

42:18 And if it's getting complicated, write the full size thing.

42:22 It's a lot easier to debug.

42:23 If nothing else is usually to debug, but it's also a lot easier to read.

42:26 It is.

42:27 It is.

42:28 Just why we're kind of ranching on these things.

42:31 Like, I wish there was a way to specify a sort in these list comprehensions, generate expressions, and maybe not the generators because that gets tricky.

42:40 But so often I find myself I'm doing a list comprehension and then I got to turn around and sort it.

42:44 Like if it could just just add that in there, that would make me happy.

42:47 All right.

42:48 Moving on.

42:48 Packaging.

42:50 Yes.

42:51 Deciding on packaging and all that right up front.

42:54 Actually, before we get to that, I guess the other one that you threw in here is virtual environments.

42:58 Indeed.

42:59 Yeah.

42:59 Virtual environments.

43:00 Use them.

43:01 They're the only.

43:02 I avoided them for years because I thought, oh, these are scary.

43:04 I don't understand them.

43:06 They're not scary.

43:07 I'm actually going to.

43:08 I'm actually going to hijack something.

43:09 I learned from one of my guests on my own podcast recently is what containers actually are.

43:14 By the way, Docker containers.

43:15 This is where this is relevant.

43:17 You know what a Docker container actually is?

43:19 Is it's the folder on your operating system?

43:20 And it's just changing the route.

43:22 So when you're working inside of it, it's changing the route.

43:24 There's a bunch of magic logic around that.

43:27 But that's all it is.

43:27 It's like it's pretending that that folder is the route and it's running things accordingly.

43:32 That's all a virtual environment really is doing either.

43:35 So there's not a lot of difference between.

43:37 It lies to Python about the path.

43:39 Yeah.

43:40 There's not a lot of difference between the virtual environment and the Docker image.

43:43 Like they're the same fundamental concept.

43:45 So all you're doing is you're creating this little spot on your, this little folder.

43:49 And you're saying, these are my Python packages.

43:51 And unless you tell it otherwise, don't go looking anywhere else.

43:55 These are the ones I want to use.

43:56 And they're super easy to create.

43:59 It's like one line to create a virtual environment.

44:01 And it lets you maintain exactly the packages you want for that particular situation.

44:06 And it's great.

44:07 It is.

44:08 Yeah.

44:08 It's once you get used to it, it's great.

44:11 I remember it being frustrating at the beginning.

44:13 You're like, ah, I don't remember exactly.

44:15 How do I create the environment?

44:17 How do I activate it?

44:17 How do I know?

44:18 Do I really need this?

44:19 And yeah, I don't, I don't start any projects without having them as just straight up using

44:25 them right away.

44:25 It's like, absolutely.

44:26 Yeah.

44:26 I probably have created them for projects that don't even have external dependencies because I was just

44:30 like on autopilot.

44:31 Like, oh, actually I didn't need that, but whatever.

44:32 It's still good to have.

44:33 Yeah.

44:33 I would absolutely control which Python version you're using.

44:36 And then you can install whatever packages you want.

44:38 And you don't not worry about messing up your system or user scope packages.

44:42 And I had a coworker recently say, I dream of the day when I can actually remember how to

44:46 activate this, but it's actually really easy.

44:48 So once you, so assuming you know the name of the folder, so you can pick the name of the

44:53 folder you use for your virtual environment.

44:55 Traditionally, it's called like them, V-E-N-B.

44:57 And it's, here's the thing to remember.

44:59 You don't actually have to activate your virtual environment.

45:02 Put that out of your mind for a moment.

45:03 You can run things directly in the virtual environment.

45:06 So in Unix systems like Linux and Mac, there's a bin folder.

45:11 Vim slash bin slash, and then whatever Python, whatever Python executable you want to run.

45:16 It could be Python.

45:17 It could be pip.

45:18 It could be your own package.

45:19 You can actually just navigate into that bin folder.

45:22 And so I can say Vim, bin, pip, and I've got the pip.

45:25 And that knows to only work within its virtual environment.

45:29 It's not going to break out of that container.

45:30 Once you understand that, here's where they activate script is.

45:33 Vim, bin, activate.

45:35 It lives in the bin file.

45:37 And if you're on Windows, the only thing you're changing is you're going from bin to capital S scripts.

45:42 And then the things have an ending of like either .bat or .ps1, depending on what show you're using.

45:49 But that's the structure of a virtual environment.

45:52 And once you understand that, they're a little overscuried.

45:54 Yeah, my RC file for my user profile on my Mac is just full of here's the Python executable in some virtual environment.

46:02 And here's the script and commands to send to it for like little aliases and stuff.

46:06 And like none of those activate the virtual environments.

46:08 They just run the Python in the virtual environment to make all the magic happen.

46:12 Another thing I would recommend is people think about creating aliases that make virtual environments happier and easier to use.

46:20 So, for example, and you could do this in PowerShell on Windows.

46:23 You can obviously do it in your RC files on Mac and Linux.

46:26 So, for example, I have vnv, which is for venv, but I don't want it to accidentally change the directory.

46:32 And if you haven't, like whatever.

46:33 Like I just have a short version, so I know it's something slightly different.

46:36 But it'll go run the virtual environment.

46:38 And it'll always remember to add --prompt equals dot, which will make the virtual environment not just be called venv,

46:45 but it'll be called the name of the project that it's in.

46:48 So, for example, if it's in like talk Python, like the virtual environment, when I activate it, we'll say it's the virtual environment talk Python.

46:54 Right.

46:55 And then it activates it.

46:56 And then I think I think this is almost a bug in Python that most of the time I would say like 49 weeks out of the year.

47:05 When I create a virtual environment, the first thing I hear is warning your pip is out of date.

47:10 You know, like, OK, well, I'm starting a new project.

47:13 How about we start with a new one?

47:14 Let's just go with that right now.

47:15 Right.

47:15 And so my just that is I always want the new one.

47:19 I don't ever want the old one because this is brand new.

47:22 So I want the new one.

47:23 Why don't we fix that?

47:24 So my alias then says pip install dash you pip setup tools, wheels, pip-tools, et cetera.

47:30 And just ditches that output.

47:31 So like, yeah, right.

47:32 Immediately, like I have when I create a virtual environment, it's immediately active and it's immediately named the project and it's immediately up to date.

47:39 So I don't see those silly warnings, at least for a while.

47:41 And all I got to remember is type V and V.

47:44 Boom.

47:44 And that's it.

47:45 Right.

47:45 So if you get into this, if you do a few little steps.

47:47 I do this too.

47:48 Yeah.

47:48 Yeah.

47:49 Yeah.

47:49 I imagine you did.

47:50 And so it's I just strongly encourage people to somehow either do this in PowerShell or do this in their terminal setup because like that big, long thing to type, that's a hassle.

48:00 Three letters is not a hassle.

48:02 And they just makes it.

48:03 There's something about lowering the friction that makes it really nice.

48:06 And there's tools you can also install that do it.

48:08 But like I recommend just writing your own aliases too, because everyone's got a sub of a different workflow instead of perfect anyway.

48:14 That's why Vim has so many options.

48:16 So writing this yourself, it's super easy and it teaches you a bit about how it's working so that you're not surprised at the results.

48:24 Yeah.

48:25 Yeah.

48:25 Indeed.

48:26 Okay.

48:27 Now packaging.

48:27 Yes.

48:29 And I'm going to keep this one super short because it's a deep, deep, deep, deep, deep, deep, deep rabbit hole.

48:35 Figure out your packaging day one.

48:37 Like you get, I will figure out packaging before I even figure out what I'm, you know, how I'm building stuff.

48:42 I am just going to set up a basic hello world and then I'm going to get the packaging working.

48:48 Because I actually, and I learned this the hard way.

48:51 I built a project once.

48:52 It took me one month.

48:54 I was very happy with it.

48:55 It was gorgeous.

48:56 It was a game.

48:56 You can actually find it online now.

48:58 I was very happy with myself.

49:00 And then I spent the next two years trying to package the dark bag.

49:02 Turned out it was impossible to package because of certain assumptions that one of the libraries I was using made.

49:08 Had I actually tried to package it from the beginning, I could have saved myself a heck of a lot of time and just use a different library.

49:12 Yeah.

49:13 So you figure it out from day one.

49:16 And then the benefit you get to this is that then you are constantly, especially if you're doing out of place testing, which I strongly recommend.

49:23 Run your tests.

49:24 Your tests should not be in your source folder.

49:26 Put them separate.

49:27 That's going to force you to actually install your package into a virtual environment and then run the test against that package.

49:34 It is not as scary as it sounds.

49:35 I have a whole, by the way, if you go to my website, code mouse92.com, I've got a whole talk on this called Escape in the Gargo Cult.

49:41 And I explain how these patterns work.

49:43 But when you do it that way, that means from day one and ever onward, you are testing your packaging.

49:48 You're testing your assumptions about the working directories and all that.

49:53 And you are testing it in the same way that it's going to be used on someone else's computer.

49:58 So you never again have to have the thing of, well, it works on my machine because you are, all of your tests are depending on actually installing your package.

50:08 So if something breaks, you bring in a dependency that just doesn't work with your packaging scheme, you know, right away.

50:14 And you can make a decision about it.

50:15 Yeah, that's good advice.

50:16 There's a lot of depth to the packaging thing.

50:18 I had a whole panel on it.

50:19 We're in Talk Python not long ago.

50:21 Definitely a lot there.

50:22 Oh, yeah.

50:23 Deep rabbit hole.

50:24 But you know what?

50:24 Don't copy it.

50:25 Don't don't copy pasta.

50:27 Don't copy paste.

50:27 Find out what the things you're using are doing because.

50:31 All right.

50:33 Last one.

50:34 Yeah.

50:35 Currency.

50:35 Currency.

50:36 Yes.

50:37 Which for years I heard concurrency, asynchrony, threading, parallelism, multiprocessing.

50:42 I heard those all used as as synonyms for one another.

50:46 They're not completely different things.

50:50 So and actually I have to thank the Go language for that for for teaching me this.

50:55 Thank you, Rob Pike, for his talk.

50:57 Concurrency and parallelism are not the same thing.

51:00 But here's the thing to understand.

51:01 Concurrency is doing.

51:03 You're switching between tasks.

51:04 You're checking your phone and you're talking to your friend.

51:07 Are you doing both at once?

51:08 No, you're not reading stuff on Twitter and listening to what your friend says.

51:13 You're actually switching between the two contexts rapidly.

51:16 You're not really multitasking.

51:18 You're just changing your focus fast enough that no one ever notices the difference.

51:21 That's concurrency.

51:23 So you have one CPU or potentially potentially multiple or involved in this where people get

51:28 confused.

51:29 Python's nice because you're locked to one CPU for concurrency.

51:32 So it's I'm gonna work on this.

51:35 Then I'll work on this.

51:35 Then I'll work on this.

51:36 Then I'll work on this.

51:37 It doesn't make your code faster.

51:38 It'll never make your code faster.

51:39 Just like multitasking will not make you get your work day faster.

51:42 What it's going to do is it's going to slow you down.

51:44 But it doesn't look like it.

51:48 In some cases, it'll make your code look faster because while you're waiting on the user to

51:52 type in their answer, this example I use in the book, while you're waiting on the user

51:55 to type in their answer to a question, you're doing some math in the background.

51:59 You're IO blocked.

52:00 You're waiting on something over which you have no control.

52:02 You can do some other work while you wait.

52:04 That's concurrency.

52:05 And you can do that with a synchrony where Python's handling that multitasking, or you can do

52:10 it with threading where the operating system is handling that multitasking.

52:13 That's different than parallelism, which is where you're actually using different CPU cores and you're

52:19 doing two things at the same time because you have two different CPUs, two different cores

52:24 working in parallel.

52:26 And they're not depending on one another.

52:28 They're not waiting.

52:29 So you're doing multiple tasks at once.

52:32 Both of them have overhead.

52:33 Near one is a silver bullet for performance.

52:35 You're really going to have to think a lot about it.

52:37 But I actually like the GIL for this reason.

52:39 Global interpreter lock, the much aligned thing that stops us from getting free multiprocess,

52:45 free parallelism, free multiprocessing with every thread.

52:48 Because this forces us to actually stop and think, which thing do I need?

52:53 Do I need concurrency or do I need parallelism?

52:56 Because you can get both in Python, but the GIL makes sure that you have to make a decision.

53:01 Because if you want multiprocessing, you're going to have to create different processes,

53:05 spin them off, manage them separately.

53:07 There's more overhead, but you're going to have to address all of those things of how do

53:12 we share resources, which is a really snarly topic.

53:14 And Python doesn't let you pretend that that's not going to be an issue.

53:17 It makes you think about that.

53:19 It does make you think about it.

53:20 While you can't get more done, you can't make an individual thing go faster by using concurrency

53:26 on it.

53:26 You can start a bunch of things that run outside of your control and outside of your program

53:31 and then get them back, right?

53:33 Like I could issue a web request, a three API endpoints, all of them, and then start gathering

53:38 up the answers as the first one comes back, right?

53:39 Like you have to think about, I think the main thing to think about here is where am I waiting

53:45 on something else when you're looking for the async and await type of benefits?

53:50 Like am I waiting on a thing?

53:51 Could I be doing something while I'm waiting?

53:54 Async and await amount makes it super easy to do stuff while you're waiting.

53:58 But your program itself is still constrained by the GIL.

54:01 And often with async and await, there's not even a second thread involved, right?

54:06 It's just slicing up, time slicing up the current thread.

54:09 Yeah.

54:09 Which I mean, technically is all that's going on with the threads anyway.

54:12 Like when you have multiple threads, the operating system is now just doing the slicing up.

54:16 You're still constrained to one process.

54:18 Then you need a separate process if you want to do multiple things.

54:22 Yeah.

54:22 But like my machine has 10 cores.

54:23 It will go across the cores according to the OS.

54:25 But the effect in Python because of the GIL means it doesn't actually make any difference.

54:30 It's still the same, right?

54:31 Right.

54:31 But if you get multi-processing them for sure.

54:33 So yeah, super, super interesting.

54:34 I think one of the things people tell, I think people say two things.

54:38 One, asynchronous programming is so hard you should never try it.

54:42 I hear that all the time.

54:43 It's so complicated.

54:44 It's so hard.

54:45 You're going to end up with race conditions.

54:46 Just don't try it.

54:47 Mere mortals don't dare.

54:50 They only regret.

54:51 Maybe if you're doing like signaled events in C++ across, you know, hardware notification.

55:00 Like there are scenarios that are super, super tricky.

55:03 Using async and await generally don't fall into that.

55:06 Like you write regular code without changing the structure.

55:10 You don't have to have callbacks and signals and locks.

55:12 You just write regular code where you have the async stuff you put away.

55:15 It's kind of the end of the story.

55:17 And it's not necessarily as simple as straight up regular programming, but it's, you know,

55:20 it's like 10% more.

55:22 It's not 10x more.

55:23 Yeah.

55:23 I think a lot of people for some, I don't understand why, but a lot of people, I think

55:27 they want to feel like I've conquered this dragon.

55:31 And if I'm going to tell other people how to do it or what to do, the first thing I'm

55:36 going to tell them is how hard it is and how daunting it is.

55:39 And like, yet I've conquered it and now I will help you.

55:41 Right.

55:41 That doesn't do people a favor.

55:43 Right.

55:43 They want to feel good about themselves.

55:45 Yeah.

55:45 I guess so.

55:46 But I feel like threading and parallelism lands deeply in that space where people are like,

55:51 oh, it's super hard.

55:53 And it's not, it really isn't.

55:55 But where it's hard is you, you have to be prepared to really think about why do I need

55:59 it?

55:59 And if I can briefly say, I have two entire chapters in this book on that topic.

56:05 Yeah.

56:05 Cool.

56:05 And I pick one example and it has nothing to do with waiting for a webpage to download

56:09 because that is both overused and something that most of us don't need.

56:12 And I apply synchrony.

56:14 I apply threading.

56:14 I apply parallelism.

56:16 And you actually see all those little concerns of how do you handle deadlocking, live locking?

56:22 How do you handle starvation?

56:24 How do you handle the producer consumer problem?

56:26 How do you handle the fact that, huh, I just added 15 multi, multi processing workers.

56:31 And now this thing is like way slower.

56:33 Why?

56:33 I go into all of that.

56:35 And I think that's one thing I wanted to say is when I was writing this book, that was the

56:40 thing that was my chief delight was coming up with examples for all of the topics from the,

56:44 from the simplest, all the way to the most complicated, where I did not hold back on the

56:48 relevant complexity.

56:49 My list comprehensions were the starliest list comprehensions you're ever going to find in

56:54 this tutorial.

56:54 You know, my, you know, I have, I go into recursion.

56:57 I go into, yes, I go into meta classes, whichever one's afraid of.

57:01 I go into all of these things.

57:02 I dig into them, but the complexity is always relevant.

57:06 So you're only going to get the complexity from the topic and you're going to understand

57:09 where all these bits and pieces interplay.

57:11 But the, in the, the complexity coming in the old, from other irrelevant topics, I try to

57:17 keep to a minimum.

57:18 So you can really focus on understanding, not just the happy path of coding, not just the

57:23 happy path of like, okay, well you can async this.

57:25 Look how easy that was.

57:26 Well, it's not easy, but it's also not painfully hard.

57:30 You just have to be prepared to think through those different factors.

57:35 And so I focused a lot of time on really helping you learn how to think through those factors

57:39 as it relates to your code.

57:41 Excellent.

57:41 It sounds like a bit of the Einstein philosophy.

57:43 Simple as possible, but not simpler.

57:45 Right.

57:45 Exactly.

57:46 Yeah.

57:46 Cool.

57:47 I, you know, last thoughts on the async stuff.

57:49 Like I think a sprinkling of async and await makes massive difference for a lot of scenarios.

57:53 And that's pretty straightforward.

57:54 If you're going to try to write trio or some full on framework that deeply coordinates and

57:59 understands it.

57:59 Like, yeah, that's super, super hard.

58:02 Again, some of the signaling stories I talked about with like C code, that's super hard, but

58:07 this stuff people, it's pretty approachable.

58:09 So don't, don't forget to check it out.

58:10 All right.

58:11 Well, I think that's it for our things.

58:14 We wish we knew if we had more time, we could wish upon more things, but that's pretty much

58:19 definitely covers it.

58:20 Doesn't it?

58:20 Yeah, I think so.

58:21 Nice.

58:22 All right.

58:22 Well, before you get out of here, let me ask you the final two questions.

58:25 If you're going to write some code, what editor to use?

58:28 Yeah.

58:29 You know, for the longest time I was a VS Code guy.

58:31 I still love it, but I broke down and I bought PyCharm Professional and I love that because

58:36 I do a lot of refactoring and PyCharm saves my butt so much when it comes to refactoring

58:41 that I fell in love with it.

58:43 And I can't imagine working without it now.

58:44 I hear you.

58:45 So I'm spoiled.

58:48 It does surprise me how people, not just a PyCharm comment, but in general, there's

58:53 a lot of people who are like, oh, I don't really want to get like a new computer or I don't

58:57 really want to pay 20 bucks for this thing.

58:59 That would save you thousands of dollars worth of time as you go for it.

59:04 You know, like, so I'm glad.

59:06 Large on your tools.

59:07 Yeah.

59:08 As far as your budget will allow you.

59:10 And I've been on both ends of that spectrum.

59:12 And as your budget allows you, large on your tools because they're what you're going to

59:15 be spending most of the day using anyway.

59:18 And the last thing you want is something you're going to be getting splinters on.

59:21 So indeed.

59:22 And then notable PyPI package, something you're like, oh, this thing's awesome.

59:26 I just ran across X, Y, Z.

59:28 I guess I have two.

59:30 One would be, I think my favorite, and this has certainly gotten plenty of press, but I

59:34 love it.

59:34 It's the hypothesis testing library.

59:36 I love hypothesis.

59:37 Hypothesis is fantastic.

59:39 It will revolutionize the way you write tests.

59:41 It's not as scary as the documentation makes it look.

59:44 If you click on what you can generate and how, start there.

59:47 I go through the quick start guide and then go right to this.

59:49 And it's so cool because you just write these strategies.

59:52 I'm like, okay, this is what my data is kind of sort of supposed to look like.

59:56 And then your test is going to try and break your code.

59:59 I love hypothesis.

01:00:01 It's going to throw all sorts of garbage at you until it finds a way to break your code.

01:00:04 And then it's going to go, hey, here's the simplest thing I could find that blew up

01:00:07 your code.

01:00:08 Fix it.

01:00:08 Lovely.

01:00:09 Nice.

01:00:09 So much fun.

01:00:10 The other favorite package I have is one of my own actually branch detective.

01:00:15 I built this at two jobs ago.

01:00:17 And basically we had the problem where we had these diverging branches where we had no

01:00:23 choice, choice, but to completely diverge them.

01:00:24 And we were cherry picking back and forth.

01:00:26 And there was always this, which commits are in this one that aren't in this one.

01:00:30 When you're doing cherry picking, your hashes are complete garbage.

01:00:33 So I built this tool that allows you to see what's in one branch.

01:00:37 It's not in another branch, any branch arbitrarily.

01:00:40 You can specify day range.

01:00:42 It knows how to handle cherry picks.

01:00:43 It knows how to handle merge revisions or filter them out.

01:00:46 And you can even pass to the dash M flag and just get this nice markdown.

01:00:51 If you're using conventional commits, we'll just spit out this markdown.

01:00:53 Then you can then copy and paste into your PR and say, hey, here's, here's all.

01:00:57 Here's the summaries of the 26 commits I'm putting in here.

01:01:00 So I use this every single day and I've improved it for every project I'm on so far.

01:01:06 And I, I just, I love it.

01:01:09 It's my favorite.

01:01:09 One of my favorite things I've ever built.

01:01:10 So you kind of wrote set for Git, right?

01:01:13 Like you can say, what's in the set?

01:01:14 Subtract that set.

01:01:15 Show me those things.

01:01:16 Yeah.

01:01:16 Speaking of data structures.

01:01:18 Exactly.

01:01:18 Like what's here that's not there.

01:01:19 Yes.

01:01:20 Yeah.

01:01:20 Cool.

01:01:21 All right, Jason.

01:01:21 Well, thanks for being here.

01:01:23 Final call to action.

01:01:24 People will get their Python better.

01:01:26 What do you tell them?

01:01:27 Yes.

01:01:27 So dead simple Python is available from wherever you buy books, but especially if you go to

01:01:33 nostarch.com, you can get this comes in ebook or in physical book form.

01:01:38 And Hey, you know what?

01:01:39 I wrote it and I refer to it every single day because this is the book I wish I had had.

01:01:43 And my favorite little feature of this is that I have a thing in the back with every single

01:01:49 command for the debugger.

01:01:50 Also, every single dunder method is also documented in here, which even the Python docs didn't

01:01:55 do that.

01:01:55 This is the entire core language bar nothing explained for the existing coder.

01:02:00 So check this out.

01:02:01 Awesome.

01:02:02 Well, thanks for taking the time for being here and enjoy the chat.

01:02:05 Thank you so much.

01:02:06 It's been a pleasure.

01:02:07 You bet.

01:02:08 See you later.

01:02:09 This has been another episode of Talk Python to Me.

01:02:12 Thank you to our sponsors.

01:02:14 Be sure to check out what they're offering.

01:02:15 It really helps support the show.

01:02:17 Take some stress out of your life.

01:02:19 Get notified immediately about errors and performance issues in your web or mobile applications with

01:02:24 Sentry.

01:02:25 Just visit talkpython.fm/sentry and get started for free.

01:02:30 And be sure to use the promo code talkpython, all one word.

01:02:33 Want to level up your Python?

01:02:35 We have one of the largest catalogs of Python video courses over at Talk Python.

01:02:39 Our content ranges from true beginners to deeply advanced topics like memory and async.

01:02:44 And best of all, there's not a subscription in sight.

01:02:47 Check it out for yourself at training.talkpython.fm.

01:02:50 Be sure to subscribe to the show.

01:02:51 Open your favorite podcast app and search for Python.

01:02:54 We should be right at the top.

01:02:55 You can also find the iTunes feed at /itunes, the Google Play feed at /play,

01:03:01 and the direct RSS feed at /rss on talkpython.fm.

01:03:05 We're live streaming most of our recordings these days.

01:03:08 If you want to be part of the show and have your comments featured on the air,

01:03:12 be sure to subscribe to our YouTube channel at talkpython.fm/youtube.

01:03:16 This is your host, Michael Kennedy.

01:03:18 Thanks so much for listening.

01:03:19 I really appreciate it.

01:03:20 Now get out there and write some Python code.

01:03:22 I'll see you next time.