#287: Testing without dependencies, mocking in Python Transcript

00:00 We know our test should be relatively independent from other parts of the system. For example, running a test shouldn't generally call a credit card processing API and talk to a database when our goal is just to test the argument validation. And yet, your method that you wanted to test it does all three of those things and more. What do you do? Some languages use elaborate dependency passing mechanisms and frameworks that go under the banner of inversion of control and dependency injection. In Python, we do sometimes have that but it's much more common to just temporarily redefine what those two functions do using patching and mocking. On this episode, we welcome back and Elena pacos. To talk us through the whole spectrum of test doubles, dummies mocks and more. This is talk Python to me, Episode 287, recorded August 24 2020.

01:03 Welcome to talk Python to me, a weekly podcast on Python, the language, the libraries, the ecosystem, and the personalities. This is your host, Michael Kennedy, follow me on Twitter, where I'm at m Kennedy. Keep up with the show and listen to past episodes at talk python.fm and follow the show on Twitter via at talk Python. This episode is brought to you by linode and monday.com. Please check out what they're offering during their segments. It really helps support the show. Do you want to learn Python, but you can't bear to subscribe to yet another service at Talk Python Training we hate subscriptions to that's where our course bundle gives you full access to the entire library of courses. For one fair price. That's right, with the course bundle, you save 70% off the full price of our courses, and you own them all forever. That includes courses published at the time of the purchase, as well as courses released within about a year of the bundle to stop subscribing and start learning at talk Python, FM slash everything. And Elena, welcome back to talk Python to me.

02:03 Thanks. It's great to be back.

02:05 Yeah, it's super to have you back. Last time, we spoke about 100 days of code and Harry Potter and things like that, if I recall correctly, is that right?

02:14 Yes, that's correct.

02:15 Yeah, that was really fun. And that was right at the height of the hundred days of code when everyone was really using that to learn to code. And so you had this unusual, but I think really engaging way of taking code and making it not just silly technical stuff, but making it this fun Harry Potter adventure world.

02:32 Yeah, I still love that approach. So I always try to find something I'm passionate about, and then try to use it to learn.

02:38 Yeah, excellent. So now we're gonna come back and talk about writing code that we're more likely to be sure that works, testing code, and using mocking and all that. But when we spoke last time, you were a was an internship at Microsoft, what what kind of,

02:56 it's something in between, it was called an AI residency at Microsoft Research in Cambridge in the UK. So it's like an intensive one year program, where you learn a lot about how to apply machine learning how to become a great research engineer also cool. It's like, yeah, it's more than an internship, like a really long extended internship.

03:17 I'm sure that was a really fun project experience.

03:19 Oh, yeah. It was amazing. Also, the place Microsoft Research in Cambridge is just great. I really loved it there.

03:26 Yeah. I could imagine. And how about now? What are you up to now?

03:31 Yeah, so I finished the residency. And I moved back to Germany and joined a German company called innovex. I also like it here a lot. And I'm working as a machine learning engineer, similar to what I did at Microsoft Research, but not so research heavy. So I'm doing more production code or production work. And I'm currently working in a pure data engineering project. So I've been learning a lot more about the work that comes before actually applying the machine learning algorithms, which is also very interesting.

04:00 Yeah. So I would guess that involves a lot of pandas, a lot of cleaning up data, like getting stuff in the right format.

04:06 Yes, yeah. And then we give the data to the data scientists, which can just use the clean data to do that nice machine learning stuff.

04:15 Yeah, you make them look good. Yeah, go here. You just feed this over, and it works great. Like it didn't start out that way. What kind of problems? Are you trying to solve our answer?

04:23 It really depends on what the customer wants. So it can go from supply chain management and apply machine learning there to right now in the cloud, where we just create kind of a data lake in the Google Cloud where we have different data sources that have to be processed and brought together and all the other stuff. So there's just a lot going on. It's a very huge project with several teams working on it.

04:49 Wow, that sounds really fun and sounds like one of these big data projects that people talk about a lot, but not that many people actually do.

04:56 Yeah, it's also very interesting for me to see this other side. I've been working on machine learning for most of my computer science life. So seeing something different is also always very nice.

05:08 Yeah, absolutely. And you also have some project going on in your spare time. Yeah.

05:12 Yes, exactly. So Chi mocks Shula? Yeah, it's I think, like it's a German project. I'm doing voluntary work there. It's called ke mochila. That's like, I think you could translate it as AI goes to school. Yeah. Where we teach AI and machine learning skills to kids in German schools. So we do classes, usually, on site, but now online. And due to the COVID crisis, and yeah, it was somewhat similar to what you talked about in the last episode, or some recent episode with Nick winter on code combat.

05:45 Yeah, code combat is really a fun thing for kids as well. I think that's super neat what they're doing,

05:51 I want to try it as well. Even if I know how to write code and Python, it sounded so much fun.

05:56 Yeah, it's, it's really neat. Like I really enjoyed just kind of poking around with it and seeing it's, again, kind of like your Harry Potter World, it's a very different way to present programming the kids were you still type, it doesn't feel like the burden of being exactly right. You can kind of type it wrong, and it'll mostly correct it for you. So it's pretty interesting. So yeah, with this project, you're teaching kids AI machine learning, what does that even look like? What kind of stuff? Are you teaching them? What do they get when they're building libraries? and so on?

06:27 Yeah, so we tried different approaches, we always have a bit of theory. So how did what is AI? How does it work, we also have a part on the ethics behind AI, which I think is really important. And there's also some practical exercises with Jupyter notebooks. And before we use an approach was where you have these blocks of programming parts, which you can stick together, like the for loops, and so on. But we didn't like that so much. So right now, we are using Jupyter notebooks. And I think it's amazing. Since right now there's one, it's called coffee camp. It's like a four day course. And the youngest one is in fifth grade. So it's like 11, or 12. And the oldest one is just about to graduate. And it's so nice to see these kids fascinated by this topic.

07:14 Yeah, it's definitely gonna give them a leg up, learn these types of things early and know about them. And just to learn about Jupiter and computational thinking at all.

07:23 Yeah. And Germany's also really not so good. With the computer science work in school, I think in the US, you're much better with teaching kids how to code in school already. For German kids, it's really hard to get to know AI or to find somewhat a way to get familiar with a topic without going out there and looking for yourself. What's out in on the internet.

07:47 Yeah, there's the small percentage of kids who will do that they'll go find it themselves. But the majority don't even know they would be interested till they get exposed to it.

07:56 Exactly. Yeah, it was the same for me. So that's why I'm really passionate about this project.

08:01 Yeah. Also, in you've definitely taken it quite far. What you're doing these days? That's definitely the pinnacle is great. All right. Well, let's focus on our main topic for a little while. I'm mocking. So how did you get interested in it? How did you get started?

08:15 Yeah, so I stumbled upon this topic a few months back when I started working in this data engineering project I was just talking about, and we have lots of production code there. And I never heard of mocking before. So it was the first time I used it. And since then, I've been using it in almost all of the tests I write. It's a very controversial topic. So I watched quite a few Python talks on it, and read quite a few blog posts. And yeah, it's just a big but very interesting thing to talk about.

08:45 Yeah. So I think one of the controversies has to do with, you can write your code in certain ways. So it's more easy to test or it's harder to test. And some people see mocking as kind of covering over the bad code you've written or something to that effect, right, that hopefully that sort of summarizes.

09:03 Yeah, exactly. And I completely understand that. So you shouldn't use mocking to fix your badly written code. So you of course, you could should first think about is my code good? Do I want to refactor it? Are there changes I can make to make it easier to test, but sometimes it can be a really

09:22 good tool. I think it's a great tool, I think, when it's used in the right ways and right places, is just what you need. Yes, I agree. Yeah. I mean, Python is so much about this sort of practicality, beats purity, no, it's part of the Zen of Python and all that. And it's just, let's just do what we need to do to make stuff work. And I think one of the examples where this is really different from other languages, is in Python. We don't have things like protected variables. We don't have as often like concrete interfaces, we don't have type enforcement, right? Like the thing you passed here must be this interface. And if it's not, we're gonna raise an exception We have type hints. But that's like an editor or like tool helper, it doesn't mean that runtime, it's not going to work or it won't compile or whatever. Yeah. Right. And so I feel like mocking and patching is like, it's sort of, it's more embraced in this world where it's like, Look, if we just do this little thing, we can avoid all this complexity, and all of these design patterns and all of this stuff that technically would be maybe a more official, pure computer science way. But if you look at the stuff that gets built from it, it gets really hard to deal with it ugly.

10:31 Yeah. Yeah, I agree.

10:33 Yeah, I remember I worked on this one project. It was originally written, I think, in Java and then converted to C#, and it used, I think, every design pattern that you could possibly name and every single thing that use some sort of dependency, it had an interface for that dependency, it would pass that in into the constructor. And then when it created objects, it would pass that further down. And what you ended up with was, the entire code that you wrote was all this abstract stuff. And you would look at it like, well, I see all the functions, but I just don't even know where I would go to find out what actually does this. It's just it was so insanely complicated. I so I've seen that far in. And I've seen the world where people just go, you know, okay, we're not going to set up all those structures. But we're just going to do a patch, like deep down inside of some function call. And if I had to pick one of those two worlds, I would definitely live in the world where we just keep things simple as it's, it was not fun to work on that project in that way.

11:32 It sounds terrible.

11:33 Yeah, it was like, probably a six month project. I'm just like, I'm constantly frustrated on this project. So let's get started the beginning, like, what is mocking? There's a lot of folks who listen who come from sort of scientific background where they haven't done official computer science, or they're self taught maybe, maybe this idea is new to them. What is what's mocking?

11:53 Yeah, so a mock simulates the existence and behavior for real object, I guess we can go over a few examples. But if you use mocking you, as a developer can improve the quality of your tests and also test code in a controlled environment. And it's especially useful if you have external dependencies, like if you want to write to a database, or do stuff like that. And there's a nice Python library, and which is included in unit test, which is called unit test dot mock, which can be used for mocking. I think there are also some more specialized libraries out there. I think there's one, especially for mocking date time, but I've never used it. So I'm only using unit test dot mock.

12:34 Yes. I think the one out there called that is called freeze gun.

12:39 Yes, exactly. Yeah, that's it.

12:40 Yes, freeze gun. And that one, basically, like you said, allows you to control the date time module. So there's obvious dependencies that you would think of like, if I have a traditional web application, maybe it calls some API, like my online courses thing, it's going to call stripe, it's probably going to call MailChimp API's does some other stuff, it's going to talk to a database. So when you think about dependencies, you think, okay, database, maybe the file system, external API's, and those are all things you can mock, and you probably want to but when I first got into this world, one of the things that really surprised me was how hard it is to work with time. If I want to say, I would like to call this thing before, like, let's just say it's ecommerce, like, I want to make sure that this discount code is not expired now. So it gives me the discount, but in a week, it will be expired. You know, like, that's really hard to deal with. Right?

13:35 Yeah, I think that's a very good place to use mocking for like, it's a great example for that, where it can be really useful.

13:41 Yeah. And the alternative, I talked about this dystopian world that I lived in for a while, the alternative would be you would have to create a fake class that reads like that you use to get what now is right? Instead of just saying date, time dot now you're to create a class like time provider now, and then everybody has to agree to not use date time now, but they'll have to share this other thing that you're going to pass around. And that's just, that's just crazy, right? Like, it makes no sense. That's

14:09 unnecessarily complicated.

14:12 Yeah, it's so complicated. And so you can if you just mock out what now means that's it? Yeah. Then you're good. It's fixed. Right? So yeah, it's pretty cool. And freeze gun is one of the libraries I'm sure there's a bunch out there. But that's a cool one for doing this kind of stuff with specifically with time. This portion of talk Python to me is brought to you by linode. Whether you're working on a personal project or managing your enterprises infrastructure, linode has the pricing support and scale that you need to take your project to the next level, with 11 data centers worldwide, including their newest data center in Sydney, Australia, enterprise grade hardware, s3 compatible storage and the next generation network linode delivers the performance that you expect at a price that you don't get started on the node today with a $20 credit and you get Access to native SSD storage, a 40 gigabit network industry leading processors, their revamped Cloud Manager cloud not linode.com root access to your server along with their newest API and a Python COI, just visit talk python.fm slash linode when creating a new linode account, and you'll automatically get $20 credit for your next project. Oh, and one last thing they're hiring, go to lynda.com slash careers to find out more, let them know that we sent you. So you talked about replacing dependencies, I guess, give us some more examples. Time is one that maybe doesn't first come up. But where does some of the type of dependencies you all were working with.

15:39 So I've been working a lot with the Google Cloud client. So where you want to write or either you want to communicate with the Google Cloud with different services there, or you want to write to the storage, and you need to someone, you have a client and you want to communicate with the Google Cloud. And that's the part where we mostly use mocking. So to somewhat make sure that we test that the file is written or at least that we have the correct paths, the correct function call, but we don't actually write one to write the file to Google Google Cloud every time I remove it.

16:13 Right. You know, cloud computing, I think it has a really interesting challenge around testing, just in a lot of ways, right? Like you don't if you're gonna work with an API, you obviously have to not talk to the real API, you don't really want to create a virtual machines or a Kubernetes, clone node or whatever. But you do want to test that that code is going to work. But at the same time, a lot of times the tests depend on something meaningful coming back. Yeah. Right, is there's so much going on in the cloud side that I think it's pretty tricky. Would you guys do anything special there. I know, in AWS, there's like a fake local, AWS type thing you can run. And I know I've talked about it a while ago, and I forgot what it's called. But do you guys do anything like that?

16:54 Not at the moment, I think someone is working on finding ways to improve this, since we are using mocking quite a lot. But it's a really complicated and complex thing, since we have a huge code base already. And you would have to change all the code again. So doing kind of refactoring at this point is always a big deal. But right now we have just

17:15 yeah, it might not even be worth it. Yeah,

17:17 yeah. Yeah, exactly.

17:19 Yeah. And I think there's also something to be said for that, right. Like, sometimes it's like, we could create a mock or Faker. We'll talk about the different terminologies here in a minute. Yeah. But at sometimes it's like, you know, what, I'm just gonna let it write to the file system, it's not the end of the world, we're gonna keep going. The really one of the really tricky ones, I think, has to do with data access, right? I'm gonna go talk to the database and get this and then I need to get this other thing that's related to that. And it's, I think it's a constant challenge to figure out how much of that you replace and how much of it you just test against the database or test against a file or something.

17:54 Yeah, it's sometimes difficult to find the right way. Also, if you're working in a team and other, like different people have different opinions and like different things. But there's also a middle ground somewhere.

18:05 Yeah, for sure. Yeah, I could see one person on the team is all about, like, we have to do it the right way. Where there's no external dependencies and the other person's like, we also have to get stuff done. You know, I also have to Yeah, I can't spend all my time recreating all the systems that you're trying to talk to in ways that are sufficiently accurate so that when you call them the test data comes back, but it's as accurate or realistic enough that you're really getting a meaningful test. So it's tricky. So you have a example class, speaking of Harry Potter, and all that, yes, that we're going to use when it's kind of build up some ideas. And it's always hard to talk about code in audio format. So we'll keep it simple. But maybe just give us a quick introduction.

18:46 Yeah, so we have a very simple spell class, which, where you have just the constructor in the beginning. And a spell has of course, a name, it has an incantation and a description. So for example, I think there's a queue in Harry Potter, where it moves an object towards the person who is casting the spell. And a description could be what I just said. And then you have the incantation to move object towards person.

19:13 Yeah, exactly,

19:14 exactly. So that would be an example.

19:16 Okay, cool. All right. So we'll build up on this as we go through these different ideas here. And we've been talking about mocking as if it's all kind of the same. But if you dig into it, there's actually like a spectrum of ways in which you can do mocking. One of those might be I just when I call this function, I want it to not write to the log on the file system. And it might be enough that it just doesn't do that. You don't care what it does. It just doesn't do that. On the other hand, there's like, very advanced, tricky usage. Like when I call this function, I want to make sure that it checks whether the person is an admin. If I say yes, I want to do one thing. I want to make sure that it actually always, always calls this function to me. Make sure that it checked for them, even though you couldn't observe that as a side effect, I want to make sure that it's like calling this once and this twice or in this order. Like, there's a lot of tricky things. And those feel like very different things. And so in the mocking world, there's different names for these things, right?

20:16 Yes. So and there's also disagreement, I guess about the terminology, and also the definitions of these different kinds of mocking behaviors that you can distinguish.

20:26 Yeah. And it may or may not be useful, like people can decide how much they care about being very precise about calling it one thing or the other. But yeah, let's talk about it in the fine grained view, just so people get the full exposure, and they could decide to ignore the differences if they want, right?

20:43 Yes, I think that's a good idea. So yeah, I think the overall name or the name for these kinds of mocking behaviors is called test doubles. So there are the fine grained view objects, which are not real, can be either dummies, fakes, stubs, mocks or spies. And all of these so called test doubles. Yeah. And as I just said, these definitions are also controversial. And there are some times so different sources describe them slightly differently. But I think it's still possible to get an idea of what they are about. Yeah,

21:19 yeah, absolutely. So let's start guessing at the simple side of things. I feel like this whole concept has a little bit of a I'm making fun of you. thing so mocking to like, insult somebody. And then you've got Yes, you've got dummies, you've got fakes. So let's start with the dummies.

21:37 Yeah, I think the dummies are the easiest to understand. I agree. Yeah. And a dummy is just an object which is passed around but never actually used. So the dummy is not intended to be used in your tests, and it does not have any effect on the behavior of the test. And an example would be when you have attributes that you need to instantiate a class, but you don't really care about what they are, for example, for the spell class, if you need a spell instance, in your tests, but you don't care about the description, then you could just pass in an empty string or write whatever you want to in that string. And that would be then a dummy,

22:15 right? If you don't supply it, it's going to crash. Yes. But you don't actually ever use it or care about it. Just have to make it Get out of the way and keep working. Yes, yeah. Okay, so that's dummies. Next up is fakes.

22:28 Yes. So a fake implements a fake version of a class or method. It has a working implementation, but it takes some kind of shortcut such that it's not suitable for production. And that could be in a memory database. So usually, in production, you would have a database that you write your files to. But during testing, you could have some in memory database that you only use during your tests.

22:53 Yeah, that makes a lot of sense. And that's definitely one of the tricky things, I think, is databases. The other one that would be really tricky, would be, say, an API call, like, for example, one of the things I have to do in my code is I have to figure out, where's this person located physically, so that I can pick the right video server to deliver the fastest video to them. And so there's a call to go to the API that tells me where they are based on their IP address, like what country basically they're in so I can work out what went to send, right. Every time I run a test, I don't want to call that API. Right. Yeah. And I don't really

23:31 that's also a good example.

23:32 Yeah. I mean, I probably the test doesn't actually care where it said they were. But it's, you know, it's going to have to have some kind of behavior, because it's going to have to return a value. So the program works like to maybe I could come up with a mock, or a fake specifically that says, whenever they ask where they are, they're always in Kansas City, it doesn't matter where they really are. Just tell them it's in Kansas City, so that something can happen where the thing keeps working, right?

23:57 Yes, that's also somewhat a good introduction of the next one steps where the stub returns, like it has some pre programmed behavior, most of the times they simply return fixed values, or can data, like you just said, for example, always returned Kansas City. And, yeah, let's look at the spell class. Again, maybe the spell class could have a method get similar spells, which searches some database, and then you would get similar spells back. And this is, of course, quite complex, since finding these similar spells will probably be very difficult. And you would

24:34 write maybe the real version to running machine learning or yes, some sort of crazy system. And you don't want to happen all the time, right?

24:42 Yes, since it would probably slow down your whole testing. And then you could just replace the real implementation with a stub that returns hard coded values and would only take a fraction of the time to complete.

24:55 Yeah, yeah, that seems like a really good example there. So I don't know. I You have a hard time knowing exactly. Okay, I've got this idea which one does it fit into? Oh, it's almost it's almost like a spectrum saying that colors purple, and that colors red. What is the color? That's purple, but sort of towards the red? You know what I mean? Yeah. Yeah. Cool. All right. So making our way across this rainbow, I guess we have mocks?

25:22 Yes. So mocks are closely related to stubs. And there's this huge stack overflow post on the differences between mocks and stubs. And there, you can also get a bit of an idea of the controversy surrounding this topic. But yeah, and

25:36 I'll put that link in the show notes so people can find it.

25:38 Yeah, that's a good idea. So a mock does not have predetermined behavior, which steps have, instead it has to be configured. So an important difference between mocks and stubs is that a mock records which calls have been executed, so it can be used to verify not only the result, that's something that stuff can do too. But MCC can verify how the result was achieved, so that the correct methods have been invoked on the mock object.

26:06 Yeah, and this is where it starts to get complicated, I think, is one of the ideas of just programming in general functions and classes. But also testing is you shouldn't depend upon the internal implementation of a thing when you're testing it, or calling it from the outside. So if I have a thing that says, register a new user, maybe I want to just check if I give it valid information, it gives me a new user. If I give it like a malformed email, it'll give me an exception, right? Like, that would be totally straightforward. But all of a sudden, with the this level of mocking, you're like, well, I want to make sure that it checks that the user is not already existing, that this other thing is happening. And then you're getting more and more tied to the internal details of what that piece is doing. Which is fine, maybe, but then you change your code. And all of a sudden, all your tests break, you're like, well, now we got to go rewrite these tests, because we don't have this thing that we were checking that got called inside. And it gets this sort of, they kind of get glued together a little bit more, which I think can make maintenance hard.

27:10 Yeah, yeah, we had the problem, too. So yeah, that's something you have to think about too.

27:15 But at the same time, there's good uses for this, like I came up with that admin example, because one of the things that scares me a lot is you've got some protected part of an API or an application or something. And somebody forgets to check whether someone is permitted to go in there. That's really bad. If that ever gets discovered. You don't I mean, like, yeah, oh, did we forget to check in the show me all the users in their private information part, but we forget to check if you're logged in, whoops, you know, that kind of stuff could totally happen. And with this, you could say, you are going to have a test for every one of these admin pieces that says, You checked whether or not they were an admin, right? That function was called no matter what, right? Yep. This portion of talk Python, to me is sponsored by monday.com monday.com is an online platform that powers over 100,000 teams daily work, it's an easy to use flexible and visual teamwork platform, beautifully designed to manage any team organization or online process. Now for most of us, we missed our chance to build the first apps ever in the mobile app stores. It was a once in a lifetime opportunity. But it's one that's coming around again, monday.com is launching their marketplace and running a contest for the best new apps featured right from the get go. Want to be one of the first in the monday.com Apps Marketplace, start building today, they're even giving away $184,000 in prizes, including three, Tesla's 10, MacBooks. And more, build your idea for an app and get in front of hundreds of thousands of users on day one, start building today by visiting monday.com slash Python, or just click the link in your podcast players show notes. And so these marks, I think that's the thing you could do. Right? You could say like, expect that this was was it a cert called once or something like that, right?

29:02 Yes, there are different kinds of assert calls there. Okay, we can talk about this later as well. So there's a cert called once you can assert that was called with certain arguments that it was called, not call that it was called, like, independent of the number of times so there are kind of different things you can do. And that can be very helpful,

29:23 right? I forgot about that. You have the reverse as well. You can say I want to make sure in this situation. It never ever calls this other function. Yeah. Right. Right. Yeah, that's true. That would be another good type of test. Okay, so that's mocks. And then we have spies, which is next level stuff. Yeah.

29:40 Yeah, I find them pretty hard to grasp. So I haven't used spies yet myself. But from what I read, it's that spice I used to rep real objects and by default, they wrote route all the method calls to the original object, so they somewhat intercept and recalled record all calls that are made To the real object.

30:01 Yeah. Okay, so this sounds like, you might want to do those verification things that I was talking about, but not actually change the implementation, just let it do what it's gonna do. But you can kind of record that all the stuff was interacted with in certain ways or something like that.

30:17 Yeah, it's kind of it's a bit similar to a mock in that sense that it, you can assert that certain things are called, but it does not replace the original object. That's what mocks are doing.

30:29 Yeah, one of the tricky things with a lot of these and the more you use them, the more you have to deal with it is, well, they're gonna call this function. So what are you going to return from them? Right? So if they're gonna ask is admin, well, you have to say if they call is admin, return true or return false. Right? And so you have to sort of at the beginning of your test, you have to configure the mock to say, they're going to try to do these four things to you. Here's the answer. There's a return value or throw an exception if they try that or something along those lines. Exactly. Yeah, that's it. I think it's, you know, there's certain types of errors you want to test for that are really tricky, right? Like, I want to see a a sequel operational error, like, how am I going to make it do that? without talking to the real database? Well, you could come up with a mock that just says if they call Connect, throw this exception, right away, right? Like, that'd be a good example, I guess.

31:23 Definitely.

31:24 Okay. So we have this spectrum of mocks, the dummies, the spies, the fakes, and so on, when should we use them?

31:32 So you can use a mark? Like, I'm not talking about a mark, not all of the other things, but I guess it depends on whether you want to all call them mocks or not. But yeah, you would use mocking whenever you don't actually want to call an object. For example, when you have to spell class and you want to, or you have a function in that class that saves some version of the spell, for example, as a JSON file, and you have a function removed, which deletes that file again, then when you test those methods, you don't actually want to write the file to disk every time the test runs. And the same holds for functions that remove objects. But that's just one example of many things that you where you could use mocks, when you don't actually want to do the operation that you're testing.

32:21 Yeah. Basically, when any behavior the system is in the way of you tested it, either. That's you just don't want to call an API, or you just don't want to write to the file system, or something like that. Right? Yeah. Yeah. I guess the other one is, if you need to reach down inside the system and make it do something unexpected, like I said, like, if you want the Connect call to throw a sequel exception, it's hard from the outside potentially, to set it up in that way. But with a mock, you can just say if they call this function, throw this exception, see how the system deals with it. Right?

32:52 Yeah, that's also a very good use case.

32:54 Yeah, I think testing errors. A lot of people don't test for errors. I feel like they'd be like, Okay, well, this is what it's supposed to do. So let's test that. And I think testing for the opposite is almost as important.

33:05 Yeah, definitely. I agree.

33:06 Yeah. Cool. So when I look at mocking there's actually multiple ways to do mocks. And I, I honestly don't know when I should be using them.

33:16 Yeah. So there are three core functionalities and the unit test dot mock library. One is the class mock. One is the class magic mock. And then there is the patch function. And those have different properties. But maybe we can just go through them one by one. I think that's the easiest.

33:34 Yeah, yeah. Well, if you think, Okay, I'm gonna mock something, you probably would start with a mock class. Right? That's where do you think you would start?

33:40 Yeah, I think that's the best start. So you can use the mock class to mock any object where the mock object. So the class instance, just simulates the object it replaces. And I love this, since it's also kind of magic, since the mock to achieve this to really simulate the object, it creates attributes on the fly. Yeah. So when you let's say, you import the unit test mock library, and you create an instance of the mock class, and then you can just call some attribute on it, let's call it fancy attribute. And it was just create a child mock and return it to you. So it would create this fancy attribute on the fly. And that the same holds for any kind of method. And you can call methods with different kinds of inputs, who you can just do whatever you want to do. And I really like that I find that really fascinating.

34:32 Yeah, it's quite interesting. And you don't have to do too much to set it up. You don't have to anticipate everything that's going to happen to it. You just create one. And if people think there's a function there, hey, guess what, all of a sudden, like there's a function there. Yeah, the only part where it gets tricky is where you have to make sure the function returns something. So you don't get like, you know, none attribute doesn't have whatever type of errors. Yeah, yeah,

34:54 it's really powerful the mock class but that also sometimes makes it dangerous, too. Use, since you can really just call an everything on it. And yeah, there's also some problems with it. But I guess we can talk about that separately.

35:10 Yeah, absolutely. One of the challenges, I think is, let's say you're, you've got some kind of database class, and you're going to make calls on it, it would be easy to create a mock and say, here's the mock database, you call stuff on it, it's just going to kind of go along with that. The challenge is, how do you provide that through? Right? You know, like, I was talking about that, that crazy system that everything is passing every dependency everywhere, abstractly, all right, in this world, you've got to find a way to get the mock down inside the system so that it'll use it in the right place.

35:42 Yeah. And you have to configure it exactly like you want it to behave.

35:46 Right. Right, if they call this with these parameters, return this value, right?

35:51 Yeah. Since that's also something I stumbled across in the beginning. So let's say you create a mock object that mocks the JSON library. So you would just have Jason equals mock, where mock is the class. And then you could just call Jason dot dumps and give it any kind of argument you want to. But it's not the actual JSON dot dumps method. So it has nothing to do with that implementation. And you could just give it any kind of input that variables. And that's a bit confusing. I think at the beginning, you have to understand that when you create a mock object, and you haven't configured it yet, it has nothing to do with original function or class you're replacing.

36:34 That's right. In the dump s, that one's fine, right is no problem. You don't usually don't care in a test what happens but the reverse the load, yeah, do an over load something, you got to get something back to work with afterwards. Right? Exactly. Yeah, that's tricky. The other thing, though, that you can do with these mocks, is you can set a side effect. Yes. Which I think is actually cool.

36:54 Yeah, you can set all kinds of things. So you can set a return value. We talked about that earlier, but also the exceptions. So when you For example, one that a certain exception is thrown, thrown, when you call the mock object, then you can just set a side effect. That's a property of the mock class, and then you can set it to any kind of exception you want.

37:16 Yeah, that's a really good way to test. If this error happens at this function call, what are we gonna do? Yeah, it's great. So we've got that's the standard mock. And when you call one of these functions, like you said, the, let's say JSON load s, it'll give you back something, probably, which is another mock. But if you print it out, it'll just say that it's a mock of that. And it has, you know, just be kind of useless, right? Yeah. So the next level up would be to bring in some magic.

37:45 Yes. So there is the second class called Magic mock, which is actually subclass of the mock class, but it contains all the magic methods pre created and ready to use. So for example, if you want to compute the length of an object, then the magic Mark could be very useful since the Dunder Len Lang, how do you call that English? Dunder language? Yeah, Len. Yeah. Let's call it Dunder Len. It's already pre implemented and ready to use.

38:16 Yeah, yeah, exactly. So all the so called Magic methods, understood and delenn, Dunder repper, all those things are there, right?

38:24 Yes. And usually, I use the mock class if I don't need magic methods, and otherwise, I use the metric mock. But when you look, for example, step further, and you look at the patch method that actually returns a magic mark. So you're also fine using the magic mark, but it's then sometimes implementing things that you actually don't need.

38:45 Right, right. Maybe that makes it a little bit slower? I don't know. I haven't tried, but possibly,

38:48 yeah, that might be the case. I also haven't tried it yet.

38:52 Alright, so the one that I end up using directly, most often, it's got to be the patch, you didn't test out mocked out patch, right? This one is the Yeah, I'm not gonna pass a bunch of things around, but just reach down inside this thing. I know, it's gonna try to call this function or create one of these things. Just make that at whatever level it's going to be down there. Just make that work, make that go away, or make it return this value instead, and so on.

39:17 Yeah, I also mostly use the patch method. And I like it, because you can be very specific about what you want to actually patch. So it's very easy to see from the patch statement, where you're going in the code to change its behavior. And I think I like this being most precise, or as precise as possible, with the mocking and or the patching.

39:41 Yeah. And I feel like it doesn't force you into these design patterns, like I described that like just exploding complexity and say, Well, if you're going to have a dependency that you want to replace, you have to pass the dependencies around so you can pass the mock versions of them. And that's fine when there's one but then one thing requires two more which requires four and just explodes into like all this stuff, right? And so with patch, you don't have to have the structure of every single possible dependencies passed everywhere it goes, you can just say, if you see this function called, or this object created deep down inside, do this and said,

40:15 Yeah, so that's also actually what the patch function is doing. So it looks up, you give it a path, and it looks up the object in this given module and replaces that object with a magic mock. So it's, by default, a new magic mock, but you can also configure this. So you can create some class or then configure the patch statements such that it returns not a magic mark, but an instance of the class that you hand it.

40:42 Oh, interesting. So you can say I'm going to get exact like a little test double like sort of thing. I'm going to create an if I call this function, it has this little vague implementation or something and you say use that.

40:53 Yes. So you don't have to use the magic mock.

40:56 Okay. Yeah, that's pretty cool. Actually, I didn't realize I could do that. There's different ways you can use it. It could be a decorator, context manager. Other stuff, right?

41:04 Yes. So the syntax in general is that you have a patch, and then in brackets, and as a string, you have package dot module dot target. And there's a really nice example from one of the Python talks from Lisa Roach, I think it's called demystifying the patch function. I really like that talk. We can also link it in the show notes. Yeah. And yeah, so let's say you have an example file, which is called just example.pi. And you have one import statement from dB, import dB, right? So you want to write to some database. And then you have single function, which is called foo, or whatever you want to call it. And it just called dB, right? And returns the return value that you get from dB, right?

41:48 I like this example, because it's so simple, but actually uncovered some tricky bits.

41:52 Yeah, it brings the point across very well. And it's so easy to understand, since you don't actually have to care about how DB right is implemented. But you know, it writes to database of, it's something that you don't want to do in the test.

42:04 Yeah, exactly.

42:06 Yeah. So we, in this case, we would want to patch or mark the DB right call. And you can do that with a decorator, you can do that with context manager or with manual starting and stopping and which one you use would depend on the scope. So how the scope looks like that you want the object to be patched in?

42:27 Yeah. So it seems like there's a couple of ways that are pretty straightforward, kind of comparable, you could do a decorator and say, on your test function, put the decorator say, patch this, this object or this function call, or you could do a context manager inside your test function. And I guess it really depends on how many things are you patching? How complicated is it to set up some of the stuff and so on, but they seem pretty comparable, those two and the other is I could turn it on and turn it off. But this is a like a system wide change of what that function means. And so if for some reason, the test fails, and you don't have a try finally, to turn it off, or something like that, it's gonna be bad, because it'll permanently change what that function means, which maybe you only wanted it for a few. Yeah, but I guess the scenario I see that you might want to do it is like test set up and test tear down like for this whole class, we're going to just set it up and no one has to worry about it. You just write all the tests. So maybe that's the turn it on, turn it off scenario.

43:22 Yeah, I think that was also the scenario where I've seen it since then it can be useful. Otherwise, you would always have to apply the patch decorator again, or use the context manager for each test. And that looks very messy. Yeah. And if you can just do it centrally in a single place. That's, I think, the nicest solution.

43:41 Yeah, it can get super complicated about setting these things up. Because first, you're going to call this and it needs to return that value. But if it gets called with other value than return the like, there's a lot of stuff that can get done on these. And it's, you know, maybe simple for one. But if you have three or four of them all of a sudden, like there's a lot of setup to do a little test.

43:59 Yeah. And it can also become messy to have all these patch statements stacked on top of each other, I think two or three or maybe fine, but then it becomes really messy. Yeah,

44:11 yeah, absolutely. So I guess I'm starting to come around to the idea of having it just set up in the test setup, or maybe a test fixture and pi test or something like that.

44:21 Yeah, that's also a nice way.

44:22 Yeah. Another thing that I think's interesting about this example is, it seems so simple, but you have to be a little bit aware of how those functions are being used internally, right. So the import statement in the example that py talks about says from DB import dB, right? And you have to explicitly say in a string, the full like, namespace style name of the thing that it's going to patch. So in this example, you would say example.db, right? If you get it wrong, it crashes, right?

44:53 Yeah. I think that's what most people find confusing in the beginning. But I think it's great because you start to understand what it means if you import something, and what is happening in the program, and I found it very helpful to just inspect then the module and what kind of attributes it has. So, it is important to know that you have to patch where the object is looked up. And this might not be the place where it is defined. So if we have this DB right example, where you have the from DB import DB right statement in the file example.pi, then you are importing the DB right function from the DB module. So the consequence is that the example module now knows about the DB write method, but it does not know about dB. So, if you patch it, but I think it's nice that you can see that actually, so, if you open an ipython shell, and you import this example module, then you can call directory. So, Dir dir,

46:00 yeah, how do you pronounce that dirt? I would say durva. I could be wrong. Yes, sir.

46:04 Okay, I will trust you on that. So you would call her example. And then you would see a list of the attributes for this example module. And today, you will see dB, right and foo, but you won't see dB. So therefore, to mock the call to the DB write function in the test, you would have to patch example.db. Right,

46:26 right. And it's tricky, because conceptually, what you think you're patching is the DB right thing out of the DB module. So my first thought was, well, that should probably be just DB dot, right? But that doesn't work in this case. Yeah. But what's interesting is, if they had said, import dB, and then in the implementation of foo, they said, db.db, right, then it would work. And actually this example where you say, example, db, right? patch, that it would crash and say, well, that thing doesn't have this function, so we can't patch it. And so you have to be a little bit aware of how these are written. And like you said, where they're defined, it's, it does take a little bit awareness. So I think you can look with her, which is great. You just also have to just be a little bit aware of what's going on there.

47:12 Yeah, I think once you wrap your head around it, then it's easy. Once you understand the basic idea, and you have used it a few times, then it will become natural, or much easier to understand which patch statement you have to use. Yeah, I think we can look at another example where you would not import from DB import dB, right? But you would just import the DB module. And then the Foo function would call db.db. Right? Instead of just dB, right? And now the example module knows about dB, but not dB, right? So again, if you would call her on example, you would get DB and foo in the namespace, but you would not have dB, right? So now, when you patch it, you would either have to use example.db.db, right? Or just db.db. Right? And, yeah, it can be a bit confusing, but I think that's also the hardest thing to understand. And once you get that, then you have the most important idea.

48:17 Right? Once you understand it, it's not too bad, I think. But it does, you do have to put some ideas together to figure out what to put there. Yeah, one of the challenges I think of the more that you kind of reach inside of things, and you make these changes, like we're saying here, your testing can become a little bit harder to maintain. So for example, suppose we have the first scenario where they had the from dB, import dB, right? Everything was fine. And then somebody decides we're gonna refactor the code to be more explicit and use the namespaces. And then they go change the example py to use the different types of imports and rewrite the functions to say db.db, right, versus just the function call. All of a sudden, your tests don't work anymore. You're like, what is going on here? And that's just the nature of this mocking stuff, like as you reach inside, make these changes. You're now somewhat dependent on the structure of the inside.

49:08 Yeah. Yeah. You always have to keep that in mind when you adapt the code.

49:11 Yeah. So I guess, maybe try to do as little as possible. Sometimes you have to do it to not talk to Google Cloud or not to charge the credit card for real. But you know, don't if you can avoid it. Do so maybe. I don't know, cuz it's just one more dependency, right?

49:27 Yes.

49:28 Oh, what about using mock versus patch?

49:31 So I don't think there are any clear rules. At least I have not come across any clear rules. I guess that's similar to the whole topic that it's somewhat controversial. But personally, I mostly use patch. Yeah, just as you just said, It's, I like it since it's so specific, and I rarely use mock and I don't think I ever used magic mock specifically. So I use mock. If we have this dependency injection. We talked about that. In the very beginning, for example, if you have a client, and I use that I use clients to interact with the Google Cloud. And then I can just instead of passing in the client, I can just pass in a mock object.

50:13 But Right, exactly.

50:15 In all the other cases, I use the patch function.

50:18 Yeah, I would totally hundred percent agree with that. And I think in Python, this place where you pass in all of your dependencies, it's less common than other languages. So I think that naturally makes patch a little more common.

50:32 Yeah. And might be the reason.

50:34 Yeah. Which is generally a good thing. So I think some other languages, it's very hard for them to do this idea of patching, so they have to pass all their stuff around. So

50:45 yeah,

50:46 we talked a little bit about some of the challenges and like, if the import statements like this, you got to mirror that in the way that you write your patch statement. And if it changes that changes, but there's also some other common problems, and some things to keep in mind. You want to run us through those?

51:01 Yeah, sure. So we saw that we can configure a mock object, so we can set the return value of side effects like exceptions, and so on. And the fact that the mock objects, create attributes and methods on the fly makes them very sensitive to make mistakes. So typical example is that you misspell something. So for example, you can call a search caught once on a patch, or mocked object. And you could just have a spelling mistake there. For example, you could have dessert called once with a single s instead of assert once, but the test will just pass, it won't raise an assertion error, because you just created a new method on the mock object that is called dessert called once, instead of calling the actual function.

51:48 Yeah, and this is really tricky, because the successful behavior is exactly the same as just calling a function that doesn't exist, it just do nothing. It's fine. But the verification won't happen, right? Because the only the verification will happen if you actually call the right one.

52:03 Yeah, I think also, a typical problem is that a mock object does not know the interface of your class. That's also what I tried to explain earlier that you have to understand that the mock object actually has nothing to do with the class that it replaces, but you have to configure it to behave like your class. So let's say going back to the spell class, when we called der on the mock object, then you can see all the kinds of attributes. So maybe, like, if you have ipython, open again, and you would import the spell class, you would import the patch function from unit test dot mock. And then you can just look at the attributes of a mock spell instance, by calling with patch and then spell class dot spell as mocked spell that would just be the name of this mock object. And then you could print the call of your on the mock spell object, and you would see all these different things that you can use assert any call assert, called once assert, quote, once with and so on. But you would actually

53:09 Yeah, but none of them have anything to do with spell.

53:13 Yes. So you wouldn't see all the methods that are included in the spell class, like safe remove or cast you of course, you have to cast a spell, be able to cast a spell. And that can be quite tricky, since it can lead to behavior that you don't want. So the magic Mark object that's returned by the patch call, which is completely unrelated to the spell class.

53:38 Yeah. And that's really tricky. So I've seen spec equals true and various testing things. And I didn't until just now really put together what that actually did. So you can sort of fix that sometimes,

53:51 yes. So some of the problems can be solved at least those with the spelling mistakes, and also that you want your class to somewhat look like the or you want to look want your mock to look somewhat like the class that it's replacing, you can use the spec equals true attribute and a patch call. And this will cause the magic mark to look like the object that is being patched. So it would cause this misspelled desert wants to raise an exception. And also, the you can only call methods with the correct arguments. And you can only call methods that are actually they exist in the class.

54:28 Yeah, exactly. So one of the problems with these mocks could be inside a function, you're calling something on what you think is the real like spell or the data access layer, or whatever. And you're calling a function that doesn't exist, or you misspelled it, and all of your tests pass, because the mock is like fine. We'll just you call the function that didn't exist, just like all the others. So we're just going to return a mock object there. But in production, when it really goes to run and it gets a real thing that has no Method there, it's going to crash. So with the spec equals true, you can be a little more restrictive and like catch some of those errors as well. Maybe,

55:07 yeah, what I find in the beginning, when I saw spec equals true, I thought, Oh, this is great, then I just have exactly the thing I'm patching. But it turns out, yeah, almost, that all the attributes you create in the constructor, so in the Dunder init call, are not contained in the mock class in the end. So that's due to the internal workings of spec and auto spec. And I think there's also spec set, I'm not sure how it works, but they when you use that, in the end, your mock object does not know about any of this dynamically created attributes.

55:43 Yeah, really, the only way it would know is either if it disassembled the the constructor, which would be kind of weird. Or if it actually called the constructor and those constructors could have side effects, like getting files or trying to open a database or whatever, right? So it can't really, yeah, do those two, it can't see them. I guess, if you really wanted it to work exactly. In that way, you could put all the field definitions as like type level, you know, as a class level stuff, and then just set them right. Like, that's not as common in Python. Sometimes it is, sometimes it isn't. But that would be the way to fix it, I guess.

56:18 Yeah. So yes, spec and spec set, like auto spec spec set spec, they can be used to solve some of these problems. But you should be aware that they also create problems that you might not be aware of in the beginning. And also I think it will slow down testing. I have not noticed that really that it slows them down a lot. But I guess it depends on how much you actually use this functionality.

56:44 Yeah, for sure how much you're doing computationally in your test versus just a whole bunch of creation of mocks to just call one little function? Yeah.

56:52 I also like besides spec, and auto spec, and all these other things, there's so much more that you can do with patching and mocking. And it's a huge topic, I guess you can have a whole nother podcast about how to prevent mocking, or how not to get into mock hell, there's also pay con talk on this. It's just Yeah, it's a huge topic.

57:14 Yeah, I can definitely see. Being a mock. I've been on both sides of it. And I think Python has a pretty good middle ground with the ability to patch stuff and not completely rewrite your code to deal with it. Yeah, it's, it's pretty nice.

57:27 Yeah. I also like using it.

57:28 Yeah. Cool. All right. Well, baby, what are your quick wrap up? just summarize all that before we wrap things up?

57:35 Yes, sure. So I guess we learned that mocking is controversial. And that it should be used with care not to just fix your badly written code. There are a few ways in which you can design your code such that you don't need patching all the time. Like the dependency injection, we talked about where you pass in a mock to client, as an example. Yeah. Oh, yeah. And we talked about these different naming conventions that you should not get confused by them. And maybe if you think it's unnecessarily complicated, then just don't use them. That's also fine. There are people on both sides of the spectrum.

58:12 Yeah, I've always found that the distinction trying to be very precise about Oh, no, this is a mock. And this is a fake. And this is a stub but it doesn't necessarily help you. But it just makes things a little bit messy. You know, you're trying to do to me, the big distinction is, are you just trying to stop something from being called? And maybe just return a value of Korea side effect? versus Are you trying to understand what functions have been called? Like, I want to make sure they always always call, you know, check for admin or something like that. Right? That, to me is a big distinction. Are you trying to observe the behavior or just the outcome?

58:48 Yeah, I agree. It's also because there are no clear cut definitions, it's all a bit blurry. And the definition so in some cases, similar, so it can be really hard to tear them apart and understand what each one is about. Yeah. So we also talked about the library, the mocking library unit test mock, and the three core functionalities mock magic, mock, and patch. And mocking can be or seemed confusing in the beginning. But once you understand the basics, for example, that you have to patch where something is used, which might not be where it's defined, then it can be really helpful, especially with production code, and I see that every day.

59:27 Yeah, awesome. Well, this is definitely a very powerful tool in the testing tool set. And it can make testing possible where it was impossible or much easier where it was previously really, really hard. So hopefully, this helps people write more testable code and test more their code.

59:44 Yeah. And maybe at least motivate them to look into what mocking is, and maybe see that they can use it as well.

59:52 Absolutely. All right. Before we call the show, you're gonna write some Python code and mock out some libraries. What editor Do you use,

59:58 I still use vim. Like, I think I will use it the rest of my life. It's hard to get away from it. All of my colleagues use PI charm now or most of them. But yeah, I will still continue use with

01:00:11 awesome and the notable pi package, or library. Yeah, I

01:00:15 guess I have to say unit test mock. But one thing I really like is that I was always using the ipython debugger for debugging, but a few like a month back, I switched to PDB plus, plus, yeah, we can also link that in the show notes if people don't know it yet. And what I really like it. So it's very easy to install, it just wraps around the standard PDB debugger. And it has a few nice features. I mostly love the sticky command. And when you so you enter the debugger, and you type sticky, and then you will, while you're debugging, you will see the code command stick to the top of the screen. And you can really see where you are in your code, stepping through the lines one by one. And I find that very helpful when debugging.

01:01:03 Yeah, that's cool. It's like a little bit of a blend between a GUI debugger and command line style debugger. Yeah, yeah, very good. I don't think I've heard of that one. That's, that's excellent. Thanks. All right, final call to action. People trying to write some tests. They want to get out there. They want to use mocking, what do they do?

01:01:19 I think a good start is the documentation. I think it's a very comprehensive one. There's also a real Python. How do you call a blog post? on the topic? Yeah, I think those two places are good to start. I just in general, I like the Python documentation. I always, in most cases, it's just the best source.

01:01:40 Yeah. Awesome. All right. Well, thanks again for being on the show and diving into the nuances and all the different names in the mocking world. It's been fun.

01:01:48 Yeah. Thanks for having me was fun.

01:01:50 Yeah, it's good to have you back. Bye. Bye. This has been another episode of talk Python. To me. Our guest in this episode was in Atlanta pop quiz. And it's been brought to you by linode and monday.com. Start your next Python project on the nodes state of the art cloud service, just visit talk python.fm slash linode. Li in Eau de, you'll automatically get a $20 credit when you create a new account. Build your idea for an app and get it in front of a hundreds of thousands of users on day one. Start building today at the monday.com marketplace by visiting monday.com slash Python. Want to level up your Python. If you're just getting started, try my Python jumpstart by building 10 apps course. Or if you're looking for something more advanced, check out our new async course the digs into all the different types of async programming you can do in Python. And of course, if you're interested in more than one of these, be sure to check out our everything bundle. It's like a subscription that never expires. Be sure to subscribe to the show, open your favorite pod catcher and search for Python. We should be right at the top. You can also find the iTunes feed at slash iTunes. The Google Play feed is slash play in the direct RSS feed net slash RSS on talk python.fm. This is your host Michael Kennedy. Thanks so much for listening. I really appreciate it. get out there and write some Python code.