We know our tests should be relatively independent from other parts of the system.
For example, running a test shouldn't generally call a credit card processing API and talk to a database
when our goal is just to test the argument validation.
And yet, your method that you want to test, it does all three of those things and more.
What do you do?
Some languages use elaborate dependency passing mechanisms and frameworks
that go under the banner of inversion of control and dependency injection.
In Python, we do sometimes have that, but it's much more common to just temporarily redefine
what those two functions do using patching and mocking.
On this episode, we welcome back Annalena Popkas to talk us through the whole spectrum of
test doubles, dummies, mocks, and more.
This is Talk Python to Me, episode 287, recorded August 24th, 2020.
Welcome to Talk Python to Me, a weekly podcast on Python, the language, the libraries, the
ecosystem, and the personalities.
This is your host, Michael Kennedy.
Follow me on Twitter, where I'm @mkennedy.
Keep up with the show and listen to past episodes at talkpython.fm.
And follow the show on Twitter via at talkpython.
This episode is brought to you by Linode and monday.com.
Please check out what they're offering during their segments.
It really helps support the show.
Do you want to learn Python, but you can't bear to subscribe to yet another service?
At Talk Python Training, we hate subscriptions too.
That's why our course bundle gives you full access to the entire library of courses for
one fair price.
That's right.
With the course bundle, you save 70% off the full price of our courses, and you own them
all forever.
That includes courses published at the time of the purchase, as well as courses released
within about a year of the bundle.
So stop subscribing and start learning at talkpython.fm/everything.
Anna Lena, welcome back to Talk Python to Me.
Thanks.
It's great to be back.
Yeah, it's super to have you back.
Last time we spoke about 100 days of code and Harry Potter and things like that, if I recall
correctly.
Is that right?
Yes, that's correct.
Yeah, that was really fun.
And that was right at the height of the 100 days of code when everyone was really using
that to learn to code.
And so you had this unusual, but I think really engaging way of taking code and making it not
just silly technical stuff, but making it this fun Harry Potter adventure world.
Yeah, I still love that approach.
So I always try to find something I'm passionate about and then trying to use it to learn.
Yeah, excellent.
So now we're going to come back and talk about writing code that we're more likely to be
sure that works, testing code and using mocking and all that.
But when we spoke last time, you were a, was it an internship at Microsoft?
What kind of?
It's something in between.
It was called an AI residency at Microsoft Research in Cambridge in the UK.
So it's like an intensive one year program where you learn a lot about how to apply machine
learning, how to become a great research engineer.
So cool.
It's like, yeah, it's more than an internship, like a really long extended internship.
I'm sure that was a really fun project experience.
Oh yeah, it was amazing.
Also the place Microsoft Research in Cambridge is just great.
I really loved it there.
Yeah, I can imagine.
And how about now?
What are you up to now?
Yeah, so I finished the residency and I moved back to Germany and joined a German company
called InnoVex.
I also like it here a lot.
And I'm working as a machine learning engineer, similar to what I did at Microsoft Research,
but not so research heavy.
So I'm doing more production code or production work.
And I'm currently working in a pure data engineering project.
So I've been learning a lot more about the work that comes before actually applying the machine
learning algorithms, which is also very interesting.
Yeah.
So I would guess that involves a lot of pandas, a lot of cleaning up data, like getting stuff
in the right format.
Yes.
Yeah.
And then we give the data to the data scientists, which can just use the clean data to do their
nice machine learning stuff.
Yeah, you make them look good.
You're like, oh, here, you just feed this over and it works great.
Like it didn't start out that way.
What kind of problems are you trying to solve or answer?
It really depends on what the customer wants.
So it can go from supply chain management and apply machine learning there to right now in
the cloud, where we just create kind of a data lake in the Google cloud, where we have different
data sources that have to be processed and brought together and all the other stuff.
So there's just a lot going on.
It's a very huge project with several teams working on it.
Wow.
Yeah.
It sounds really fun.
And it sounds like one of these big data projects that people talk about a lot, but not
that many people actually do.
Yeah.
And it's also very interesting for me to see this other side.
I've been working on machine learning for most of my computer science life.
So seeing something different is also always very nice.
Yeah, absolutely.
And you also have some project going on in your spare time, yeah?
Yes, exactly.
So Kai Macht Schule.
Yeah, it's I think like it's a German project.
I'm doing voluntary work there.
It's called Kai Macht Schule.
That's like, I think you could translate it as AI goes to school.
Yeah, where we teach AI and machine learning skills to kids in German schools.
So we do classes usually on site, but now online due to the COVID crisis.
And yeah, it was somewhat similar to what you talked about in the last episode or some recent
episode with Nick Winter on Code Combat.
Yeah, Code Combat is really a fun thing for kids as well.
I think that's super neat what they're doing.
I want to try it as well, even if I know how to write code in Python.
It sounded so much fun.
Yeah, it's really neat.
I really enjoyed just kind of poking around with it and seeing it's, again, kind of like
your Harry Potter world.
It's a very different way to present programming to kids where you still type.
It doesn't feel like the burden of being exactly right.
You can kind of type it wrong and it'll mostly correct it for you.
So it's pretty interesting.
So with this project, you're teaching kids AI and machine learning.
What does that even look like?
What kind of stuff are you teaching them?
What do they get when they're building little libraries and so on?
Yeah.
So we try different approaches.
We always have a bit of theory.
So what is AI?
How does it work?
We also have a part on the ethics behind AI, which I think is really important.
And there's also some practical exercises with Jupyter Notebooks.
Before we used an approach was where you have these blocks of programming parts, which you
can stick together, like the for loops and so on.
But we didn't like that so much.
So right now we are using Jupyter Notebooks.
And I think it's amazing since right now there's one, it's called KI Camp.
It's like a four-day course.
And the youngest one is in fifth grade.
So it's like 11 or 12.
And the oldest one is just about to graduate.
And it's so nice to see these kids fascinated by this topic.
Yeah.
It's definitely going to give them a leg up, learn these types of things early and know
about them and just to learn about Jupyter and computational thinking at all.
Yeah.
And Germany is also really not so good with the computer science work in school.
I think in the US you are much better with teaching kids how to code in school already.
For German kids, it's really hard to get to know AI or to find somewhat a way to get familiar
with the topic without going out there and looking for yourself what's out on the internet.
Yeah.
There's the small percentage of kids who will do that.
They'll go find it themselves.
But the majority don't even know they would be interested until they get exposed to it.
Exactly.
Yeah.
It was the same for me.
So that's why I'm really passionate about this project.
Yeah.
Awesome.
And you've definitely taken it quite far what you're doing these days.
That's definitely the pinnacle.
It's great.
All right.
Well, let's focus on our main topic for a little while on mocking.
So how'd you get interested in it?
How'd you get started?
Yeah.
So I stumbled upon this topic a few months back when I started working in this data engineering
project I was just talking about.
And we have lots of production code there.
And I never heard of mocking before.
So it was the first time I used it.
And since then, I've been using it in almost all of the tests I write.
It's a very controversial topic.
So I watched quite a few PyCon talks on it and read quite a few blog posts.
And yeah, it's just a big but very interesting thing to talk about.
Yeah.
So I think one of the controversies has to do with you can write your code in certain ways
so it's more easy to test or it's harder to test.
And some people see mocking as kind of covering over the bad code you've written or something
to that effect, right?
Hopefully that sort of summarizes it.
Yeah, exactly.
And I completely understand that.
So you shouldn't use mocking to fix your badly written code.
So you, of course, you should first think about, is my code good?
Do I want to refactor it?
Are there changes I can make to make it easier to test?
But sometimes it can be a really good tool.
I think it's a great tool.
I think when it's used in the right ways and right places, it's just what you need.
Yes, I agree.
Yeah.
I mean, Python is so much about the sort of practicality beats purity.
You know, it's part of the Zen of Python and all that.
And it's just, let's just do what we need to do to make stuff work.
And I think one of the examples where this is really different from other languages is in
Python, we don't have things like protected variables.
We don't have as often like concrete interfaces.
We don't have type enforcement, right?
Like the thing you passed here must be this interface.
And if it's not, we're going to raise an exception.
We have type hints, but that's like an editor or like tool helper.
It doesn't mean that runtime it's not going to work or it won't compile or whatever.
Yeah.
Right.
And so I feel like mocking and patching is like, it's sort of, it's more embraced in this
world where it's like, look, if we just do this little thing, we can avoid all this complexity
and all of these design patterns and all of this stuff that technically would be maybe a
more official, pure computer sciencey way.
But if you look at the stuff that gets built from it, it gets really hard to deal with it
and ugly.
Yeah.
I agree.
Yeah.
I remember I worked on this one project and it was originally written, I think in Java
and then converted to C#.
And it used, I think every design pattern that you could possibly name.
And it used every single thing that used some sort of dependency.
It had an interface for that dependency.
It would pass that in into the constructor.
And then when it created objects, it would pass that further down.
And what you ended up with was the entire code that you wrote was all this abstract stuff.
And you would look at it like, well, I see all the functions, but I just don't even know
where I would go to find out what actually does this.
It's just, it was so insanely complicated.
So I've seen that far in and I've seen the world where people just go, you know, okay, we're
not going to set up all those structures, but we're just going to do a patch like deep down
inside of some function call.
And if I had to pick one of those two worlds, I would definitely live in the world where
we just keep things simple as it's, it was not fun to work on that project in that way.
It sounds terrible.
Yeah.
It was like a, probably a six month project.
I'm just like, I'm constantly frustrated on this project.
So let's, I guess, start at the beginning.
Like what is mocking?
There's a lot of folks who listen, who come from a sort of scientific background where they
haven't done official computer science or they're self-taught.
Maybe, maybe this idea is new to them.
What is, what's mocking?
Yeah.
So a mock simulates the existence and behavior of a real object.
I guess we can go over a few examples, but if you use mocking, you as a developer can
improve the quality of your tests and also test code in a controlled environment.
And it's especially useful if you have external dependencies, like if you want to write to a
database or do stuff like that.
And there's a nice Python library, which is included in unit test, which is called unit
test.mock, which can be used for mocking.
I think there are also some more specialized libraries out there.
I think there's one, especially for mocking date time, but I've never used it.
So I'm only using unit test.mock.
Yes.
I think the one out there called that is called freeze gun.
Yes, exactly.
Yeah.
That's it.
Yes.
A freeze gun.
And that one basically, you know, like you said, allows you to control the date time module.
So there's obvious dependencies that you would think of.
Like if I have a traditional web application, maybe it calls some API, like my online courses
thing.
It's going to call Stripe.
It's probably going to call MailChimp APIs, does some other stuff.
It's going to talk to a database.
So when you think about dependencies, you think, okay, database, maybe the file system,
external APIs, and those are all things you can mock and you probably want to.
But when I first got into this world, one of the things that really surprised me was how
hard it is to work with time.
If I want to say, I would like to call this thing before, like, let's just say it's e-commerce.
Like I want to make sure that this discount code is not expired now.
So it gives me the discount, but in a week it will be expired.
You know, like that's really hard to deal with.
Right.
Yeah.
I think that's a very good place to use mocking for like, it's a great example for that,
where it can be really useful.
Yeah.
And the alternative, I talked about this dystopian world that I lived in for a while.
The alternative would be, you would have to create a fake class that read, like that you
used to get what now is, right?
Instead of just saying datetime.now, you have to create a class like time provider now, and
then everybody has to agree to not use datetime.now, but they'll have to share
this other thing that you're going to pass around.
And that's just, that's just crazy.
Right.
Like it makes no sense.
Yeah.
It's unnecessarily complicated.
Yeah.
It's just so complicated.
And so you can, if you just mock out what now means.
That's it.
Yeah.
Then you're good.
It's fixed.
Right.
So yeah, it's pretty cool.
And Freezegun is one of the libraries.
I'm sure there's a bunch out there, but that's a cool one for doing this kind of stuff with
specifically with time.
This portion of Talk Python to Me is brought to you by Linode.
Whether you're working on a personal project or managing your enterprise's infrastructure,
Linode has the pricing, support, and scale that you need to take your project to the next
level.
With 11 data centers worldwide, including their newest data center in Sydney, Australia, enterprise
grade hardware, S3 compatible storage, and the next generation network, Linode delivers the
performance that you expect at a price that you don't.
Get started on Linode today with a $20 credit and you get access to native SSD storage, a 40
gigabit network, industry leading processors, their revamped cloud manager at cloud.linode.com,
root access to your server, along with their newest API and a Python CLI.
Just visit talkpython.fm/Linode when creating a new Linode account and you'll automatically
get $20 credit for your next project.
Oh, and one last thing.
They're hiring.
Go to linode.com slash careers to find out more.
Let them know that we sent you.
So you talked about replacing dependencies.
I guess give us some more examples.
Time is one that maybe doesn't first come up, but what are some of the type of dependencies
you all were working with?
So I've been working a lot with the Google Cloud client.
So where you want to write or either you want to communicate with the Google Cloud with
different services there, or you want to write to the storage and you need to someone, you
have a client and you want to communicate with the Google Cloud.
And that's the part where we mostly use mocking.
So to somewhat make sure that we test that the file is written, or at least that we have
the correct paths, the correct function call, but we don't actually want to write the file
to the Google Cloud every time or remove it.
Right.
You know, cloud computing, I think it has a really interesting challenge around testing.
It just, in a lot of ways, right?
Like you don't, if you're going to work with the API, you obviously have to not talk to the
real API.
You don't really want to create a virtual machine or a Kubernetes node or whatever, but you do
want to test that that code's going to work.
But at the same time, a lot of times the tests depend on something meaningful coming back.
Yeah.
Right.
There's so much going on in the cloud side that I think it's pretty tricky.
Do you guys do anything special there?
Like I know in AWS, there's like a fake local AWS type thing you can run.
And I know I've talked about it a while ago and I forgot what it's called, but do you guys
do anything like that?
Not at the moment.
I think someone is working on finding ways to improve this since we are using mocking
quite a lot, but it's a really complicated and complex thing since we have a huge code base
already and you would have to change all the code again.
So doing kind of refactoring at this point is always a big deal.
But right now we are just...
Yeah, it might not even be worth it.
Yeah.
Yeah, exactly.
Yeah.
And I think there's also something to be said for that, right?
Like sometimes it's like we could create a mock or fake or we'll talk about the different
terminologies here in a minute.
Yeah.
But sometimes it's like, you know what?
I'm just going to let it ride to the file system.
It's not the end of the world.
We're going to keep going.
One of the really tricky ones I think has to do with data access, right?
I'm going to go talk to the database and get this and then I need to get this other thing
that's related to that.
And I think it's a constant challenge to figure out how much of that you replace and how much
of it you just test against the database or test against a file or something.
Yeah.
It's sometimes difficult to find the right way.
Also, if you are working in a team and other like different people have different opinions
and like different things, but there's also a middle ground somewhere.
Yeah, for sure.
Yeah.
I could see one person on the team is all about like we have to do it the right way where there's
no external dependencies.
And the other person's like, we also have to get stuff done.
You know, I also have to.
I can't spend all my time recreating all of these systems that you're trying to talk
to in ways that are sufficiently accurate so that when you call them, the test data comes
back, but it's as accurate or realistic enough that you're really getting a meaningful test.
So it's tricky.
So you have a example class speaking of Harry Potter and all that.
Yes.
That we're going to use.
We're going to kind of build up some ideas.
And it's always hard to talk about code in audio format.
So we'll keep it simple.
But maybe just give us a quick introduction.
Yeah.
So we have very simple spell class, which where you have just the constructor in the beginning.
And a spell has, of course, a name.
It has an incantation and a description.
So for example, I think there's Accio in Harry Potter, where it moves an object towards the
person who is casting the spell.
And a description could be what I just said.
And then you have the incantation.
Yeah, moves object towards person.
Yeah, exactly.
Exactly.
So that would be an example.
Okay, cool.
All right.
So we'll build up on this as we go through these different ideas here.
And we've been talking about mocking as if it's all kind of the same.
But if you dig into it, there's actually like a spectrum of ways in which you can do mocking.
One of those might be, I just, when I call this function, I want it to not write to the
log on the file system.
And it might be enough that it just doesn't do that.
You don't care what it does.
It just doesn't do that.
On the other hand, there's like very advanced, tricky usage.
Like when I call this function, I want to make sure that it checks whether the person is an admin.
If I say yes, I want to do one thing.
But I want to make sure that it actually always, always calls this function to make sure that it
checked for them, even though you couldn't observe that as a side effect.
I want to make sure that it's like calling this once and this twice or in this order.
Like there's a lot of tricky things.
And those feel like very different things.
So in the mocking world, there's different names for these things, right?
Yes.
And there's also disagreement, I guess, about the terminology and also the definitions of
these different kinds of mocking behaviors that you can distinguish.
Yeah.
And it may or may not be useful.
Like people can decide how much they care about being very precise about calling it one thing
or the other.
But yeah, let's talk about it in the fine grained view, just so people get the full exposure
and they can decide to ignore the differences if they want, right?
Yes, I think that's a good idea.
So yeah, I think the overall name or the name for these kinds of mocking behaviors is called
test doubles.
So there are the fine grained view objects, which are not real, can be either dummies,
fakes, stubs, mocks, or spies.
And all of these are so-called test doubles.
Yeah.
And as I just said, these definitions are also controversial.
And there are sometimes so different sources describe them slightly differently.
But I think it's still possible to get an idea of what they are about.
Yeah.
Yeah, absolutely.
So let's start, I guess, at the simple side of things.
I feel like this whole concept has a little bit of a I'm making fun of you thing.
So mocking to like insult somebody.
And then you've got dummies, you've got fakes.
So let's start with the dummies.
Yeah, I think the dummies are the easiest to understand.
I agree.
Yeah.
And a dummy is just an object which is passed around, but never actually used.
So a dummy is not intended to be used in your tests.
And it does not have any effect on the behavior of the test.
And an example would be when you have attributes that you need to instantiate a class,
but you don't really care about what they are.
For example, for the spell class, if you need a spell instance in your tests,
but you don't care about the description,
then you could just pause in an empty string or write whatever you want to in that string.
And that would be then a dummy.
Right.
If you don't supply it, it's going to crash.
Yes.
But you don't actually ever use it or care about it.
You just have to make it get out of the way and keep working.
Yes.
Yeah.
Okay.
So that's dummies.
Next up is fakes.
Yes.
So a fake implements a fake version of a class or method.
It has a working implementation, but it takes some kind of shortcut such that it's not suitable
for production.
And that could be in a memory database.
So usually in production, you would have a database that you write your files to.
But during testing, you could have some in-memory database that you only use during your tests.
Yeah.
That makes a lot of sense.
And that's definitely one of the tricky things, I think, is databases.
The other one that would be really tricky would be, say, an API call.
Like, for example, one of the things I have to do in my code is I have to figure out where
is this person located physically so that I can pick the right video server to deliver
the fastest video to them.
And so there's a call to go to the API that tells me where they are based on their IP address,
like what country basically they're in.
So I can work out what one to send, right?
Every time I run a test, I don't want to call that API, right?
Yeah.
And I don't really...
That's also a good example.
Yeah.
And I mean, probably the test doesn't actually care where it said they were.
But it's, you know, it's going to have to have some kind of behavior because it's going
to have to return a value so the program works.
Like, so maybe I could come up with a mock or a fake specifically that says whenever they
ask where they are, they're always in Kansas City.
It doesn't matter where they really are.
Just tell them it's in Kansas City so that something can happen where the thing keeps working,
right?
Yes.
That's also somewhat a good introduction of the next one, Stubs.
Stubs returns.
It has some pre-programmed behavior.
Most of the times they simply return fixed values or canned data.
Like you just said, for example, always return Kansas City.
And yeah, let's look at the spell class again.
Maybe the spell class could have a method get similar spells, which searches some database
and then you would get similar spells back.
And this is, of course, quite complex since finding these similar spells will probably be very difficult.
And you would.
Right.
Maybe the real version is running machine learning or some sort of crazy system.
And you don't want that to happen all the time, right?
Yes.
Since it would probably slow down your whole testing.
And then you could just replace the real implementation with a stop that returns hard-coded values and would only take a fraction of the time to complete.
Yeah.
That seems like a really good example there.
So I don't know.
I do have a hard time knowing exactly.
Okay.
I've got this idea of which one does it fit into.
It's almost like a spectrum saying that color is purple and that color is red.
What is the color that's purple but sort of towards red?
You know what I mean?
Yeah.
Cool.
All right.
So making our way across this rainbow, I guess, we have mocks.
Yeah.
So mocks are closely related to stops.
And there's this huge stack overflow post on the differences between mocks and stops.
And there you can also get a bit of an idea of the controversy surrounding this topic.
But yeah.
Yeah.
And I'll put that link in the show notes so people can find it.
Yeah.
That's a good idea.
So a mock does not have predetermined behavior, which stops have.
Instead, it has to be configured.
So an important difference between mocks and stops is that a mock records which calls have
been executed.
So it can be used to verify not only the result.
That's something that a stop can do too.
But a mock can verify how the result was achieved so that the correct methods have been invoked
on the mock object.
Yeah.
And this is where it starts to get complicated, I think.
One of the ideas of just programming in general functions and classes, but also testing is
you shouldn't depend upon the internal implementation of a thing when you're testing it or calling
it from the outside.
So if I have a thing that says register a new user, maybe I want to just check if I give
it valid information, it gives me a new user.
If I give it like a malformed email, it'll give me an exception, right?
Like that would be totally straightforward.
But all of a sudden with the this level of mocking, you're like, well, I want to make sure that it checks that the user is not already existing, that this other thing is happening.
And then you're getting more and more tied to the internal details of what that piece is doing, which is fine, maybe.
But then you change your code.
And all of a sudden, all your tests break.
You're like, well, now we got to go rewrite these tests because we don't have this thing that we were
checking that got called inside and it gets this sort of they kind of get glued together a little bit more, which I think can make maintenance hard.
Yeah.
We had that problem, too.
So, yeah, that's something you have to think about, too.
But at the same time, there's good uses for this.
Like I came up with that admin example, because one of the things that scares me a lot is you've got some protected part of an API or an application or something.
And somebody forgets to check whether someone is permitted to go in there.
That's really bad if that ever gets discovered.
You know what I mean?
Like, oh, did we forget to check in the show me all the users and their private information part?
We forget to check if you're logged in.
Whoops.
You know, that kind of stuff could totally happen.
And with this, you could say you are going to have a test for every one of these admin pieces that says you checked whether or not they were an admin.
Right.
That function was called no matter what.
Right.
Yeah.
This portion of Talk Python to Me is sponsored by Monday.com.
Monday.com is an online platform that powers over 100,000 teams daily work.
It's an easy to use, flexible and visual teamwork platform beautifully designed to manage any team, organization or online process.
Now, for most of us, we missed our chance to build the first apps ever in the mobile app stores.
It was a once in a lifetime opportunity, but it's one that's coming around again.
Monday.com is launching their marketplace and running a contest for the best new apps featured right from the get-go.
Want to be one of the first in the Monday.com apps marketplace?
Start building today.
They're even giving away $184,000 in prizes, including three Teslas, 10 MacBooks and more.
Build your idea for an app and get in front of hundreds of thousands of users on day one.
Start building today by visiting Monday.com slash Python or just click the link in your podcast player's show notes.
And so, these mocks, I think that's the thing you could do, right?
You could say, like, expect that this was, what is it, assert called once or something like that, right?
Yes, there are different kinds of assert calls there.
Okay.
We can talk about this later as well.
So, there is assert called once.
You can assert that was called with certain arguments, that it was called not called, that it was called, like, independent of the number of times.
So, there are kind of different things you can do.
And that can be very helpful.
Right.
I forgot about that.
You have the reverse as well.
You can say, I want to make sure in this situation, it never, ever calls this other function.
Yeah.
Right.
Yeah, that's true.
That would be another good type of test.
Okay.
So, that's mocks.
And then we have spies, which is next level stuff.
Yeah.
I find them really hard to grasp.
So, I haven't used spies yet myself.
But from what I read, it's that spies are used to wrap real objects.
And by default, they route all the method calls to the original object.
So, they somewhat intercept and record all calls that are made to the real object.
Yeah.
Okay.
So, this sounds like you might want to do those verification things that I was talking about, but not actually change the implementation.
Just let it do what it's going to do.
But you can kind of record that all the stuff was interacted with in certain ways or something like that.
Yeah.
It's kind of, it's a bit similar to a mock in that sense that you can assert that certain things are called, but it does not replace the original object.
That's what mocks are doing.
Yeah.
One of the tricky things with a lot of these, and the more you use them, the more you have to deal with it is, well, they're going to call this function.
So, what are you going to return from them, right?
So, if they're going to ask, is admin, well, you have to say, if they call, is admin, return true or return false, right?
And so, you have to sort of, at the beginning of your test, you have to configure the mock to say, they're going to try to do these four things to you.
Here's the answer.
Here's the return value.
Throw an exception if they try that or something along those lines.
Exactly.
Yeah.
That said, I think it's, you know, there's certain types of errors you want to test for that are really tricky, right?
Like, I want to see a SQL operational error.
Like, hmm, how am I going to make it do that without talking to the real database?
Well, you could come up with a mock that just says, if they call connect, throw this exception right away, right?
Like, that would be a good example, I guess.
Definitely.
Okay.
So, we have this spectrum of mocks, the dummies, the spies, the fakes, and so on.
When should we use them?
So, you can use a mock.
Like, I'm not talking about a mock, not all of the other things, but I guess it depends on whether you want to all call them mocks or not.
But, yeah, you would use mocking whenever you don't actually want to call an object.
For example, when you have the spell class and you want to, or you have a function in that class that saves some version of the spell, for example, as a JSON file,
and you have a function remove, which deletes that file again.
Then when you test those methods, you don't actually want to write the file to disk every time the test runs.
And the same holds for functions that remove objects.
But that's just one example of many things that you, where you could use mocks when you don't actually want to do the operation that you're testing.
Yeah.
Basically, when any behavior of the system is in the way of you testing it, either that's you just don't want to call an API, or you just don't want to write to the file system, or something like that, right?
Yeah.
I guess the other one is, if you need to reach down inside the system and make it do something unexpected.
Like I said, if you want the connect call to throw a SQL exception, it's hard from the outside, potentially, to set it up in that way.
But with a mock, you can just say, if they call this function, throw this exception, see how the system deals with it, right?
Yeah, that's also a very good use case.
Yeah, I think testing errors, a lot of people don't test for errors.
I feel like they're like, okay, well, this is what it's supposed to do, so let's test that.
And I think testing for the opposite is almost as important.
Yeah, definitely.
I agree.
Yeah, yeah, yeah.
Cool.
So when I look at mocking, there's actually multiple ways to do mocks, and I honestly don't know when I should be using them.
Yeah, so there are three core functionalities in the unit test.mock library.
One is the class mock, one is the class magic mock, and then there is the patch function.
And those have different properties, but maybe we can just go through them one by one.
I think that's the easiest.
Yeah, yeah.
Well, if you think, okay, I'm going to mock something, you probably would start with the mock class, right?
That's where you think you would start.
Yeah, I think that's the best start.
So you can use the mock class to mock any object, where the mock object, so the class instance, just simulates the object it replaces.
And I love this, since it's also kind of magic, since the mock, to achieve this, to really simulate the object, it creates attributes on the fly.
So when you, let's say you import the unit test.mock library, and you create an instance of the mock class, and then you can just call some attribute on it.
Let's call it fancy attribute.
And it was just create a child mock and return it to you.
So it would create this fancy attribute on the fly.
The same holds for any kind of method, and you can call methods with different kinds of inputs, so you can just do whatever you want to do.
And I really like that.
I find that really fascinating.
Yeah, it's quite interesting.
And you don't have to do too much to set it up.
You don't have to anticipate everything that's going to happen to it.
You just create one, and if people think there's a function there, hey, guess what?
All of a sudden, there's a function there.
The only part where it gets tricky is where you have to make sure the function returns something so you don't get like, you know, none attribute doesn't have whatever type of errors.
Yeah.
It's really powerful, the mock class, but that also sometimes makes it dangerous to use, since you can really just call in everything on it.
And yeah, there's also some problems with it, but I guess we can talk about that separately.
Yeah, absolutely.
One of the challenges, I think, is, let's say you've got some kind of database class, and you're going to make calls on it.
It would be easy to create a mock and say, here's the mock database.
You call stuff on it.
It's just going to kind of go along with that.
The challenge is, how do you provide that through, right?
You know, I was talking about that crazy system that everything is passing every dependency everywhere abstractly, right?
In this world, you've got to find a way to get the mock down inside the system so that it'll use it in the right place.
Yeah.
And you have to configure it exactly like you want it to behave.
Right.
If they call this with these parameters, return this value, right?
Yeah, since that's also something I stumbled across in the beginning.
So let's say you create a mock object that mocks the JSON library.
So you would just have JSON equals mock, where mock is the class.
And then you could just call JSON.dumps and give it any kind of argument you want to.
But it's not the actual JSON.dumps method.
So it has nothing to do with that implementation.
And you could just give it any kind of input variables.
And that's a bit confusing, I think, at the beginning.
You have to understand that when you create a mock object and you haven't configured it yet,
it has nothing to do with original function or class you're replacing.
That's right.
And the dumps, that one's fine, right?
There's no problem.
You usually don't care in a test what happens.
But the reverse, the load, when it loads something, you've got to get something back to work with afterwards, right?
Exactly.
Yeah, that's tricky.
The other thing, though, that you can do with these mocks is you can set a side effect.
Yes.
Which I think is actually cool.
Yeah, you can set all kinds of things.
So you can set a return value.
We talked about that earlier.
But also the exceptions.
So when you, for example, want that a certain exception is thrown when you call the mock object, then you can just set a side effect.
That's a property of the mock class.
And then you can set it to any kind of exception you want.
Yeah, that's a really good way to test.
If this error happens at this function call, what are we going to do?
Yeah, it's great.
So we've got, that's the standard mock.
And when you call one of these functions, like you said, let's say JSON load us, it'll give you back something, probably, which is another mock.
But if you print it out, it'll just say that it's a mock of that.
And it has, you know, just be kind of useless, right?
Yeah.
So the next level up would be to bring in some magic.
Yeah.
So there is the second class called magic mock, which is actually subclass of the mock class.
But it contains all the magic methods pre-created and ready to use.
So, for example, if you want to compute the length of an object, then the magic mock could be very useful.
Since the dunder len, how do you call that in English?
Dunder len, I guess?
Len?
Yeah.
Let's call it dunder len.
It's already pre-implemented and ready to use.
Yeah.
Yeah, exactly.
So all the so-called magic methods, dunder stir, dunder len, dunder repper, all those things are there, right?
Yes.
And usually I use the mock class if I don't need magic methods.
And otherwise I use the magic mock.
But when you look, for example, a step further and you look at the patch method, that actually returns a magic mock.
So you're also fine using the magic mock, but it's then sometimes implementing things that you actually don't need.
Right, right.
Maybe that makes it a little bit slower.
I don't know.
I haven't tried, but possibly.
Yeah, that might be the case.
I also haven't tried it yet.
All right.
So the one that I end up using directly most often, it's got to be the patch.
Unit test.mock.patch, right?
This one is the, I'm not going to pass a bunch of things around, but just reach down inside this thing.
I know it's going to try to call this function or create one of these things.
Just make that at whatever level it's going to be down there.
Just make that work.
You know, make that go away or make it return this value instead and so on.
Yeah, I also mostly use the patch method and I like it because you can be very specific about what you want to actually patch.
So it's very easy to see from the patch statement where you're going in the code to change its behavior.
And I think I like this being most precise or as precise as possible with the mocking or the patching.
Yeah, and I feel like it doesn't force you into these design patterns like I described.
Like just explode in complexity and say, well, if you're going to have a dependency that you want to replace, you have to pass the dependencies around so you can pass the mock versions of them.
And that's fine when there's one, but then one thing requires two more, which requires four.
And it just explodes into like all this stuff, right?
And so with patch, you don't have to have the structure of every single possible dependencies passed everywhere it goes.
You can just say, if you see this function called or this object created deep down inside, do this instead.
Yeah.
So that's also actually what the patch function is doing.
So it looks up, you give it a path and it looks up the object in this given module and replaces that object with a magic mock.
So it's by default a new magic mock, but you can also configure this so you can create some class or then configure the patch statement such that it returns not a magic mock, but an instance of the class that you hand it.
Oh, interesting.
So you can say, I'm going to get like a little test double like sort of thing that I'm going to create.
And if I call this function, it has this little fake implementation or something and just say use that?
Yes.
So you don't have to use the magic mock.
Okay.
Yeah, that's pretty cool.
Actually, I didn't realize I could do that.
There's different ways you can use it.
It could be a decorator, context manager, other stuff, right?
Yeah.
So the syntax in general is that you have patch and then in brackets and as a string, you have package.module.target.
And there's a really nice example from one of the PyCon talks from Lisa Roach, I think.
It's called demystifying the patch function.
I really like that talk.
We can also link it in the show notes.
Yeah.
And yeah, so let's say you have an example file, which is called just example.py.
And you have one import statement from db, import dbwrite.
So you want to write to some database.
And then you have a single function, which is called foo or whatever you want to call it.
And it just calls dbwrite and returns the return value that you get from dbwrite.
I like this example because it's so simple, but it actually uncovers some tricky bits.
Yeah, it brings the point across very well.
And it's so easy to understand since you don't actually have to care about how dbwrite is implemented,
but you know it writes to database and that's something that you don't want to do in the test.
Yeah, exactly.
Yeah.
So we, in this case, we would want to patch or mock the dbwrite call.
And you can do that with a decorator.
You can do that with context manager or with manual starting and stopping.
And which one you use would depend on the scope.
So how the scope looks like that you want the object to be patched in.
Yeah.
So it seems like there's a couple of ways that are pretty straightforward, kind of comparable.
You could do a decorator and say on your test function, put the decorator, say patch this,
this object or this function call.
Or you could do a context manager inside your test function.
And I guess it really depends on how many things are you patching, how complicated is it to set up some of the stuff and so on.
But they seem pretty comparable, those two.
And the other is I could turn it on and turn it off.
But this is a system-wide change of what that function means.
And so if for some reason the test fails and you don't have a try finally to turn it off or something like that, it's going to be bad because it'll permanently change what that function means, which maybe you only wanted it for a few.
Yeah.
But I guess the scenario I see that you might want to do it is like test set up and test tear down.
Like for this whole class, we're going to just set it up and no one has to worry about it.
You just write all the tests.
So maybe that's the turn it on, turn it off scenario.
Yeah, I think that was also the scenario where I've seen it.
Since then it can be useful.
Otherwise, you would always have to apply the patch decorator again or use it to context manager in each test.
And that looks very messy.
And if you can just do it centrally in a single place, that's, I think, the nicest solution.
Yeah, and it can get super complicated about setting these things up because first you're going to call this and it needs to return that value.
But if it gets called with other value, then return the, like there's a lot of stuff that can get done on these.
And it's, you know, maybe simple for one.
But if you have three or four of them, all of a sudden, like there's a lot of setup to do a little test.
Yeah.
And it can also become messy to have all these patch statements stacked on top of each other.
I think two or three are maybe fine, but then it becomes really messy.
Yeah.
Yeah, absolutely.
So I guess I'm starting to come around to the idea of having it just set up in the test setup or maybe a test fixture in pytest or something like that.
Yeah, that's also a nice way.
Yeah.
Another thing that I think is interesting about this example is it seems so simple, but you have to be a little bit aware of how those functions are being used internally.
Right.
So the import statement in the example that you talked about says from DB import DB write.
And you have to explicitly say in a string the full like namespace style name of the thing that it's going to patch.
So in this example, you would say example dot DB write.
And if you get it wrong, it crashes.
Right.
Yeah, I think that's what most people find confusing in the beginning.
But I think it's great because you start to understand what it means if you import something and what is happening in the program.
And I found it very helpful to just inspect then the module and what kind of attributes it has.
So it is important to know that you have to patch where the object is looked up.
And this might not be the place where it is defined.
So if we have this DB write example where you have the from DB import DB write statement in the file example dot pi, then you are importing the DB write function from the DB module.
So the consequence is that the example module now knows about the DB write method, but it does not know about DB.
So if you patch it, but I think it's nice that you can see that actually.
So if you open an IPython shell and you import this example module, then you can call directory.
So dear, dir.
Yeah.
How do you pronounce that?
I would say dir, but I could be wrong.
Yeah.
Dir.
Okay.
I will trust you on that.
So you would call dir example, and then you would see a list of the attributes for this example module.
And there you will see DB write and foo, but you won't see DB.
So therefore, to mock the call to the DB write function in the test, you would have to patch example dot DB write.
Right.
And it's tricky because conceptually, what you think you're patching is the DB write thing out of the DB module.
So my first thought was, well, that should probably be just DB dot write, but that doesn't work in this case.
Yeah.
But what's interesting is if they had said import DB, and then in the implementation of foo, they said DB dot DB write, then it would work.
And actually, this example where you say example dot DB write, patch that, it would crash and say, well, that thing doesn't have this function, so we can't patch it.
And so you have to be a little bit aware of how these are written.
And like you said, where they're defined, it does take a little bit of awareness.
So I think you can look with dir, which is great.
You just also have to just be a little bit aware of what's going on there.
Yeah, I think once you wrap your head around it, then it's easy.
Once you understand the basic idea and you have used it a few times, then it would become natural or much easier to understand which patch statement you have to use.
Yeah, I think we can look at another example where you would not import from DB import DB write, but you would just import the DB module.
And then the foo function would call DB dot DB write instead of just DB write.
And now the example module knows about DB, but not DB write.
So again, if you would call dir on example, you would get DB and foo in the namespace, but you would not have DB write.
So now when you patch it, you would either have to use example dot DB dot DB write or just DB dot DB write.
And yeah, it can be a bit confusing, but I think that's also the hardest thing to understand.
And once you get that, then you have the most important idea.
Right.
Once you understand it, it's not too bad, I think.
But it does.
You do have to put some ideas together to figure out what to put there.
Yeah.
One of the challenges, I think, of the more that you kind of reach inside of things and you make these changes, like we're saying here, your testing can become a little bit harder to maintain.
So, for example, suppose we have the first scenario where they had the from DB import DB write.
Everything was fine.
And then somebody decides we're going to refactor the code to be more explicit and use the namespaces.
And then they go change the example pi to use the different types of imports and rewrite the functions to say DB dot DB write versus just the function call.
All of a sudden, your test doesn't work anymore.
You're like, what is going on here?
And that's just the nature of this mocking stuff.
Like, as you reach inside to make these changes, you're now somewhat dependent on the structure of the inside.
Yeah.
You always have to keep that in mind when you adapt the code.
Yeah.
So, I guess maybe try to do as little as possible.
I mean, sometimes you have to do it to not talk to Google Cloud or not to charge the credit card for real.
But, you know, don't if you can avoid it, do so maybe.
I don't know, because it's just one more dependency, right?
Yes.
Well, what about using mock versus patch?
So, I don't think there are any clear rules.
At least I have not come across any clear rules.
I guess that's similar to the whole topic that it's somewhat controversial.
But personally, I mostly use patch, just as you just said.
I like it since it's so specific, and I rarely use mock.
And I don't think I ever use magic mock specifically.
So, I use mock if we have this dependency injection we talked about in the very beginning.
For example, if you have a client, and I use clients to interact with Google Cloud,
and then I can just, instead of passing in the client, I can just pass in a mock object.
But in all the other cases, I use the patch function.
Yeah, I would totally, 100% agree with that.
And I think in Python, this place where you pass in all of your dependencies,
it's less common than other languages.
So, I think that naturally makes patch a little more common.
Yeah, that might be the reason.
Yeah, which is generally a good thing.
Also, I think some other languages, it's very hard for them to do this idea of patching.
So, they have to pass all their stuff around.
So, we've talked a little bit about some of the challenges.
And like, if the import statement's like this, you got to mirror that in the way that you write your patch statement.
And if it changes, that changes.
But there's also some other common problems and some things to keep in mind.
You want to run us through those?
Yeah, sure.
So, we saw that we can configure mock objects.
So, we can set the return value, side effects like exceptions and so on.
And the fact that the mock objects create attributes and methods on the fly makes them very sensitive to mistakes.
So, a typical example is that you misspell something.
So, for example, you can call assert called once on a patch or mocked object.
And you could just have a spelling mistake there.
For example, you could have assert called once with a single S instead of assert called once.
But the test will just pass.
It won't raise an assertion error because you just created a new method on the mock object that is called assert called once instead of calling the actual function.
Yeah, and this is really tricky because the successful behavior is exactly the same as just calling a function that doesn't exist.
It's just do nothing.
It's fine.
But the verification won't happen, right?
Because only the verification will happen if you actually call the right one.
Yeah.
I think also a typical problem is that a mock object does not know the interface of your class.
That's also what I tried to explain earlier, that you have to understand that the mock object actually has nothing to do with the class that it replaces.
But you have to configure it to behave like your class.
So, let's say going back to the spell class, when we called dir on the mocked object, then you can see all the kinds of attributes.
So, maybe like if you have IPython open again and you would import the spell class, you would import the patch function from unit test.mock.
And then you can just look at the attributes of a mocked spell instance by calling with patch and then spell class.spell as mocked spell.
That would just be the name of this mocked object.
And then you could print the call of dir on the mocked spell object.
And then you would see all these different things that you can use.
Assert any call, assert call once, assert call once with, and so on.
But you would actually...
Yeah, but none of them have anything to do with spell.
Yes.
So, you wouldn't see all the methods that are included in the spell class, like save, remove, or cast.
Of course, you have to cast a spell.
Be able to cast a spell.
And that can be quite tricky, since it can lead to behavior that you don't want.
So, the magic mocked object that's returned by the patch call, which is completely unrelated to the spell class.
Yeah.
And that's really tricky.
So, I've seen spec equals true in various testing things.
And I didn't, until just now, really put together what that actually did.
So, you can sort of fix that sometimes.
Yes.
So, some of the problems can be solved, at least those with the spelling mistakes,
and also that you want your class to somewhat look like the...
Or you want your mock to look somewhat like the class that it's replacing.
You can use the spec equals true attribute in the patch call.
And this will cause the magic mock to look like the object that is being patched.
So, it would cause this misspelled assert call once to raise an exception.
And also, you can only call methods with the correct arguments.
And you can only call methods that actually exist in the class.
Yeah, exactly.
So, one of the problems with these mocks could be, inside a function,
you're calling something on what you think is the real spell or the data access layer or whatever.
And you're calling a function that doesn't exist or you misspelled it.
And all of your tests pass because the mock is like, fine.
We'll just...
You called a function that didn't exist, just like all the others.
So, we're just going to return a mock object there.
But in production, when it really goes to run and it gets a real thing that has no method there,
it's going to crash.
So, with the spec equals true, you can be a little more restrictive and catch some of those errors as well, maybe.
Yeah.
What I find in the beginning when I saw spec equals true, I thought, oh, this is great.
Then I just have exactly the thing I'm patching.
But it turns out, yeah, almost, that all the attributes you create in the constructor,
so in the Dunder init call, are not contained in the mocked class in the end.
So, that's due to the internal workings of spec and auto spec.
And I think there's also spec set.
I'm not sure how it works.
But when you use that in the end, your mock object does not know about any of this dynamically created attributes.
Yeah, really, the only way it would know is either if it disassembled the constructor,
which would be kind of weird, or if it actually called the constructor.
And those constructors could have side effects like creating files or trying to open a database or whatever, right?
So, it can't really do those.
So, it can't see them.
I guess if you really wanted it to work exactly in that way,
you could put all the field definitions as type level, as class level stuff,
and then just set them, right?
That's not as common in Python.
Sometimes it is, sometimes it isn't.
But that would be the way to fix it, I guess.
Yeah.
So, yeah, spec and spec set, like auto spec, spec set, spec,
they can be used to solve some of these problems.
But you should be aware that they also create problems
that you might not be aware of in the beginning.
And also, I think it will slow down testing.
I have not noticed that really, that it slows them down a lot.
But I guess it depends on how much you actually use this functionality.
Yeah, yeah, for sure.
How much are you doing computationally in your test versus just a whole bunch of creation of mocks to just call one little function?
Yeah.
I also like besides spec and auto spec and all these other things,
there is so much more that you can do with patching and mocking.
And it's a huge topic.
I guess you can have a whole another podcast about how to prevent mocking or how not to get into mock hell.
There's also PyCon talk on this.
It's just, yeah, it's a huge topic.
Yeah, I can definitely see being a mock hell.
I've been on both sides of it.
And I think Python has a pretty good middle ground with the ability to patch stuff and not completely rewrite your code to deal with it.
Yeah, it's pretty nice.
Yeah, I also like using it.
Yeah, cool.
All right.
Well, maybe you want to do a quick wrap up, just summarize all that before we wrap things up?
Yes, sure.
So I guess we learned that mocking is controversial.
And that it should be used with care, not to just fix your badly written code.
There are a few ways in which you can design your code such that you don't need patching all the time,
like the dependency injection we talked about, where you pass in a mock to client, as an example.
Yeah.
Oh, yeah.
And we talked about these different naming conventions that you should not get confused by them.
And maybe if you think it's unnecessarily complicated, then just don't use them.
That's also fine.
There are people on both sides of the spectrum.
Yeah, I've always found that the distinction, trying to be very precise about, oh, no, this is a mock and this is a fake and this is a stub.
It doesn't necessarily help you, but it just makes things a little bit messy.
Like, you know what you're trying to do?
To me, the big distinction is, are you just trying to stop something from being called and maybe just return a value or create a side effect versus are you trying to understand what functions have been called?
Like, I want to make sure they always, always call, you know, check for admin or something like that.
Right.
Like, that to me is a big distinction.
Are you trying to observe the behavior or just the outcome?
Yeah, I agree.
It's also because there are no clear cut definitions.
It's all a bit blurry.
And the definitions are, in some cases, similar.
So, it can be really hard to tear them apart and understand what each one is about.
Yeah.
So, we also talked about the library, the mocking library, unit test.mock, and the three core functionalities, mock, magic, mock, and patch.
And mocking can be or seem confusing in the beginning.
But once you understand the basics, for example, that you have to patch where something is used, which might not be where it's defined, then it can be really helpful, especially with production code.
And I see that every day.
Yeah.
Awesome.
Well, this is definitely a very powerful tool in the testing tool set.
And it can make testing possible where it was impossible or much easier where it was previously really, really hard.
So, hopefully, this helps people write more testable code and test more of their code.
Yeah.
And maybe at least motivate them to look into what mocking is and maybe see that they can use it as well.
Absolutely.
All right.
Before we call the show, you're going to write some Python code, mock out some libraries,
what editor do you use?
I still use Vim.
Like, I think I will use it the rest of my life.
It's hard to get away from it.
All of my colleagues use PyCharm now, or most of them.
But yeah, I will still continue using Vim.
Awesome.
And the notable PyPI package or library?
Yeah, I guess I have to say unit test.mock.
But one thing I really like is that I was always using the IPython debugger for debugging.
But a few, like a month back, I switched to PDB++.
Yeah.
We can also link that in the show notes if people don't know it yet.
And what I really like it.
So, it's very easy to install.
It just wraps around the standard PDB debugger.
And it has a few nice features.
I mostly love the sticky command.
And when you...
So, you enter the debugger and you type sticky.
And then you will...
While you're debugging, you will see the code command stick to the top of the screen.
And you can really see where you are in your code, stepping through the lines one by one.
And I find that very helpful when debugging.
Yeah, that's cool.
It's like a little bit of a blend between a GUI debugger and a command line style debugger.
Yeah.
Yeah, yeah.
Very good.
I don't think I've heard of that one.
That's excellent.
Thanks.
All right.
Final call to action.
People trying to write some tests.
They want to get out there.
They want to use mocking.
What do they do?
I think a good start is the documentation.
I think it's a very comprehensive one.
There's also a real Python...
How do you call it?
Blog post on the topic.
Yeah.
I think those two places are good to start.
I just...
In general, I like the Python documentation.
I always...
In most cases, it's just the best source.
Yeah.
Awesome.
All right.
Well, thanks again for being on the show and diving into the nuances and all the different
names in the mocking world.
It's been fun.
Yeah.
Thanks for having me.
It was fun.
Yeah.
It's great to have you back.
Bye.
This has been another episode of Talk Python to Me.
Our guest in this episode was Annalena Popkes.
And it's been brought to you by Linode and Monday.com.
Start your next Python project on Linode's state-of-the-art cloud service.
Just visit talkpython.fm/Linode.
L-I-N-O-D-E.
You'll automatically get a $20 credit when you create a new account.
Build your idea for an app and get it in front of hundreds of thousands of users on day one.
Start building today at the Monday.com marketplace by visiting Monday.com slash Python.
Want to level up your Python?
If you're just getting started, try my Python Jumpstart by Building 10 Apps course.
Or if you're looking for something more advanced, check out our new async course that digs into
all the different types of async programming you can do in Python.
And of course, if you're interested in more than one of these, be sure to check out our
Everything Bundle.
It's like a subscription that never expires.
Be sure to subscribe to the show.
Open your favorite podcatcher and search for Python.
We should be right at the top.
You can also find the iTunes feed at /itunes, the Google Play feed at /play,
and the direct RSS feed at /rss on talkpython.fm.
This is your host, Michael Kennedy.
Thanks so much for listening.
I really appreciate it.
Now get out there and write some Python code.
I'll see you next time.