WEBVTT 00:00:00.001 --> 00:00:02.580 Do you write tests for your code? You probably should. 00:00:02.580 --> 00:00:06.520 And most of the time, pytest is the industry standard these days. 00:00:06.520 --> 00:00:11.400 But pytest can be much more than what you just get from installing it as a tool. 00:00:11.400 --> 00:00:15.480 There are many amazing plugins that improve pytest in many aspects. 00:00:15.480 --> 00:00:19.460 That's why I've invited Brian Okken to the show to tell us about his favorites. 00:00:19.460 --> 00:00:23.600 Listen in and your Python testing will be faster, stronger, and more beautiful. 00:00:23.600 --> 00:00:29.060 This is Talk Python To Me, episode 267, recorded June 4th, 2020. 00:00:29.060 --> 00:00:48.480 Welcome to Talk Python To Me, a weekly podcast on Python, the language, the libraries, the ecosystem, and the personalities. 00:00:48.480 --> 00:00:52.560 This is your host, Michael Kennedy. Follow me on Twitter where I'm @mkennedy. 00:00:52.560 --> 00:00:56.320 Keep up with the show and listen to past episodes at talkpython.fm. 00:00:56.320 --> 00:00:58.800 And follow the show on Twitter via at Talk Python. 00:00:59.200 --> 00:01:01.980 This episode is brought to you by Linode and Sentry. 00:01:01.980 --> 00:01:04.140 Please check out what they're offering during their segments. 00:01:04.140 --> 00:01:05.780 It really helps support the show. 00:01:05.780 --> 00:01:08.040 Brian, welcome back to Talk Python To Me. 00:01:08.040 --> 00:01:09.820 Thanks. It's fun to be back. 00:01:09.820 --> 00:01:10.920 Yeah, it's good to have you back. 00:01:10.920 --> 00:01:12.300 You've been on a lot of episodes. 00:01:12.300 --> 00:01:14.240 You've been on five. 00:01:14.240 --> 00:01:20.280 2017 Python Year in Review, 2018 Python Year in Review, the Python Testing Column is a Thing, 00:01:20.280 --> 00:01:25.120 30 Amazing Python Projects, which is one of the most popular episodes, actually. 00:01:25.760 --> 00:01:28.460 And the book author panel discussion. 00:01:29.040 --> 00:01:32.220 So happy to have one more on the list to have you here. 00:01:32.220 --> 00:01:33.140 Yeah, it's nice. 00:01:33.140 --> 00:01:33.860 I love your show. 00:01:33.860 --> 00:01:34.660 Listen to it all the time. 00:01:34.660 --> 00:01:34.940 Thanks. 00:01:34.940 --> 00:01:36.380 Yeah, and it's good to catch up with you. 00:01:36.380 --> 00:01:36.880 It's been a while. 00:01:36.880 --> 00:01:38.960 Yeah. 00:01:38.960 --> 00:01:41.860 Well, we, I mean, we talk to each other weekly. 00:01:41.860 --> 00:01:42.860 That's right. 00:01:42.860 --> 00:01:43.440 Of course. 00:01:43.440 --> 00:01:43.920 Of course. 00:01:44.400 --> 00:01:44.620 Yeah. 00:01:44.620 --> 00:01:50.340 So those of you listening who don't know about Python Bytes, that's our other podcast that 00:01:50.340 --> 00:01:51.340 Brian and I co-host. 00:01:51.340 --> 00:01:53.600 We each have separate podcasts, right? 00:01:53.600 --> 00:01:54.780 Talk Python and I do alone. 00:01:54.780 --> 00:01:57.540 You do Test and Code, which is really good. 00:01:57.540 --> 00:01:57.700 Yes. 00:01:57.700 --> 00:01:58.720 Where do people find Test and Code? 00:01:58.720 --> 00:02:00.960 At testandcode.com. 00:02:01.080 --> 00:02:01.320 Perfect. 00:02:01.320 --> 00:02:06.980 But we also do Python Bytes, which is like a newsletter for Python with analysis and audio 00:02:06.980 --> 00:02:07.260 form. 00:02:07.260 --> 00:02:08.000 It's a lot of fun. 00:02:08.000 --> 00:02:09.100 Yeah, it is. 00:02:09.100 --> 00:02:10.300 People should check that out as well. 00:02:10.300 --> 00:02:15.560 So I want to have you tell your story because you probably told it four times out of five there 00:02:15.560 --> 00:02:15.900 anyway. 00:02:15.980 --> 00:02:22.160 So let's just sort of check in and see what you've been up to in the last year or so. 00:02:22.160 --> 00:02:24.080 It's been actually pretty exciting. 00:02:24.080 --> 00:02:29.020 I think I've told the story before, but I work at Roden Schwartz. 00:02:29.020 --> 00:02:30.780 So I work at a test and measurement company. 00:02:30.780 --> 00:02:34.780 I'm an embedded software developer and a team lead and a lead engineer. 00:02:34.780 --> 00:02:39.180 But I've been focusing for the last 10 to 15. 00:02:39.180 --> 00:02:42.380 Half of my career has been on communication testing. 00:02:43.180 --> 00:02:47.340 Most of it's been cellular stuff and we're switching over the last year or so. 00:02:47.340 --> 00:02:50.680 I've been focusing mostly on Wi-Fi testing. 00:02:50.680 --> 00:02:52.020 So that's been a lot of fun. 00:02:52.020 --> 00:02:52.600 Okay, nice. 00:02:52.600 --> 00:02:54.420 Like Wi-Fi 6, that kind of stuff? 00:02:54.420 --> 00:02:54.960 Yes. 00:02:54.960 --> 00:03:00.820 One of the fun things lately is some of the work on 802.11x, which is just starting to roll 00:03:00.820 --> 00:03:02.600 out around the world in different places. 00:03:02.600 --> 00:03:03.660 Yeah, that's cool. 00:03:03.660 --> 00:03:05.020 I'm actually very excited about that. 00:03:05.020 --> 00:03:05.540 Yeah. 00:03:05.540 --> 00:03:05.840 Yeah. 00:03:05.840 --> 00:03:06.340 It's neat. 00:03:06.340 --> 00:03:12.120 Also, one of the reasons why I kind of focus on testing a lot is I am a proponent of developing 00:03:12.120 --> 00:03:17.180 software with testing hand in hand so that you develop tests and code at the same time. 00:03:17.180 --> 00:03:17.540 Yeah. 00:03:17.540 --> 00:03:24.140 It's always very error prone and unlikely to succeed to say, now that you've written the 00:03:24.140 --> 00:03:26.120 code, let's go back and add all the tests to it. 00:03:26.120 --> 00:03:27.140 Yeah, definitely. 00:03:27.140 --> 00:03:30.940 But people don't learn how to write tests when they're in. 00:03:30.940 --> 00:03:35.920 I mean, even college classes hardly ever talk about how to write good tests. 00:03:36.100 --> 00:03:39.500 Yeah, there's definitely that sort of splintering there. 00:03:39.500 --> 00:03:44.440 Like on one hand, the stuff you're building in college, it's probably not going to be long 00:03:44.440 --> 00:03:45.080 lived, right? 00:03:45.080 --> 00:03:48.120 It's an assignment or a short project. 00:03:48.120 --> 00:03:52.540 You just want to get it done because you also got that English paper and you didn't really 00:03:52.540 --> 00:03:53.480 even want to take English. 00:03:53.480 --> 00:03:54.860 But now you're in there and you got to do it. 00:03:54.860 --> 00:03:55.760 So let's just get it done. 00:03:55.760 --> 00:04:00.760 And it's a very different thing to think a team of people have to live with this for a 00:04:00.760 --> 00:04:01.440 period of time. 00:04:01.660 --> 00:04:02.380 Yeah, definitely. 00:04:02.380 --> 00:04:07.400 And with a more and more reliant on, it's a little bit easier of a thing for me to try 00:04:07.400 --> 00:04:10.620 to tell people the importance because they're used to working. 00:04:10.620 --> 00:04:14.360 A lot of people are used to working with open source projects now and you would never pull 00:04:14.360 --> 00:04:18.060 an open source project off the shelf and use it if it had zero tests around it. 00:04:18.060 --> 00:04:18.280 Yeah. 00:04:18.280 --> 00:04:23.300 And a lot of times you've got to have tests to accompany your contributions back if you're 00:04:23.300 --> 00:04:24.140 doing a PR, right? 00:04:24.140 --> 00:04:24.460 Yeah. 00:04:24.460 --> 00:04:25.420 Yeah, definitely. 00:04:25.420 --> 00:04:27.540 It's getting more and more attention. 00:04:27.540 --> 00:04:28.560 And I think that's a good thing. 00:04:28.560 --> 00:04:30.700 What's the work from home story for you these days? 00:04:30.860 --> 00:04:32.160 Do you got to go to the office? 00:04:32.160 --> 00:04:36.720 I mean, the world scrambled for everyone, but the only part of my world that hasn't changed 00:04:36.720 --> 00:04:42.240 is my work because I've already been fully distributed and whatnot, I guess, other than 00:04:42.240 --> 00:04:42.640 conferences. 00:04:42.640 --> 00:04:43.400 But how about you? 00:04:43.400 --> 00:04:43.720 Yeah. 00:04:43.720 --> 00:04:45.500 So it was a hard shift. 00:04:45.500 --> 00:04:48.980 I think it was early March, was it? 00:04:48.980 --> 00:04:50.100 Really early March. 00:04:50.100 --> 00:04:54.480 There was a day where we just got told, grab your stuff and go home. 00:04:54.640 --> 00:04:57.920 And we're working from home from now on for a while, at least. 00:04:57.920 --> 00:05:01.960 And we had just moved into a beautiful office in July. 00:05:01.960 --> 00:05:02.620 So it's... 00:05:02.620 --> 00:05:03.660 It is really nice. 00:05:03.660 --> 00:05:07.280 Like, it's a place you would want to go to because it's got cool decor and whatnot. 00:05:07.280 --> 00:05:07.660 Yeah. 00:05:07.660 --> 00:05:09.820 And these giant training rooms and stuff. 00:05:09.820 --> 00:05:11.240 It's a good building. 00:05:11.240 --> 00:05:13.500 So working from home, the entire... 00:05:13.500 --> 00:05:15.520 I had just hired a couple people for the team. 00:05:15.520 --> 00:05:18.320 And so training people remotely was... 00:05:18.320 --> 00:05:20.740 I thought it was going to be a trick, but it's actually been... 00:05:20.740 --> 00:05:21.800 It's gone really smoothly. 00:05:21.800 --> 00:05:22.400 Oh, that's good. 00:05:22.680 --> 00:05:24.740 And yeah, so we're a remote team. 00:05:24.740 --> 00:05:25.860 We are... 00:05:25.860 --> 00:05:27.100 So in our county... 00:05:27.100 --> 00:05:30.640 So I'm in the county, Washington County that we're in around Portland. 00:05:30.640 --> 00:05:33.460 We are supposedly entering phase one. 00:05:33.460 --> 00:05:36.740 We're doing this three-stage phase going back to work thing. 00:05:36.740 --> 00:05:37.800 We don't know. 00:05:37.800 --> 00:05:41.940 We haven't allowed people to go back yet, but hopefully there will be... 00:05:41.940 --> 00:05:45.160 There's some people that can more effectively work from the office. 00:05:45.160 --> 00:05:49.300 So we're going to try to get those people back as soon as we can, but do it safely. 00:05:49.300 --> 00:05:52.560 Do you think you'll have more work from home remote options? 00:05:53.420 --> 00:05:55.260 Generally, at the company because of this? 00:05:55.260 --> 00:05:56.020 Yeah, I think so. 00:05:56.020 --> 00:05:56.860 I think we've... 00:05:56.860 --> 00:05:57.680 One of the fears was... 00:05:57.680 --> 00:06:00.140 Then I think a lot of companies have seen this. 00:06:00.140 --> 00:06:02.940 The fear is that people working from home won't... 00:06:02.940 --> 00:06:04.320 They'll be more distracted. 00:06:04.320 --> 00:06:06.120 They won't be as productive. 00:06:06.120 --> 00:06:10.820 And I don't even know if this is a fair test because everybody's also... 00:06:10.820 --> 00:06:13.940 It's not just working from home, but their kids are home also. 00:06:13.940 --> 00:06:15.940 Yeah, exactly. 00:06:15.940 --> 00:06:21.340 There's also the stress of just what's going on in the world is a stressful situation. 00:06:21.620 --> 00:06:25.340 So even though it's not a fair comparison, we're comparing anyway. 00:06:25.340 --> 00:06:27.200 And things are going okay. 00:06:27.200 --> 00:06:32.400 We're still being productive, just about as productive or more so than we were before in 00:06:32.400 --> 00:06:33.280 a lot of cases. 00:06:33.280 --> 00:06:35.420 The lack of a commute is really easy. 00:06:35.420 --> 00:06:37.680 The ease of discussion. 00:06:37.680 --> 00:06:38.800 I mean, you can... 00:06:38.800 --> 00:06:39.160 I don't know. 00:06:39.160 --> 00:06:39.960 It just is... 00:06:39.960 --> 00:06:45.340 I get more time focused, but also I think I'm talking with my team more often now because 00:06:45.340 --> 00:06:47.780 we make it a point to talk every day as a team. 00:06:48.120 --> 00:06:48.720 Right, right. 00:06:48.720 --> 00:06:51.600 Focus type stuff, not just, oh, I passed them in the hall sort of thing. 00:06:51.600 --> 00:06:52.140 Yeah, that's cool. 00:06:52.140 --> 00:06:57.420 Now, one more thing, I guess while we're catching up on this stuff is you've got these really 00:06:57.420 --> 00:07:00.400 beautiful training rooms with nice giant screens. 00:07:00.400 --> 00:07:06.640 And one of the things that we were doing together is the Python Portland West meetup, which kind 00:07:06.640 --> 00:07:08.160 of got polished from all this as well, right? 00:07:08.380 --> 00:07:13.220 Yeah, and that's a big bummer because I think it was really serving a community in the west 00:07:13.220 --> 00:07:15.480 side of Portland that didn't have a meetup. 00:07:15.480 --> 00:07:17.680 And I would really like to... 00:07:17.680 --> 00:07:21.680 I have no idea when we're going to get back to being able to do that, but I'd like to be 00:07:21.680 --> 00:07:22.360 able to do something. 00:07:22.360 --> 00:07:24.200 Yeah, I don't even know when I can go to a restaurant. 00:07:24.200 --> 00:07:26.020 So, you know, things are all up in the air. 00:07:26.020 --> 00:07:27.980 The world is crazy. 00:07:27.980 --> 00:07:29.440 Yeah, definitely. 00:07:29.440 --> 00:07:30.280 Yeah. 00:07:30.280 --> 00:07:31.360 All right, cool. 00:07:31.360 --> 00:07:32.700 Well, good to catch up. 00:07:32.700 --> 00:07:35.120 Now, let's talk some pie tests. 00:07:35.220 --> 00:07:38.980 I'd like to do just a little bit of high-level stuff to just like set the stage, and then 00:07:38.980 --> 00:07:40.200 we're going to dive into the plugins. 00:07:40.200 --> 00:07:47.600 So, pytest is an external separate way to write and test Python code. 00:07:47.600 --> 00:07:49.820 Just not, say, part of Python like UnitTest. 00:07:49.820 --> 00:07:55.360 So, maybe let's just do like a quick survey of the modern testing tools that people might 00:07:55.360 --> 00:08:00.000 potentially use, and then like how we'd write tests in pytest for those who maybe have only 00:08:00.000 --> 00:08:03.240 done UnitTest type testing, or they're like, testing, maybe I should do that. 00:08:03.240 --> 00:08:04.260 Yeah, sure. 00:08:04.260 --> 00:08:04.300 Sure. 00:08:04.300 --> 00:08:07.560 Well, actually, you had a few on the list that I deleted, but... 00:08:07.560 --> 00:08:08.920 Because they're no longer modern, right? 00:08:08.920 --> 00:08:11.760 Well, I would say not. 00:08:11.760 --> 00:08:16.380 There still is UnitTest, and UnitTest is being supported, and I do think, and it's part of 00:08:16.380 --> 00:08:18.880 the Python distribution. 00:08:18.880 --> 00:08:21.260 That's the one winning point, I think, right? 00:08:21.260 --> 00:08:27.820 It's like, I don't have to have a requirements.txt or pip install anything, potentially, and I 00:08:27.820 --> 00:08:28.500 can still write tests. 00:08:28.640 --> 00:08:28.880 Yeah. 00:08:28.880 --> 00:08:34.260 One of the main reasons why it's there is to test Python and test the standard library. 00:08:34.260 --> 00:08:36.980 And I think that that's a good reason for it. 00:08:36.980 --> 00:08:41.800 There's a bunch of people that have built on tools around it, and I think that's fine, but 00:08:41.800 --> 00:08:43.540 it's just not my thing. 00:08:43.980 --> 00:08:48.320 One of the things, while we're talking about other testing app things, is it's a good thing 00:08:48.320 --> 00:08:54.580 to note that Nose is still available, but I don't recommend anybody use Nose or Nose2 or 00:08:54.580 --> 00:08:59.900 any of its variants, because, I don't know, Nose is not being supported at all anymore. 00:09:00.280 --> 00:09:06.620 And Nose2 has very little traction, and I don't know if it's, I don't currently know 00:09:06.620 --> 00:09:07.180 if it's active. 00:09:07.180 --> 00:09:12.920 So, if you're currently working on Nose, sorry about the disparaging remark, but maybe correct 00:09:12.920 --> 00:09:13.720 me if I'm wrong. 00:09:13.720 --> 00:09:14.300 Yeah, yeah. 00:09:14.300 --> 00:09:16.440 Well, I mean, the thing is, you've got all these choices, right? 00:09:16.440 --> 00:09:21.840 Like, if it's pytest or some of these other things like Nose or something, it's like, well, 00:09:21.840 --> 00:09:22.500 okay. 00:09:22.500 --> 00:09:24.140 Which one are you going to pick, right? 00:09:24.140 --> 00:09:27.220 First got into it, I really, I was looking for the easiest. 00:09:27.220 --> 00:09:30.920 And I thought that maybe the easiest and the most powerful would be two different things, 00:09:30.920 --> 00:09:35.420 but it turns out that pytest is both the easiest and the most powerful, which is nice. 00:09:35.420 --> 00:09:35.760 Nice. 00:09:35.760 --> 00:09:38.120 There's some other ones, though, that you might consider. 00:09:38.120 --> 00:09:38.640 Yeah. 00:09:38.640 --> 00:09:43.680 Well, in conjunction with pytest and other testing things, there's a lot of other tools that people 00:09:43.680 --> 00:09:44.040 use. 00:09:44.040 --> 00:09:50.280 Tox is used, it is not just a testing tool, but one of the neat things about it is you can 00:09:50.280 --> 00:09:53.380 do a test matrix or running different Python environments. 00:09:53.380 --> 00:09:55.820 It creates a, it tests a lot of your application. 00:09:56.380 --> 00:09:58.380 It's pretty, almost an episode in itself. 00:09:58.380 --> 00:09:58.600 Right. 00:09:58.600 --> 00:10:04.300 You can test on like various versions, like Python 3.5, 3.6, 3.7, and stuff with that, right? 00:10:04.300 --> 00:10:04.760 Yeah. 00:10:04.760 --> 00:10:10.100 And typically, if you're running, like running pytest, pytest doesn't automatically build your 00:10:10.100 --> 00:10:16.800 application for you, but Tox, part of its process is creating the wheel of your, like if you're 00:10:16.800 --> 00:10:21.220 doing a package, it'll create the package for you and then install it. 00:10:21.220 --> 00:10:24.460 And it does the whole round trip thing. 00:10:24.460 --> 00:10:25.280 So that's cool. 00:10:25.280 --> 00:10:25.480 Nice. 00:10:25.480 --> 00:10:26.120 Okay, cool. 00:10:26.120 --> 00:10:31.820 A lot of people are moving to, so continuous integration is definitely a thing that's important 00:10:31.820 --> 00:10:32.440 everywhere. 00:10:32.820 --> 00:10:39.200 And a couple years ago, it was probably Jenkins or Travis that people were using, but now 00:10:39.200 --> 00:10:44.080 I think more, a lot of people are moving to GitHub Actions because they're just right there 00:10:44.080 --> 00:10:44.960 with your code. 00:10:44.960 --> 00:10:45.940 So why not? 00:10:45.940 --> 00:10:47.060 Yeah, that's super easy. 00:10:47.060 --> 00:10:51.860 And it totally solves the problem that is, I think, one of the really big challenges of 00:10:51.860 --> 00:10:57.240 testing in teams is you get different levels of buy-in from different participants in the 00:10:57.240 --> 00:10:57.580 team. 00:10:57.580 --> 00:11:03.640 And some people are very, they just are reluctant to just run the test before they'll do a check-in. 00:11:03.640 --> 00:11:04.000 Yeah. 00:11:04.000 --> 00:11:07.620 They'll break the test, they'll do a check-in, and they don't even know they broke it, but 00:11:07.620 --> 00:11:08.360 they don't care that much. 00:11:08.360 --> 00:11:11.980 I mean, people care to differ in degrees, but it's like, well, they're kind of a nuisance, 00:11:11.980 --> 00:11:14.000 but I guess if you're going to make me write tests, I will. 00:11:14.400 --> 00:11:14.560 Right. 00:11:14.560 --> 00:11:19.580 And so the continuous integration is like, everyone gets checked before we accept your 00:11:19.580 --> 00:11:20.040 stuff, right? 00:11:20.040 --> 00:11:20.460 Well, right. 00:11:20.460 --> 00:11:26.180 So hooking it up within a GitHub Action, you could do this before, but it's really easy with 00:11:26.180 --> 00:11:30.860 these, is to just say, if you'd like, for instance, a merge request doesn't even go, 00:11:30.860 --> 00:11:33.860 doesn't even go very far if the test fails. 00:11:33.860 --> 00:11:35.320 So yeah. 00:11:35.320 --> 00:11:40.340 Listing coverage.py, because I'm, if you care about coverage, that's how you do that in Python. 00:11:40.680 --> 00:11:45.920 I don't do a lot of web testing, but I wanted to point out that Selenium and a wrapper around 00:11:45.920 --> 00:11:49.140 Selenium called Splinter are very commonly used. 00:11:49.140 --> 00:11:54.260 And then these are all packaged very well within a plugin called pytest Splinter so that you 00:11:54.260 --> 00:11:56.940 can easily control web applications. 00:11:56.940 --> 00:11:57.540 Nice. 00:11:57.540 --> 00:11:58.300 I hadn't heard of that one. 00:11:58.300 --> 00:11:59.760 I've heard of Selenium, but not Splinter. 00:11:59.760 --> 00:12:00.180 Yeah. 00:12:00.180 --> 00:12:06.080 A property-based testing is gaining a lot more traction in conjunction with functionality testing. 00:12:06.080 --> 00:12:10.660 And the solution within Python is a hypothesis for that. 00:12:10.660 --> 00:12:10.980 Nice. 00:12:10.980 --> 00:12:15.040 So everything you've listed here is pytest Plus, not instead of, right? 00:12:15.040 --> 00:12:15.660 Well, sure. 00:12:15.660 --> 00:12:17.360 It's pytest Plus these other things. 00:12:17.360 --> 00:12:20.400 However, they, I mean, none of them are pytest only. 00:12:20.400 --> 00:12:23.620 So you can use unit test with all of these things as well. 00:12:23.620 --> 00:12:23.880 Yeah. 00:12:23.880 --> 00:12:25.880 Or yeah, testing framework plus, right? 00:12:25.880 --> 00:12:27.120 They're not necessarily alternatives. 00:12:27.120 --> 00:12:27.600 Yeah. 00:12:27.600 --> 00:12:28.060 Right. 00:12:28.060 --> 00:12:28.480 All right. 00:12:28.480 --> 00:12:28.680 Cool. 00:12:28.680 --> 00:12:31.100 So those are definitely neat things to look into. 00:12:31.100 --> 00:12:32.380 They're separate projects. 00:12:33.080 --> 00:12:37.680 This portion of Talk Python To Me is brought to you by Linode. 00:12:37.680 --> 00:12:41.800 Whether you're working on a personal project or managing your enterprise's infrastructure, 00:12:41.800 --> 00:12:46.620 Linode has the pricing, support, and scale that you need to take your project to the next level. 00:12:46.620 --> 00:12:51.420 With 11 data centers worldwide, including their newest data center in Sydney, Australia, 00:12:51.420 --> 00:12:57.100 enterprise-grade hardware, S3-compatible storage, and the next-generation network, 00:12:57.100 --> 00:13:01.260 Linode delivers the performance that you expect at a price that you don't. 00:13:01.400 --> 00:13:06.700 Get started on Linode today with a $20 credit, and you get access to native SSD storage, 00:13:06.700 --> 00:13:11.240 a 40-gigabit network, industry-leading processors, their revamped cloud manager, 00:13:11.240 --> 00:13:17.340 cloud.linode.com, root access to your server, along with their newest API, and a Python CLI. 00:13:17.340 --> 00:13:22.580 Just visit talkpython.fm/Linode when creating a new Linode account, 00:13:22.580 --> 00:13:25.360 and you'll automatically get $20 credit for your next project. 00:13:25.360 --> 00:13:26.640 Oh, and one last thing. 00:13:26.640 --> 00:13:27.420 They're hiring. 00:13:27.640 --> 00:13:30.540 Go to linode.com slash careers to find out more. 00:13:30.540 --> 00:13:31.840 Let them know that we sent you. 00:13:34.720 --> 00:13:39.900 Now, like I said, just for people who maybe haven't written a test in pytest, 00:13:39.900 --> 00:13:41.720 like, what's the really quick flow? 00:13:41.720 --> 00:13:42.740 Like, why is it so simple? 00:13:42.740 --> 00:13:47.000 pytest has a really great discovery mechanism, but a function that just starts with, 00:13:47.000 --> 00:13:50.380 like, if you have a test file that starts with test underscore, 00:13:50.380 --> 00:13:52.320 there's some naming conventions. 00:13:52.320 --> 00:13:56.080 So the easiest thing is to have a test file that has test underscore in it, 00:13:56.440 --> 00:13:59.700 and then start your functions, test functions with test underscore, 00:13:59.700 --> 00:14:01.940 and then it'll all just get run. 00:14:01.940 --> 00:14:07.280 So pytest will find all of the files that start with test underscore and run all of the functions 00:14:07.280 --> 00:14:11.160 within them that start with test underscore, and then that's it, really. 00:14:11.160 --> 00:14:13.240 And so how does a test pass or fail? 00:14:13.660 --> 00:14:18.160 It fails if an exception is raised, and the easiest way to raise an exception is to, 00:14:18.160 --> 00:14:21.180 if you're intentionally checking something, is to use assert. 00:14:21.180 --> 00:14:27.380 So you, like, assert A equals B or something, and if the assertion fails, your test fails. 00:14:27.380 --> 00:14:28.140 It's that easy. 00:14:28.140 --> 00:14:33.680 Yeah, so you don't have to have decorators or derive from certain classes or anything like that. 00:14:33.680 --> 00:14:36.780 It's just the name involves test underscore, and you assert stuff. 00:14:36.780 --> 00:14:37.700 That's pretty good. 00:14:37.700 --> 00:14:40.620 Yeah, you don't even have to import pytest into your test files. 00:14:40.620 --> 00:14:42.920 They can be just functions. 00:14:42.920 --> 00:14:44.160 Yeah, that's right. 00:14:44.160 --> 00:14:47.760 So unless you're using some of the unique features that, like, extend pytest, 00:14:47.760 --> 00:14:51.000 like you talked about the way a test fails if there's an exception. 00:14:51.000 --> 00:14:53.340 Well, if you're testing for error handling, 00:14:53.340 --> 00:14:56.680 sometimes you want to trigger an exception and verify it has happened. 00:14:56.680 --> 00:14:59.840 So in that case, you'd have to include some pytest stuff to do, like, 00:14:59.840 --> 00:15:01.880 the width raises sort of stuff on it. 00:15:01.880 --> 00:15:06.260 But in general, for, like, the straight-ahead test, 00:15:06.260 --> 00:15:07.600 you don't even need to import pytest. 00:15:07.640 --> 00:15:08.380 Which is kind of weird. 00:15:08.380 --> 00:15:09.640 That freaked me out a little bit at first. 00:15:09.640 --> 00:15:11.340 I'm like, why am I not interacting with pytest? 00:15:11.340 --> 00:15:13.600 This is so weird at a code level. 00:15:13.600 --> 00:15:14.640 No, it's beautiful. 00:15:14.640 --> 00:15:17.520 It's one of the things I like is that it's just easy to start. 00:15:17.520 --> 00:15:17.920 Absolutely. 00:15:17.920 --> 00:15:22.480 Now, one other thing I want to cover before we dig into the list of plugins 00:15:22.480 --> 00:15:29.320 that we're going to cover is there's some non-obvious other ways to extend pytest, right? 00:15:29.320 --> 00:15:35.240 Like, it has this concept of fixtures that allow you to create and initialize stuff, 00:15:35.280 --> 00:15:39.780 and then automatically pytest has a way to, like, pass it along to your tests and whatnot. 00:15:39.780 --> 00:15:46.660 You'll see some of these plugins actually use fixtures, like some stuff about modifying time and whatnot that we'll talk about. 00:15:46.660 --> 00:15:48.160 What's the deal with those? 00:15:48.160 --> 00:15:58.140 Well, so the unit test model, and the unit test is similar to a lot of the X unit type, like J unit and PHP unit and stuff, 00:15:58.140 --> 00:16:02.800 have a similar model where you just, things are just named also and the tests get run. 00:16:03.020 --> 00:16:07.620 But there's often times where you need stuff to happen before your test gets run, 00:16:07.620 --> 00:16:11.680 like setting up a resource or generating data or something like that. 00:16:11.680 --> 00:16:17.260 And there's, so there's setup and teardown functions that you can, that are associated with these other test frameworks. 00:16:17.260 --> 00:16:21.500 pytest actually also supports those sorts of setup and teardown functions. 00:16:21.500 --> 00:16:23.640 If you want to use them, they are supported. 00:16:23.640 --> 00:16:27.460 But it's really not recommended because fixtures are so awesome. 00:16:27.460 --> 00:16:35.580 So a fixture is a way to combine, like, to write a function that is handling a resource or handling some data 00:16:35.580 --> 00:16:42.020 and be able to do the setup and teardown within one function and, if necessary. 00:16:42.020 --> 00:16:45.180 And then also just be able to scope those. 00:16:45.180 --> 00:16:53.440 And you can, you can have them have the resource or data shared between one test or many tests or even across the entire project. 00:16:53.440 --> 00:16:59.660 And you can control, you know, like if you're initializing a database or connecting to a web resource, 00:16:59.660 --> 00:17:01.340 you might not want to do that for every test. 00:17:01.440 --> 00:17:07.160 You can do it for one, once per session and then, and then do things like rollbacks and stuff per test. 00:17:07.160 --> 00:17:14.540 And that granularity of being able to shift your, the granularity of your setup and teardown is super powerful 00:17:14.540 --> 00:17:16.880 and really helps makes pytest shine. 00:17:16.880 --> 00:17:17.120 Yeah. 00:17:17.120 --> 00:17:19.320 And the way that you write it is super cool as well, right? 00:17:19.320 --> 00:17:21.660 Just have a test function that just takes a parameter. 00:17:21.660 --> 00:17:27.840 The parameter is named the same as the fixture and pytest just says, okay, I really call this function and pass it over. 00:17:27.840 --> 00:17:29.420 It could do really cool things. 00:17:29.440 --> 00:17:34.860 Like you could pass off your database data access layer and it could start a transaction before, hand it to you, 00:17:34.860 --> 00:17:38.160 and then roll back the transaction after the test is over. 00:17:38.160 --> 00:17:38.900 Right. 00:17:38.900 --> 00:17:40.160 And you don't have to worry about it. 00:17:40.160 --> 00:17:41.260 It could just take care of that for you. 00:17:41.260 --> 00:17:41.620 Yeah. 00:17:41.620 --> 00:17:42.220 Yep. 00:17:42.220 --> 00:17:44.740 And the fixtures are really what brought me to pytest. 00:17:44.740 --> 00:17:48.460 Some of the, there's some massively cool things you can do with that. 00:17:48.460 --> 00:17:55.100 Like one of the things that people don't think about is if you've got a system under test that has error logs somewhere, 00:17:55.100 --> 00:18:02.360 one of the things you can do within a fixture is check your error logs after the test runs to make sure there's no fatal errors that got saved somewhere. 00:18:02.360 --> 00:18:03.060 Yeah. 00:18:03.060 --> 00:18:03.460 Yeah. 00:18:03.460 --> 00:18:03.840 Very cool. 00:18:03.840 --> 00:18:04.800 Love it. 00:18:04.800 --> 00:18:05.080 Okay. 00:18:05.080 --> 00:18:05.940 So that's one way. 00:18:05.940 --> 00:18:08.480 The other one is this plugins in general, right? 00:18:08.540 --> 00:18:13.200 Like there's ways to extend pytest with plugins as a concept. 00:18:13.200 --> 00:18:16.600 The pytest has a whole set of hook functions as well. 00:18:16.600 --> 00:18:18.260 There's, there's a lot of things you can do. 00:18:18.260 --> 00:18:22.380 You can, you can write fixtures and package those fixtures for other people to use. 00:18:22.500 --> 00:18:25.200 So some plugins are just packaged fixtures. 00:18:25.200 --> 00:18:27.520 Other things you can do with hook functions. 00:18:27.520 --> 00:18:40.020 So with, I actually was intimidated when I first was reading about hook functions, but they're just, there's just a way for pytest to allow different parts of the way pytest runs to be hooked into. 00:18:40.020 --> 00:18:45.300 So you can, you know, alter a few things if you need to, like, like some of the test ordering ones. 00:18:45.300 --> 00:18:52.900 Once all the tests are collected, you can hook into that section and reorder them if you need to, or, or filter them. 00:18:52.900 --> 00:18:54.360 Choose which ones execute, right? 00:18:54.360 --> 00:18:54.960 Yeah. 00:18:54.960 --> 00:19:05.220 There's a whole bunch of ways you can hook into it and it's fairly complex, but a lot of people have come up with some really clever things and plugins allow you to allow you to do that. 00:19:05.220 --> 00:19:19.300 And I mean, not just third party plugins that you can pull off of PyPI, but you can also just, if you want to share some code that you have that works with your tests amongst different projects, you can package your own plugins to do that. 00:19:19.300 --> 00:19:19.620 Yeah. 00:19:19.620 --> 00:19:20.280 Oh, that's cool. 00:19:20.280 --> 00:19:23.140 You could even have them for like your company, right? 00:19:23.140 --> 00:19:26.000 Like we always do this in RCI and whatnot. 00:19:26.000 --> 00:19:28.640 You can just grab a plugin and do stuff. 00:19:28.640 --> 00:19:28.920 Yeah. 00:19:28.920 --> 00:19:34.420 Like I've got plugins that I've written that, that just are particular to one particular test piece of test equipment. 00:19:34.800 --> 00:19:39.840 So to control this piece of equipment, you can use this plugin and it helps you with some of the common things. 00:19:39.840 --> 00:19:40.460 Oh, that's cool. 00:19:40.460 --> 00:19:41.800 Makes it a little easier for new people. 00:19:41.800 --> 00:19:42.140 Yeah. 00:19:42.140 --> 00:19:42.680 Yeah. 00:19:42.680 --> 00:19:43.220 Lots easier. 00:19:43.220 --> 00:19:43.680 Nice. 00:19:43.680 --> 00:19:44.680 All right. 00:19:44.680 --> 00:19:47.060 I think it's time we go down the list. 00:19:47.060 --> 00:19:48.120 We've set the stage. 00:19:48.120 --> 00:19:49.420 We know what fixtures are. 00:19:49.420 --> 00:19:52.080 We know what plugins are, at least in general. 00:19:52.080 --> 00:19:55.040 Let's put some sugar on pytest to make it sweet. 00:19:55.040 --> 00:19:56.600 Yeah. 00:19:56.600 --> 00:20:00.280 So we're going to run through a whole bunch of really cool pytest plugins, right? 00:20:00.280 --> 00:20:01.000 Yeah, absolutely. 00:20:01.000 --> 00:20:02.180 All right. 00:20:02.180 --> 00:20:03.040 What's this first one here? 00:20:03.340 --> 00:20:04.900 So sugar is a fun one, actually. 00:20:04.900 --> 00:20:07.560 I ran into this right away with looking at pytest. 00:20:07.560 --> 00:20:16.160 So the default output of pytest is you, when you run it, you get a dot shows up for every, for every test that gets run. 00:20:16.160 --> 00:20:19.740 And it does list the files, but a little dot shows up. 00:20:19.740 --> 00:20:22.000 And maybe that's not enough for you. 00:20:22.000 --> 00:20:26.280 So pytest sugar shows you, like, green check marks instead. 00:20:26.280 --> 00:20:33.780 So, and then, you know, instead of, like, Fs for failure, I think there's, like, red Xs or something that show up. 00:20:33.780 --> 00:20:38.200 So it alters the output of pytest so that it looks a little nicer. 00:20:38.400 --> 00:20:46.500 Yeah, it also looks like it might be suppressing the module name, the folder names, and only, like, highlighting the actual file names. 00:20:46.500 --> 00:20:49.320 So it does, like, gray out a little part of the UI. 00:20:49.320 --> 00:20:51.840 So, like, the actual test file being run stands out. 00:20:51.840 --> 00:20:56.540 And there's a vertical progress bar going along that runs as well. 00:20:56.540 --> 00:20:57.100 That's pretty sweet. 00:20:57.180 --> 00:20:58.080 Yeah, that's pretty cool. 00:20:58.080 --> 00:21:02.620 And that was especially really helpful before pytest added the percentage done. 00:21:02.620 --> 00:21:07.040 So now, for each file, pytest lists the percent done now by default. 00:21:07.040 --> 00:21:10.500 But it didn't always have that, and pytest sugar had that. 00:21:10.500 --> 00:21:11.140 It's pretty cool. 00:21:11.140 --> 00:21:11.740 Yeah, it's cool. 00:21:11.740 --> 00:21:20.360 And there's a nice little animated GIF, as all things should have, on there that I'll link to in the show notes so people can check it out and see it going. 00:21:20.360 --> 00:21:21.380 So, yeah, pytest sugar. 00:21:21.380 --> 00:21:26.480 And this is cool because all you've got to do is install this, and it doesn't require effort, right? 00:21:26.520 --> 00:21:28.500 And, like, oh, and now I can do this thing in code. 00:21:28.500 --> 00:21:31.100 It's just, like, I install this, and it looks better. 00:21:31.100 --> 00:21:31.440 Yeah. 00:21:31.440 --> 00:21:32.880 How do you install plugins? 00:21:32.880 --> 00:21:34.160 Just do you pip install them? 00:21:34.160 --> 00:21:37.760 Like, do I need to register it with pytest, or is it sufficient to just pip install it? 00:21:37.760 --> 00:21:38.860 Just pip install it. 00:21:38.860 --> 00:21:41.000 It's about the design of the plugin as well. 00:21:41.000 --> 00:21:45.580 So some plugins, like Sugar, you just install it, and it starts working. 00:21:45.580 --> 00:21:46.520 You don't have to do anything. 00:21:46.520 --> 00:21:47.000 All right. 00:21:47.000 --> 00:21:50.100 But some of them intentionally don't start right away. 00:21:50.100 --> 00:21:55.940 So some plugins, they are installed, but you have to turn on the functionality with a flag or something like that. 00:21:56.000 --> 00:21:56.240 Got you. 00:21:56.240 --> 00:21:56.760 Okay. 00:21:56.760 --> 00:21:57.680 Cool. 00:21:57.680 --> 00:21:58.000 All right. 00:21:58.000 --> 00:21:58.780 What's the next one? 00:21:58.780 --> 00:21:59.360 Number two. 00:21:59.360 --> 00:22:07.980 So this one's definitely something, pytest cov, dash cov, it's a way to run coverage with pytest together. 00:22:08.220 --> 00:22:13.280 So, I mean, there's really two ways to run coverage on pytest tests. 00:22:13.280 --> 00:22:15.620 Coverage doesn't require pytest, right? 00:22:15.620 --> 00:22:18.080 So you can use any test framework. 00:22:18.080 --> 00:22:22.680 So you can run pytest from coverage, or you can run coverage from pytest. 00:22:22.680 --> 00:22:24.460 It's kind of a choice. 00:22:24.940 --> 00:22:29.920 But I usually use pytest cov because it's just really well supported. 00:22:29.920 --> 00:22:31.420 It does some really cool things. 00:22:31.420 --> 00:22:36.420 One of the things that it does really nice is, we'll talk about xDisk later, but xDisk allows 00:22:36.420 --> 00:22:37.240 parallel runs. 00:22:37.240 --> 00:22:44.280 And pytest cov cleanly collects all of the different parallel runs and pulls the report, coverage 00:22:44.280 --> 00:22:45.420 report together for you. 00:22:45.420 --> 00:22:46.000 Oh, that's cool. 00:22:46.000 --> 00:22:46.860 Which is really nice. 00:22:47.060 --> 00:22:50.300 Yeah, it has xDisk support, which we'll talk about what that means. 00:22:50.300 --> 00:22:56.580 But that's a pretty high-end piece of kit there to go grab a parallel executing stuff and bring 00:22:56.580 --> 00:22:57.020 it all together. 00:22:57.020 --> 00:23:01.440 Yeah, and there's also, it's really good about catching the coverage from right from the beginning. 00:23:01.440 --> 00:23:06.560 So you've got like some startup code that happens, like your fixtures and stuff that you've set 00:23:06.560 --> 00:23:13.220 up that run before your tests, making sure that those have coverage turned on as soon as 00:23:13.220 --> 00:23:17.040 possible so that you don't have startup code that doesn't get run and coverage. 00:23:17.040 --> 00:23:20.880 That's one of the issues with hooking up coverage often. 00:23:20.880 --> 00:23:23.880 But pytest cov handles that really cleanly. 00:23:23.880 --> 00:23:24.700 Yeah, awesome. 00:23:24.700 --> 00:23:29.780 It also has sub-process support, so you can fork things into a sub-process and it'll get 00:23:29.780 --> 00:23:33.800 covered automatically, which you might think, okay, who's going to do that? 00:23:33.800 --> 00:23:37.140 And like, do they really care about tracking the coverage of it? 00:23:37.140 --> 00:23:39.140 But, you know, multiprocessing, right? 00:23:39.140 --> 00:23:40.320 They're doing parallelism. 00:23:40.320 --> 00:23:41.380 Yeah, definitely. 00:23:41.380 --> 00:23:42.960 Yeah, that's good as well. 00:23:42.960 --> 00:23:44.160 Very nice. 00:23:44.160 --> 00:23:49.800 The other nice thing about it is just, so if you're, it makes it easy to have a text output. 00:23:49.800 --> 00:23:55.700 So normally when you run coverage outside of, like if you run pytest from coverage, once 00:23:55.700 --> 00:23:59.780 you're finished, you have to then tell coverage to give you the reports. 00:23:59.780 --> 00:24:02.820 You have to run coverage report to get your coverage report. 00:24:02.820 --> 00:24:05.480 But pytest cov just prints that at the end. 00:24:05.480 --> 00:24:06.940 So you get that automatically. 00:24:06.940 --> 00:24:08.080 Don't have to run it again. 00:24:08.080 --> 00:24:08.320 Cool. 00:24:08.320 --> 00:24:08.980 Very nice. 00:24:08.980 --> 00:24:09.320 Very nice. 00:24:09.320 --> 00:24:10.920 Would you say it's low stress? 00:24:11.200 --> 00:24:12.780 It is low stress, yes. 00:24:12.780 --> 00:24:16.740 Unless the stress is trying to get 100% coverage. 00:24:16.740 --> 00:24:17.960 Well, that's true. 00:24:17.960 --> 00:24:22.940 Actually, what are your thoughts on, I'll share my thoughts with you as well, but, you know, 00:24:22.940 --> 00:24:27.160 a lot of people feel like they're not testing well if they don't have 100% coverage or they 00:24:27.160 --> 00:24:27.960 don't have any coverage. 00:24:27.960 --> 00:24:31.880 So they don't even know like how much of their code is being run by tests. 00:24:31.880 --> 00:24:32.700 Like what are your thoughts there? 00:24:32.700 --> 00:24:39.280 I am a strong proponent for open source projects to have 100% coverage because it eases new 00:24:39.280 --> 00:24:41.220 people, new developers into the project. 00:24:41.220 --> 00:24:45.540 When they add code, it's easy for them to find the code that they added that isn't being covered 00:24:45.540 --> 00:24:45.800 yet. 00:24:45.800 --> 00:24:48.380 It doesn't necessarily mean that your tests are good. 00:24:48.380 --> 00:24:50.740 It just means that their code's covered. 00:24:51.020 --> 00:24:52.780 For open source stuff, it's good. 00:24:52.780 --> 00:25:00.000 And then if you take that argument, then it's good for internal projects as well because it's 00:25:00.000 --> 00:25:01.300 easier to run stuff. 00:25:01.300 --> 00:25:03.120 However, I'm not a zealot about it. 00:25:03.120 --> 00:25:08.400 I mean, if you're like jumping into a legacy project and adding software tests to a project 00:25:08.400 --> 00:25:13.520 and you're testing the new stuff and the things that you change, you're never going to hit 00:25:13.520 --> 00:25:16.980 100% coverage of like the rest of the legacy code. 00:25:16.980 --> 00:25:17.900 And, you know. 00:25:17.900 --> 00:25:19.820 Yeah, I agree with that. 00:25:19.820 --> 00:25:26.060 I feel like a lot of people get hung up on like the last mile for applications, like for 00:25:26.060 --> 00:25:28.840 open source projects where it's like a library or something. 00:25:28.840 --> 00:25:29.280 Sure. 00:25:29.280 --> 00:25:30.900 It should totally all be there. 00:25:30.900 --> 00:25:34.240 But there's like in certain apps, there's just little edge cases. 00:25:34.240 --> 00:25:38.660 Like if I'm running on Windows, I got to do this other thing, but I'm only testing this 00:25:38.660 --> 00:25:40.600 on macOS or on Linux. 00:25:40.600 --> 00:25:41.060 Right. 00:25:41.060 --> 00:25:46.280 So like how much should I stress about trying to like get that little edge piece of code to 00:25:46.280 --> 00:25:47.460 run or like there's this. 00:25:47.900 --> 00:25:51.880 This one utility that only I run and a cron job to do this thing. 00:25:51.880 --> 00:25:52.140 I don't know. 00:25:52.140 --> 00:25:56.200 There's just all these little edge cases that can be like a whole lot of work to get to. 00:25:56.200 --> 00:26:01.040 But I feel like, you know, most of the time, you know, you should probably not stress too 00:26:01.040 --> 00:26:01.520 much about it. 00:26:01.520 --> 00:26:01.700 Yeah. 00:26:01.700 --> 00:26:04.780 So all those edge cases are the reasons why I think you should test those. 00:26:04.780 --> 00:26:06.680 Because. 00:26:06.920 --> 00:26:07.320 Okay. 00:26:07.320 --> 00:26:09.700 So we have ways to combine coverage reports. 00:26:09.700 --> 00:26:16.900 So you can use something like GitHub Actions or Travis or something to run your, run your 00:26:16.900 --> 00:26:21.840 code on multiple platforms and combine the coverage from the test runs from multiple platforms. 00:26:21.840 --> 00:26:24.880 So why not run them on multiple platforms? 00:26:24.880 --> 00:26:30.300 If that, if you have platform specific code, if you have a platform specific code, I think 00:26:30.300 --> 00:26:33.040 there's a reason is because some of your customers use it. 00:26:33.040 --> 00:26:38.100 And if it's different than what the developers use, I think you should test that code. 00:26:38.100 --> 00:26:39.040 Yeah, that's probably true. 00:26:39.040 --> 00:26:39.720 That's a good point. 00:26:39.720 --> 00:26:43.260 That's the only place it gets run before it gets deployed. 00:26:43.260 --> 00:26:44.480 All right. 00:26:44.480 --> 00:26:48.340 Well, my transition to the stress one, we got kind of sidetracked. 00:26:48.340 --> 00:26:50.400 But this next one is all about stress. 00:26:50.400 --> 00:26:50.800 Yeah. 00:26:50.800 --> 00:26:53.120 So I think we should actually just cover a couple of them. 00:26:53.120 --> 00:26:54.000 So the next two. 00:26:54.000 --> 00:26:54.400 Yeah. 00:26:54.400 --> 00:26:55.540 They're kind of a good pair. 00:26:55.540 --> 00:26:56.020 Yeah. 00:26:56.180 --> 00:26:56.460 Yeah. 00:26:56.460 --> 00:26:57.560 And pytest repeat. 00:26:57.560 --> 00:27:00.220 So pytest stress is a nice thing. 00:27:00.220 --> 00:27:03.640 So let's say you've got some code that you just run it like overnight. 00:27:03.640 --> 00:27:06.220 I know that I'm going to leave it like a mid. 00:27:06.220 --> 00:27:09.180 I want to, I know everybody's going to be out of the building by midnight. 00:27:09.180 --> 00:27:16.160 And then I want to just beat on the code and run it as many times as I can for eight hours. 00:27:16.600 --> 00:27:24.760 So pytest stress allows you to be able to run a set of time to say, hey, run this for eight hours or run it for a half an hour, run it for a certain time period. 00:27:24.760 --> 00:27:27.580 And that's really cool if that's where you're thinking. 00:27:27.580 --> 00:27:28.900 And then repeat. 00:27:28.900 --> 00:27:34.520 pytest repeat allows you to run your test suite or test functions or whatever a certain number of times. 00:27:34.520 --> 00:27:37.440 So I want to set it up for a thousand times. 00:27:37.440 --> 00:27:41.780 I bring these up together because I found pytest repeat first when I needed to. 00:27:41.960 --> 00:27:47.980 I had some tests that, a test that was failing once in a while and I just wanted to run it all night long. 00:27:47.980 --> 00:27:51.140 And so I just set it for a large number like 2000. 00:27:51.140 --> 00:27:51.600 I don't know. 00:27:51.600 --> 00:27:53.000 Like I just guessed. 00:27:53.000 --> 00:27:56.160 But really what I wanted it to do was run all night. 00:27:56.160 --> 00:27:59.220 And so pytest stress would have been better for that case. 00:27:59.220 --> 00:28:00.700 Oh yeah, that's a cool one. 00:28:00.700 --> 00:28:03.200 You know where I would see these as being useful? 00:28:03.200 --> 00:28:09.200 Sometimes I'm trying to do profiling, like CPU profiling to figure out or maybe even memory profiling. 00:28:09.620 --> 00:28:19.740 And the startup of getting the pytest or whatever it is up and running and all the stuff in place and then running a little bit of code and then tearing it all down. 00:28:19.740 --> 00:28:20.980 It's like gets dwarfed. 00:28:20.980 --> 00:28:25.040 You know, the actual bit you're trying to understand gets dwarfed by all that startup. 00:28:25.040 --> 00:28:30.020 But here you could just let it run for 10 minutes and then it'll be really focused on what it was slow at. 00:28:30.020 --> 00:28:34.420 Then that's where actually with profiling, I think that would be a good place to use the repeat. 00:28:34.420 --> 00:28:40.640 So you could say do a nice round number like 100 times or a thousand times because it's easy to divide by that. 00:28:40.640 --> 00:28:42.240 Yeah, yeah, that's a good point. 00:28:42.240 --> 00:28:42.580 Exactly. 00:28:42.580 --> 00:28:43.500 Nice. 00:28:43.500 --> 00:28:44.880 What's this next one? 00:28:44.880 --> 00:28:45.580 InstaFail. 00:28:45.580 --> 00:28:51.980 So if you had a whole bunch of like print statements and whatever in your code, it doesn't get shown normally with pytest. 00:28:51.980 --> 00:28:55.280 So pytest will capture the output by default. 00:28:55.280 --> 00:28:56.280 I mean, you can turn it off. 00:28:56.280 --> 00:29:00.840 But by default, it captures the output and any errors and failures and stuff. 00:29:00.840 --> 00:29:03.920 Those get captured also, like tracebacks and stuff. 00:29:03.920 --> 00:29:15.220 And then afterwards, so after the test is done, you get your report and then you get a lot of this output that you can help debug your test failures with. 00:29:15.580 --> 00:29:24.420 With long running tests, sometimes it'd be really nice if you really could see those errors while it's happening. 00:29:24.420 --> 00:29:26.920 So you can start looking at the code and start debugging right away. 00:29:26.920 --> 00:29:34.520 And that's where InstaFail will report your failures and errors while the test run is happening, even with output being captured. 00:29:34.520 --> 00:29:35.120 Nice. 00:29:35.120 --> 00:29:44.980 Is there a way with plugins where if I've installed InstaFail, but I only want to see it some of the time to like encode, say, you know, don't run InstaFail right now. 00:29:44.980 --> 00:29:48.860 But sometimes when I want it, let it run without pip uninstalling it. 00:29:48.860 --> 00:29:50.360 I'm not sure, actually. 00:29:50.360 --> 00:29:51.140 That would be sweet. 00:29:51.140 --> 00:29:52.120 Yeah. 00:29:52.120 --> 00:29:53.040 I don't know. 00:29:53.040 --> 00:29:57.760 It could be up to them individually, but it would be nice if maybe a command line or something. 00:29:57.760 --> 00:29:58.100 I don't know. 00:29:58.100 --> 00:30:01.520 Some way to say, actually, I want this other one turned on now, but not always. 00:30:01.520 --> 00:30:02.620 Well, I'm just looking at it. 00:30:02.620 --> 00:30:05.180 So it looks like even if it has it installed, it doesn't. 00:30:05.180 --> 00:30:07.400 This is one of those that it doesn't run automatically. 00:30:07.400 --> 00:30:09.000 You have to pass in the flag. 00:30:09.000 --> 00:30:09.380 I see. 00:30:09.380 --> 00:30:09.680 Okay. 00:30:09.680 --> 00:30:10.140 Cool, cool. 00:30:10.140 --> 00:30:10.780 To turn it on. 00:30:10.780 --> 00:30:11.200 Nice. 00:30:11.200 --> 00:30:15.500 Speaking of passing flags and whatnot, pytest metadata is the next one. 00:30:15.500 --> 00:30:15.960 Number six. 00:30:15.960 --> 00:30:16.300 Yeah. 00:30:16.300 --> 00:30:17.760 This is a kind of an obscure one. 00:30:17.760 --> 00:30:22.900 And this is often used with pytest HTML, which actually we're not covering. 00:30:22.900 --> 00:30:23.660 Maybe we should. 00:30:23.660 --> 00:30:26.800 But pytest InstaFail, or not InstaFail. 00:30:26.800 --> 00:30:28.520 Metadata. 00:30:28.520 --> 00:30:29.400 Metadata. 00:30:29.400 --> 00:30:33.320 Metadata allows you to either through, it gives you a couple of things. 00:30:33.320 --> 00:30:39.700 It has a command line flag where you can pass in extra information to go into your test report. 00:30:39.700 --> 00:30:46.540 And then also you can within, so like for instance, if you're running against a particular set of servers, 00:30:46.540 --> 00:30:50.440 you could have the server name getting listed in the report. 00:30:50.740 --> 00:31:03.400 The other thing is maybe the data that you really want to log, the extra metadata about a test run, is generated or like in real time is developed or gotten from within a fixture or something. 00:31:03.400 --> 00:31:09.360 So a fixture or a test can set this extra metadata and it gets output with your test data. 00:31:09.700 --> 00:31:10.100 Oh, cool. 00:31:10.100 --> 00:31:17.860 Like maybe what GitHub branch you were running on or what environment like staging or production testing or whatever. 00:31:17.860 --> 00:31:18.520 Yeah. 00:31:18.520 --> 00:31:19.100 Yeah. 00:31:19.100 --> 00:31:19.520 Yeah, cool. 00:31:19.520 --> 00:31:22.260 Or maybe adding, you know, what operating system. 00:31:22.260 --> 00:31:24.620 I mean, your operating system's already being logged. 00:31:24.780 --> 00:31:30.820 But if there's other information like, yeah, what branch, what version, what some of the extra specifics. 00:31:30.820 --> 00:31:31.540 Yeah. 00:31:31.540 --> 00:31:31.740 Yeah. 00:31:31.740 --> 00:31:32.480 That's cool. 00:31:32.480 --> 00:31:33.220 Yeah. 00:31:33.220 --> 00:31:33.960 That sounds useful. 00:31:33.960 --> 00:31:40.420 Like if you're saving the report and you're going to look at it later, like maybe continuous integration or something like that. 00:31:40.500 --> 00:31:40.640 Yeah. 00:31:40.640 --> 00:31:49.380 So like for instance, when I'm testing against electronic test equipment, we add the instrument that we're testing against. 00:31:49.380 --> 00:31:51.480 So that's just part of the log. 00:31:51.480 --> 00:31:56.200 This portion of Talk Python To Me is brought to you by Sentry. 00:31:56.200 --> 00:31:58.860 How would you like to remove a little stress from your life? 00:31:58.860 --> 00:32:04.600 Do you worry that users may be having difficulties or are encountering errors with your app right now? 00:32:04.600 --> 00:32:07.480 Would you even know it until they send that support email? 00:32:07.780 --> 00:32:17.860 How much better would it be to have the error details immediately sent to you, including the call stack and values of local variables, as well as the active user stored in the report? 00:32:17.860 --> 00:32:21.660 With Sentry, this is not only possible, it's simple and free. 00:32:21.660 --> 00:32:25.020 In fact, we use Sentry on all the Talk Python web properties. 00:32:25.020 --> 00:32:31.600 We've actually fixed a bug triggered by our user and had the upgrade ready to roll out as we got the support email. 00:32:31.600 --> 00:32:33.400 That was a great email to write back. 00:32:33.400 --> 00:32:35.840 We saw your error and have already rolled out the fix. 00:32:35.840 --> 00:32:37.080 Imagine their surprise. 00:32:37.760 --> 00:32:39.720 Surprise and delight your users today. 00:32:39.720 --> 00:32:47.600 Create your free account at talkpython.fm/sentry and track up to 5,000 errors a month across multiple projects for free. 00:32:47.600 --> 00:32:58.880 And if you use the code talkpython, all one word, it's good for two free months of Sentry's team plan, which will give you up to 20 times as many monthly events and some other features. 00:32:58.880 --> 00:33:00.600 So create that free account today. 00:33:02.840 --> 00:33:04.300 Yeah, I don't want to just be random. 00:33:04.300 --> 00:33:05.420 Not just random. 00:33:05.420 --> 00:33:06.020 Yeah. 00:33:06.020 --> 00:33:07.080 But sometimes you do. 00:33:07.080 --> 00:33:12.740 So one of the things that pytest will actually, I think I've intentionally forgotten the test order. 00:33:12.740 --> 00:33:15.840 I think it's alphabetical or just it's not alphabetical. 00:33:15.840 --> 00:33:19.060 It's like the order that the tests are in in the file. 00:33:19.060 --> 00:33:22.120 That's how the default order that things get run in. 00:33:22.120 --> 00:33:26.320 But you really don't want your test to be order dependent. 00:33:26.560 --> 00:33:30.160 So you want to be able to run any test in any order. 00:33:30.160 --> 00:33:35.560 And sometimes a good way to test that is with the pytest Randomly plugin. 00:33:35.560 --> 00:33:37.040 So it does a couple of things. 00:33:37.040 --> 00:33:43.140 One of the things you can do is it can reorder your tests so that they're running in a random order. 00:33:43.400 --> 00:33:48.900 But also you might be using random as a seed for some randomized information. 00:33:48.900 --> 00:33:54.400 Like say, like Faker, for instance, and other fake data generators use random seeds. 00:33:54.400 --> 00:33:59.760 So pytest Randomly can help you randomize that stuff so that it's more random. 00:33:59.760 --> 00:34:00.500 Yeah, that's cool. 00:34:00.500 --> 00:34:07.320 It says that if Faker, which is a really cool library, if Faker is installed, it'll automatically update it. 00:34:07.320 --> 00:34:10.180 So that Faker is different this time than another time. 00:34:10.180 --> 00:34:11.600 Same thing for NumPy. 00:34:11.600 --> 00:34:13.040 Yeah, this is really cool. 00:34:13.180 --> 00:34:23.260 Anytime you have your test running reliably in the same order, you could unintentionally pick up some form of dependency on order. 00:34:23.260 --> 00:34:27.240 Like if you call this test, it like initializes the database connection string. 00:34:27.240 --> 00:34:29.440 And then you can call this other thing and it'll work. 00:34:29.440 --> 00:34:30.980 But if you call it in the other order, it would crash. 00:34:30.980 --> 00:34:34.320 I mean, maybe that's a bad example because you really want to avoid that dependency. 00:34:34.320 --> 00:34:35.020 But you know what I mean? 00:34:35.020 --> 00:34:39.860 Like it could set something where the next thing depends on it being set that way. 00:34:39.860 --> 00:34:42.740 And if you randomly order it, then you're going to discover those. 00:34:42.960 --> 00:34:52.160 Yeah, especially with your scoping your fixtures to try to speed up test time, you might inadvertently have dependencies like system dependencies. 00:34:52.160 --> 00:34:57.460 For instance, you get the system in a particular state in one test and then you're running another test. 00:34:57.680 --> 00:35:04.240 One of the problems often is you got a test that runs fine when you're running it from your editor. 00:35:04.480 --> 00:35:09.060 And then you go into on the CI system where it's running everything. 00:35:09.060 --> 00:35:10.100 It fails. 00:35:10.100 --> 00:35:14.640 And you can consistently fail it with the entire suite, but not by itself. 00:35:14.640 --> 00:35:18.160 Those are just a pain in the rear to find out where the problem really is. 00:35:18.160 --> 00:35:20.760 And I mean, randomly can help you. 00:35:20.760 --> 00:35:25.840 I'm not sure how well it can help you debug that problem, but it can help find dependency. 00:35:25.840 --> 00:35:29.160 Yeah, at least it won't leave it covered up, hidden. 00:35:29.160 --> 00:35:29.540 Yeah. 00:35:29.540 --> 00:35:29.940 Yeah. 00:35:29.940 --> 00:35:36.440 Well, you already mentioned exodist earlier when we talked about pytestCov. 00:35:36.440 --> 00:35:38.800 So what's this exodist? 00:35:38.800 --> 00:35:40.800 Well, exodist is super powerful. 00:35:40.800 --> 00:35:43.420 It's a little bit of a grab bag of a few features. 00:35:43.520 --> 00:35:49.380 One of the things that it does, and people usually grab it for this, is to run tests in parallel. 00:35:49.380 --> 00:35:54.860 So you can tell it how many CPUs to run on and it'll just run a whole bunch of tests in parallel. 00:35:54.860 --> 00:35:56.100 And that's pretty cool. 00:35:56.100 --> 00:35:57.260 It can speed up tests. 00:35:57.260 --> 00:36:03.160 It does have a little bit of startup time because it's communicating with the different, you know, 00:36:03.160 --> 00:36:06.300 it's doing multi-process communications, a little bit of startup time. 00:36:06.300 --> 00:36:10.840 So if you have independent tests that don't share a resource, this can be really cool. 00:36:11.240 --> 00:36:16.320 The other thing that you can also just offload it onto different processes or different computers, 00:36:16.320 --> 00:36:19.460 you can have it be running on different machines as well. 00:36:19.460 --> 00:36:20.320 So that's pretty cool. 00:36:20.320 --> 00:36:20.580 That's cool. 00:36:20.580 --> 00:36:21.560 That's really cool. 00:36:21.560 --> 00:36:22.720 Yeah, that's very fascinating. 00:36:22.720 --> 00:36:27.180 And then I don't know why this is bundled into it, but this is a really cool feature of pytest 00:36:27.180 --> 00:36:29.620 exodist is the loop on fail feature. 00:36:29.620 --> 00:36:35.760 So you can pass it loop on fail and it'll run your tests repeatedly in a sub process. 00:36:35.760 --> 00:36:38.620 And then you get to see what the failures are. 00:36:39.120 --> 00:36:46.060 But if it detects you've changed your project at all, it'll rerun the test, but it just reruns the failing tests. 00:36:46.060 --> 00:36:50.480 So it just keeps rerunning all of your failures while you're trying to fix it. 00:36:50.480 --> 00:36:57.540 And then once all of them are, when everything's running and fixed, it'll rerun everything. 00:36:57.740 --> 00:37:00.060 So when they all pass, it'll rerun the whole suite. 00:37:00.060 --> 00:37:00.960 Yeah, super cool. 00:37:00.960 --> 00:37:07.000 So it's a little bit like in PyCharm where you can say, run the test when anything changes, 00:37:07.000 --> 00:37:08.740 but this only does it for the failure. 00:37:08.740 --> 00:37:13.860 So you're like, you got to, you write a failing test or you have a failing test you're trying to fix and you can just work on it. 00:37:13.860 --> 00:37:15.700 And it only focuses on that thing. 00:37:15.700 --> 00:37:16.040 Yeah. 00:37:16.100 --> 00:37:22.120 I mean, it kind of is all the passing ones too, because if your suite is passing, it just keeps rerunning it every time it sees a change. 00:37:22.120 --> 00:37:22.420 I see. 00:37:22.420 --> 00:37:22.840 Okay. 00:37:22.840 --> 00:37:23.520 Yeah. 00:37:23.520 --> 00:37:24.040 Cool. 00:37:24.040 --> 00:37:24.760 All right. 00:37:24.760 --> 00:37:31.040 Well, one way your code can fail is it doesn't execute properly, makes a mistake. 00:37:31.040 --> 00:37:37.880 The other one is it could just not be formatted the way you like or break some of the idioms that you're supposed to write. 00:37:37.880 --> 00:37:38.080 Yeah. 00:37:38.080 --> 00:37:42.360 There's a whole bunch of extra plugins that are like this, but they're style guide stuff. 00:37:42.360 --> 00:37:51.000 So pytest Flakes is one where it can run Flake 8 and other things against your code as part of the pytest run. 00:37:51.000 --> 00:37:54.960 It can test both your source code and your test code if you wanted to. 00:37:54.960 --> 00:38:00.980 I actually think this is a pretty cool idea, but there is some controversy as to whether this should be really part of, 00:38:00.980 --> 00:38:06.340 your pytest run or it should be part of like, say, do a pre-commit or something. 00:38:06.340 --> 00:38:09.060 But I think having it in both places is fine. 00:38:09.060 --> 00:38:17.660 If your workflow is mostly people run pytest to make sure all their code's working and then they're ready to check it in. 00:38:17.660 --> 00:38:26.400 If you want to make sure you don't, it's just your workflow, whether you like to catch your Flake errors during commit time or during test time. 00:38:26.400 --> 00:38:26.640 Yeah. 00:38:26.720 --> 00:38:32.920 And I just noticed the one that I grabbed to throw into the list, there was pytest Flakes, I think. 00:38:32.920 --> 00:38:35.760 And they say you should use pytest dash Flake 8. 00:38:35.760 --> 00:38:37.040 So I'm going to link to that one. 00:38:37.040 --> 00:38:38.520 But same idea. 00:38:38.520 --> 00:38:38.920 Yeah. 00:38:38.920 --> 00:38:39.740 Yeah. 00:38:39.740 --> 00:38:43.400 There's also a pytest PEP 8, I think it is. 00:38:43.400 --> 00:38:46.000 There's like various different ones for this. 00:38:46.000 --> 00:38:46.320 Yeah. 00:38:46.320 --> 00:38:48.800 And I got to admit, I'm like a little, I guess, over the top. 00:38:48.800 --> 00:38:53.160 I use both pytest Flake 8 and hookup Flake 8 to pre-commit hook. 00:38:53.160 --> 00:38:56.680 So just because, I mean, pre-commit is so fast. 00:38:56.680 --> 00:38:57.160 Why not? 00:38:57.160 --> 00:38:58.220 Yeah, absolutely. 00:38:58.220 --> 00:38:59.040 Cool. 00:38:59.040 --> 00:38:59.340 Cool. 00:38:59.340 --> 00:39:04.640 Speaking of being fast, you know, your tests sometimes are slow because it's a lot of work to run your tests. 00:39:04.640 --> 00:39:06.600 Or they could just be stuck waiting on something. 00:39:08.260 --> 00:39:15.060 Yeah, I really like, so pytest Timeout is a plugin that is a fairly big hammer to a problem. 00:39:15.060 --> 00:39:19.060 But you've got tests that maybe it loops forever. 00:39:19.060 --> 00:39:21.120 You forgot to put a timer on something. 00:39:21.120 --> 00:39:22.360 Or it's stuck on a resource. 00:39:22.360 --> 00:39:25.180 Or for any reason, it's hung. 00:39:25.180 --> 00:39:30.280 And I suggest people use pytest Timeout for any long-running suites all the time. 00:39:30.280 --> 00:39:35.020 For really short, fast tests, there is a little bit of overhead. 00:39:35.440 --> 00:39:40.400 But for any sort of long-running stuff, the overhead is negligible and you don't see it. 00:39:40.400 --> 00:39:46.720 But what it does is it runs your tests in another thread and then is able to kill it. 00:39:46.720 --> 00:39:52.100 So it will just put a timer around it and kill the thread if it doesn't come back in time. 00:39:52.100 --> 00:39:57.380 Why I call it a big hammer is because if it fails, you don't get your test output. 00:39:57.380 --> 00:39:59.440 But it also doesn't hang forever. 00:39:59.440 --> 00:40:02.940 And these are especially good for CI systems. 00:40:03.140 --> 00:40:08.440 Because you really don't want to come back in the morning and notice that your test suite has been running for 12 hours. 00:40:08.440 --> 00:40:11.020 And it's only really supposed to run for 20 minutes. 00:40:11.020 --> 00:40:12.460 Yeah, absolutely. 00:40:12.460 --> 00:40:19.120 Yeah, if it's unattended or it's really long-running or you're talking to something that who knows how long it's going to take to respond. 00:40:19.120 --> 00:40:23.580 Yeah, maybe even have that with instruments that you're testing and whatnot. 00:40:23.780 --> 00:40:24.920 Yeah, we definitely use it. 00:40:24.920 --> 00:40:31.640 And the nice thing is you can have a global timeout to say, hey, no tests should run longer than this long. 00:40:31.640 --> 00:40:33.200 So whatever, 10 minutes. 00:40:33.200 --> 00:40:35.480 Whatever is in your system is a reasonable amount of time. 00:40:35.480 --> 00:40:38.040 And then you can override that within. 00:40:38.040 --> 00:40:43.040 We can put decorators on individual tests to say, well, this test is really fast. 00:40:43.040 --> 00:40:44.660 It really should only be this long. 00:40:44.660 --> 00:40:47.120 And whatever, you can fine-tune it if you need to. 00:40:47.260 --> 00:40:47.420 Nice. 00:40:47.420 --> 00:40:48.800 All right, another one. 00:40:48.800 --> 00:40:50.900 I think I threw this one on the list is spec. 00:40:50.900 --> 00:40:53.260 Yeah, I don't know anything about this. 00:40:53.260 --> 00:40:55.800 Yeah, so spec, it's a little bit like sugar. 00:40:55.800 --> 00:40:58.300 It allows you to sort of control the output. 00:40:58.800 --> 00:41:02.420 So you can group tests by classes or by files. 00:41:02.420 --> 00:41:06.200 The failed, passed, and skipped ones are marked and colored. 00:41:06.200 --> 00:41:11.120 You can remove like test underscore and underscore stuff from the test. 00:41:11.120 --> 00:41:17.740 So it'll like, you might have a name where it's like test user can log in with underscores. 00:41:17.740 --> 00:41:22.840 And I think it'll take those out so it looks more like English, like user can log in or something like that. 00:41:22.840 --> 00:41:25.900 So it's just a nice little one for customizing. 00:41:25.900 --> 00:41:27.700 There's some options you can pass to it and whatnot. 00:41:27.700 --> 00:41:28.000 Cool. 00:41:28.220 --> 00:41:28.560 Yeah, yeah. 00:41:28.560 --> 00:41:31.360 I've heard you talk about pytest Picked before. 00:41:31.360 --> 00:41:33.520 Yeah, that's a fun little plugin. 00:41:33.520 --> 00:41:36.000 So do you remember what this does? 00:41:36.000 --> 00:41:36.500 Yes. 00:41:36.500 --> 00:41:38.320 Yeah, so this is the one. 00:41:38.320 --> 00:41:41.480 And we weren't totally sure like how smart it was. 00:41:41.480 --> 00:41:43.240 We had a bit of a conversation about it. 00:41:43.240 --> 00:41:51.380 But this one will run tests that are not checked into GitHub or have changes that are not checked into GitHub. 00:41:51.380 --> 00:41:52.320 Oh, that's nice. 00:41:52.320 --> 00:41:56.380 Yeah, so it'll run tests related to unstaged files. 00:41:57.060 --> 00:42:07.240 And if it sees some that are, you know, there, it'll basically, if you're in a GitHub repo, you can run tests that are modified, but not checked in, not committed. 00:42:07.440 --> 00:42:09.760 Yeah, and that's usually often what you want to be doing, right? 00:42:09.760 --> 00:42:10.620 Exactly. 00:42:10.620 --> 00:42:21.320 What I would really like it to do, and I don't think it does, and people can add a comment at the bottom of the show page if that's changed or still misunderstood. 00:42:21.660 --> 00:42:29.740 But I would love it to somehow use coverage to see what are the files that need to be retested. 00:42:29.740 --> 00:42:34.240 Like what tests touch, like I've got a non-test file I change. 00:42:34.240 --> 00:42:35.700 What tests need to be rerun? 00:42:35.700 --> 00:42:39.180 I think it only just reruns the tests that are changed. 00:42:39.180 --> 00:42:41.600 I don't think it uses coverage, but I could be wrong about that. 00:42:41.600 --> 00:42:43.540 But nonetheless, it's still pretty cool. 00:42:43.720 --> 00:42:47.440 Yeah, I mean, all of the pieces are together for somebody to do something like this. 00:42:47.440 --> 00:42:47.800 Yeah. 00:42:47.800 --> 00:42:52.040 Because coverage has a context feature now, so you can tell it. 00:42:52.040 --> 00:42:59.480 And with pytestCov, it automatically sets it up so that you can tell which tests touched which parts of your code. 00:43:00.100 --> 00:43:01.760 And the idea seems like great. 00:43:01.760 --> 00:43:07.280 One of the issues with that is there's a whole bunch of your code that gets hit by every test. 00:43:07.280 --> 00:43:07.640 Yeah. 00:43:07.640 --> 00:43:10.340 Because it's like startup code or something like that. 00:43:10.340 --> 00:43:11.080 Right, right. 00:43:11.080 --> 00:43:12.740 Well, you just don't change the startup code. 00:43:12.740 --> 00:43:13.920 You can't edit that part. 00:43:13.920 --> 00:43:14.680 Yeah. 00:43:14.680 --> 00:43:24.040 No, but still, it is a really cool idea to say like just run the stuff that might have had some influence on. 00:43:24.040 --> 00:43:24.600 That's really cool. 00:43:24.600 --> 00:43:24.940 Yeah. 00:43:24.940 --> 00:43:28.400 So pytestPick doesn't do that, but it has this. 00:43:28.400 --> 00:43:33.000 It does a really cool, it's a nice addition to be able to run the tests that you're working on. 00:43:33.000 --> 00:43:40.980 Yeah, I could totally see just running this and then using like a pre-commit hook to run everything or just, you know, right before you check in, just run everything. 00:43:40.980 --> 00:43:44.240 Or even just check it in and it runs in CI and doesn't get merged if it fails, right? 00:43:44.240 --> 00:43:48.240 Like if there's some other gated mechanism further down, then this seems pretty cool. 00:43:48.240 --> 00:43:49.040 Yeah, definitely. 00:43:49.040 --> 00:43:49.420 Yeah. 00:43:49.420 --> 00:43:53.680 So this next one, FreezeGun, is all about time. 00:43:53.680 --> 00:43:56.980 And this one I feel like is probably packaged as a fixture. 00:43:56.980 --> 00:43:59.120 At least it has a fixture concept. 00:43:59.120 --> 00:44:00.260 Definitely as a fixture. 00:44:00.260 --> 00:44:04.040 It doesn't run by default, but you have to kind of turn it on when you know it. 00:44:04.040 --> 00:44:07.200 Yeah, this one, people know it once they hear what it does. 00:44:07.200 --> 00:44:08.560 You don't want this to run by default. 00:44:08.560 --> 00:44:12.140 I like to list this one, but just, it's such a great name, FreezeGun. 00:44:12.140 --> 00:44:13.020 Oh, yeah. 00:44:13.020 --> 00:44:15.060 It's for changing time. 00:44:15.360 --> 00:44:24.740 You can easily, there's fixtures around it so that you can easily either freeze time so that you're, anytime you grab the time of day or the date time stuff, it's always going to be the same. 00:44:24.740 --> 00:44:29.760 Or you can, you know, fast forward or change the date or do things like that. 00:44:29.820 --> 00:44:36.840 So one of the hard things is around testing code that deals with time because you have to know what the answer is going to be. 00:44:36.840 --> 00:44:38.640 And it's going to be different every time. 00:44:38.640 --> 00:44:42.300 So FreezeGun allows you to test time-related tests really easily. 00:44:42.300 --> 00:44:43.980 Plus, it's just such a great name. 00:44:44.860 --> 00:44:45.460 It is. 00:44:45.460 --> 00:44:54.800 And you have these functions, they can take a freezer object and then basically that, what is that probably behind the scenes patch, like the time functions, date time.now and whatnot. 00:44:54.800 --> 00:44:58.300 And then you can also do like freezer, move to some other time. 00:44:58.300 --> 00:45:02.440 And then like instantly you are transformed either into the future or the past. 00:45:02.440 --> 00:45:02.720 Yeah. 00:45:02.720 --> 00:45:08.120 And I think some people might be using mock for that sort of thing in the past, but this is cleaner. 00:45:08.120 --> 00:45:09.040 Use this. 00:45:09.040 --> 00:45:13.400 Look, if you could say, let's mock out date time.date time.now, or you could use FreezeGun. 00:45:13.400 --> 00:45:14.220 Like, which do you want to do? 00:45:14.220 --> 00:45:14.580 Come on. 00:45:14.720 --> 00:45:15.080 Exactly. 00:45:15.080 --> 00:45:15.660 Yeah. 00:45:15.660 --> 00:45:16.140 FreezeGun. 00:45:16.140 --> 00:45:17.520 All right. 00:45:17.520 --> 00:45:20.480 We've got one more that's on your list and then an extra one I threw in at the end. 00:45:20.480 --> 00:45:21.480 pytest-check. 00:45:21.480 --> 00:45:21.980 Yeah. 00:45:21.980 --> 00:45:26.140 I wrote a plugin called pytest-check and solved a problem I had. 00:45:26.140 --> 00:45:34.140 So the problem is the normal way to stop or to fail a test is to use assert or raise an exception. 00:45:34.140 --> 00:45:36.220 Both of those stop the execution. 00:45:36.220 --> 00:45:39.300 I wanted to be able to do lots of things. 00:45:39.300 --> 00:45:44.580 Like if I want to test lots of attributes of some object, I want to be able to have a lot of things. 00:45:44.580 --> 00:45:45.740 I want to have lots of checks in there. 00:45:45.740 --> 00:45:48.840 And I want to see all of the failures, not just the first one. 00:45:48.840 --> 00:45:54.660 So pytest-check solves this by allowing multiple failures per test with these little check functions. 00:45:54.660 --> 00:46:06.220 But you can also, it also provides a context manager so that you can say within this context, you know, you can use normal asserts, but within a context that will continue after that context. 00:46:06.220 --> 00:46:11.900 So a similar solution is available called subtests and with the pytest SubTest plugin. 00:46:12.160 --> 00:46:18.620 But I still like pytest-check better because it works with stop on fail and X fail, whereas subtests do not. 00:46:18.620 --> 00:46:18.960 Okay. 00:46:18.960 --> 00:46:19.940 Yeah, this is really cool. 00:46:19.940 --> 00:46:28.080 So you can pass in a check fixture to your test and you can say check greater AB less than or equal to this. 00:46:28.940 --> 00:46:30.100 This is not in that. 00:46:30.100 --> 00:46:30.720 This is not in that. 00:46:30.720 --> 00:46:35.700 And then it'll tell you like it was not greater and it's not in there as the error. 00:46:35.700 --> 00:46:36.560 It's just something like that, right? 00:46:36.560 --> 00:46:38.040 It's a fairly simple thing. 00:46:38.040 --> 00:46:40.180 It's a fixture that just collects failure. 00:46:40.180 --> 00:46:45.560 It like wraps things right with an assert context manager or a try except block. 00:46:45.840 --> 00:46:50.920 And if a failure happens, it just, instead of stopping the test, it records it into a list. 00:46:50.920 --> 00:46:57.040 And then after the test is complete, it'll like kind of hack with the pytest guts to make it fail. 00:46:57.040 --> 00:46:57.400 Okay. 00:46:57.400 --> 00:46:58.300 That's really cool. 00:46:58.300 --> 00:46:59.540 I definitely like this. 00:46:59.540 --> 00:47:08.180 I'm thinking like if I get just the text printout of the build failed or the CI failed, you know, even, you know, that I just see it. 00:47:08.180 --> 00:47:15.720 And then when I'm running it and like PyCharm or something, I'd like to just look at it and go, oh, it failed because if this is true and that's true and this, 00:47:15.780 --> 00:47:24.140 like I see, right, you don't have to go and actually debug through it or be, it might be that there's enough information here to actually know what's going on and just fix it. 00:47:24.140 --> 00:47:24.360 Yep. 00:47:24.360 --> 00:47:31.800 And also just more information is good sometimes dealing with, I mean, I definitely use it a lot with test equipment, with signal processing and stuff. 00:47:31.800 --> 00:47:34.940 A signal might be wrong, but why is the signal wrong? 00:47:34.940 --> 00:47:36.860 It might be the wrong frequency. 00:47:36.860 --> 00:47:37.900 It might be the wrong bandwidth. 00:47:37.900 --> 00:47:39.640 It might be, I mean, lots of different things. 00:47:39.720 --> 00:47:46.040 And if I'm checking four or five attributes of the signal, I really want all of that information for a failure. 00:47:46.040 --> 00:47:46.900 Yeah, I agree. 00:47:46.900 --> 00:47:50.060 There might be different aspects that all come together to make it successful. 00:47:50.060 --> 00:47:51.700 So report all the aspects. 00:47:51.700 --> 00:47:51.940 Yeah. 00:47:51.940 --> 00:47:52.780 That's cool. 00:47:52.780 --> 00:47:53.440 All right. 00:47:53.440 --> 00:48:00.780 I want to throw one more in that's not really part of pytest per se, but absolutely can be used with pytest because, hey, it raises exceptions. 00:48:01.660 --> 00:48:03.260 And that's called fluent check. 00:48:03.260 --> 00:48:14.060 So this one allows you to use a, well, a fluent API, you know, where you say operation one dot operation two dot operation three to have a more English language thing. 00:48:14.060 --> 00:48:19.840 And to me, it feels a little bit like your check as well, but it's not a pytest plugin or fixture. 00:48:19.840 --> 00:48:22.780 So it has a couple of APIs. 00:48:22.780 --> 00:48:23.840 I like the is API. 00:48:23.840 --> 00:48:33.700 So you can say like is in dot not none dot float dot between zero and one, and it'll actually check all those things and report like nice messages. 00:48:33.700 --> 00:48:41.180 Like if it's not a float, it would say, you know, whatever it is, here's its value is not a float and you expected it to be right. 00:48:41.180 --> 00:48:43.480 So a little more readable. 00:48:43.480 --> 00:48:44.740 Some people like it. 00:48:44.740 --> 00:48:47.160 Some people just want to assert stuff and either one's fine. 00:48:47.160 --> 00:48:49.200 But if you like the style, this is a pretty cool library. 00:48:49.200 --> 00:48:51.960 It's not mine, but I did do some PRs. 00:48:52.020 --> 00:48:54.220 I did basically create the is API for it. 00:48:54.220 --> 00:48:54.560 Cool. 00:48:54.560 --> 00:48:56.080 Does it have not a hot dog? 00:48:56.080 --> 00:49:02.020 It should also have is a teapot or something like that from the for web testing, you know, the four for 18. 00:49:02.020 --> 00:49:05.320 Yeah, we should throw some of those in there. 00:49:05.320 --> 00:49:05.860 Like, why not? 00:49:05.860 --> 00:49:06.320 We do it. 00:49:06.320 --> 00:49:12.980 Anyway, those are kind of fun for just like inside your test expressing what you're testing a little bit better. 00:49:12.980 --> 00:49:14.340 I think check is really cool as well. 00:49:14.340 --> 00:49:15.320 And it's a bit similar. 00:49:15.320 --> 00:49:17.660 Yeah, but this one doesn't stop. 00:49:17.660 --> 00:49:18.980 It just throws the first one. 00:49:18.980 --> 00:49:19.600 All right. 00:49:19.600 --> 00:49:21.660 Well, that's a great, a great list. 00:49:21.660 --> 00:49:23.440 You know, maybe just really quick, we could check in. 00:49:23.440 --> 00:49:24.420 How's your book doing? 00:49:24.420 --> 00:49:26.140 You wrote a book about some of the stuff. 00:49:26.140 --> 00:49:26.560 Yeah. 00:49:26.560 --> 00:49:28.700 So I wrote Python testing with pytest. 00:49:28.700 --> 00:49:36.020 That was gosh, it was it's been a couple of years now that it's been out and it's still going going strong. 00:49:36.200 --> 00:49:43.880 The of course, the big chunk of sales right after the book was released and quite a few during beta too. 00:49:43.880 --> 00:49:44.460 And that was neat. 00:49:44.460 --> 00:49:47.280 But it's still selling really well. 00:49:47.280 --> 00:49:48.740 And I'm really happy with it. 00:49:48.740 --> 00:49:52.480 And it's still valid, even though it's it's a couple of years old now. 00:49:52.660 --> 00:50:01.720 And I wrote it in conjunction with talking with a lot of the pytest core maintainers to make sure I wasn't putting anything in there that would be deprecated soon. 00:50:01.720 --> 00:50:02.420 Yeah. 00:50:02.420 --> 00:50:04.520 So it's still valid. 00:50:04.520 --> 00:50:05.020 It's good. 00:50:05.020 --> 00:50:05.340 Awesome. 00:50:05.340 --> 00:50:05.640 Yeah. 00:50:05.640 --> 00:50:06.600 People can take that up. 00:50:06.600 --> 00:50:08.640 They want to read, you know, dive into this much more. 00:50:08.940 --> 00:50:13.780 And then also you did an episode that also talked about pytest plugins over on testing code. 00:50:13.780 --> 00:50:14.120 Yeah. 00:50:14.120 --> 00:50:17.740 It was episode 104 with Anthony Sotil. 00:50:17.740 --> 00:50:23.740 The episode that you and I just did, we kind of picked our own, some of our favorites, some of the things that we think people should know about. 00:50:23.740 --> 00:50:30.000 What we did with episode 104, we took download counts and took the top some of the top 28 downloads. 00:50:30.000 --> 00:50:30.520 I see. 00:50:30.520 --> 00:50:33.340 Popular by usage, not by curation. 00:50:33.340 --> 00:50:33.900 Yeah. 00:50:33.900 --> 00:50:34.360 Yep. 00:50:34.360 --> 00:50:34.820 Exactly. 00:50:34.820 --> 00:50:35.680 Awesome. 00:50:35.680 --> 00:50:36.640 All right. 00:50:36.640 --> 00:50:37.540 Well, this is really fun, Brian. 00:50:37.660 --> 00:50:40.440 But before we get out of here, two final questions as per usual. 00:50:40.440 --> 00:50:43.180 If you're going to write some Python code, what editor do you use? 00:50:43.180 --> 00:50:43.700 PyCharm. 00:50:43.700 --> 00:50:44.160 All right. 00:50:44.160 --> 00:50:44.460 Right on. 00:50:44.460 --> 00:50:46.200 It's got some sweet built-in test runners. 00:50:46.200 --> 00:50:46.560 Yeah. 00:50:46.560 --> 00:50:47.880 It's really clean with testing. 00:50:47.880 --> 00:50:48.280 Yep. 00:50:48.280 --> 00:50:50.960 And then notable PyPI package. 00:50:50.960 --> 00:50:52.680 I mean, I guess many of these are. 00:50:52.680 --> 00:50:55.640 Anything in particular comes to mind you want to throw out there? 00:50:55.640 --> 00:50:56.620 It's something different. 00:50:56.620 --> 00:51:00.440 I'm kind of a huge fan of Flit for packaging. 00:51:00.440 --> 00:51:04.320 So that's the easiest way to package right now, I think. 00:51:04.320 --> 00:51:04.660 All right. 00:51:04.780 --> 00:51:12.700 As opposed to some of the other alternatives like pip and PMP and setup tools and all those 00:51:12.700 --> 00:51:13.120 things, right? 00:51:13.120 --> 00:51:14.160 Yeah. 00:51:14.160 --> 00:51:14.380 Yeah. 00:51:14.380 --> 00:51:14.660 Sweet. 00:51:14.660 --> 00:51:15.300 All right. 00:51:15.300 --> 00:51:15.680 Yeah. 00:51:15.680 --> 00:51:16.000 What is that? 00:51:16.000 --> 00:51:17.180 There's poetry and... 00:51:17.180 --> 00:51:17.360 Yeah. 00:51:17.360 --> 00:51:18.400 Poetry as well. 00:51:18.400 --> 00:51:18.860 That's right. 00:51:18.860 --> 00:51:20.140 All right. 00:51:20.140 --> 00:51:21.360 Well, final call to action. 00:51:21.360 --> 00:51:23.700 People are excited about pytest and pytest plugins. 00:51:23.700 --> 00:51:24.700 What should they do? 00:51:24.700 --> 00:51:26.060 They should buy my book, actually. 00:51:26.580 --> 00:51:31.780 So one of the things I made sure to do is to tell people how to create their own plugins 00:51:31.780 --> 00:51:32.640 within the book. 00:51:32.640 --> 00:51:36.100 So I think it's a really good introduction to that. 00:51:36.100 --> 00:51:43.560 Also, I guess, final call to action, go subscribe to Testing Code and podcast also and test your 00:51:43.560 --> 00:51:43.760 code. 00:51:43.760 --> 00:51:44.600 Go out and test something. 00:51:44.600 --> 00:51:46.680 Do you guys talk about pytests over there sometimes? 00:51:46.680 --> 00:51:47.360 Sometimes. 00:51:47.360 --> 00:51:47.700 Yeah. 00:51:47.700 --> 00:51:48.840 A lot of times we do. 00:51:48.840 --> 00:51:49.940 Yeah. 00:51:49.940 --> 00:51:50.760 That's great. 00:51:50.760 --> 00:51:51.180 Awesome. 00:51:51.180 --> 00:51:52.280 Cool. 00:51:52.280 --> 00:51:53.640 Well, thanks for being on the show. 00:51:53.640 --> 00:51:55.120 Nice to have you back as always. 00:51:55.120 --> 00:51:55.320 Yeah. 00:51:55.320 --> 00:51:55.700 Thank you. 00:51:55.760 --> 00:51:55.980 You bet. 00:51:55.980 --> 00:51:56.180 Bye. 00:51:56.180 --> 00:51:56.400 Bye. 00:51:56.400 --> 00:51:59.800 This has been another episode of Talk Python To Me. 00:51:59.800 --> 00:52:05.260 Our guest on this episode was Brian Okken, and it's been brought to you by Linode and Sentry. 00:52:05.260 --> 00:52:10.100 Start your next Python project on Linode's state-of-the-art cloud service. 00:52:10.100 --> 00:52:14.420 Just visit talkpython.fm/Linode, L-I-N-O-D-E. 00:52:14.420 --> 00:52:17.720 You'll automatically get a $20 credit when you create a new account. 00:52:17.720 --> 00:52:20.420 Take some stress out of your life. 00:52:20.420 --> 00:52:25.040 Get notified immediately about errors in your web applications with Sentry. 00:52:25.440 --> 00:52:29.100 Just visit talkpython.fm/sentry and get started for free. 00:52:29.100 --> 00:52:31.260 Want to level up your Python? 00:52:31.260 --> 00:52:36.120 If you're just getting started, try my Python Jumpstart by Building 10 Apps course. 00:52:36.120 --> 00:52:41.220 Or if you're looking for something more advanced, check out our new async course that digs into 00:52:41.220 --> 00:52:44.280 all the different types of async programming you can do in Python. 00:52:44.540 --> 00:52:48.940 And of course, if you're interested in more than one of these, be sure to check out our Everything Bundle. 00:52:48.940 --> 00:52:50.820 It's like a subscription that never expires. 00:52:50.820 --> 00:52:52.980 Be sure to subscribe to the show. 00:52:52.980 --> 00:52:55.380 Open your favorite podcatcher and search for Python. 00:52:55.380 --> 00:52:56.600 We should be right at the top. 00:52:56.600 --> 00:53:01.440 You can also find the iTunes feed at /itunes, the Google Play feed at /play, 00:53:01.440 --> 00:53:05.600 and the direct RSS feed at /rss on talkpython.fm. 00:53:06.100 --> 00:53:07.680 This is your host, Michael Kennedy. 00:53:07.680 --> 00:53:09.180 Thanks so much for listening. 00:53:09.180 --> 00:53:10.240 I really appreciate it. 00:53:10.240 --> 00:53:12.000 Now get out there and write some Python code. 00:53:12.000 --> 00:53:32.880 I'll see you next time.