WEBVTT 00:00:00.001 --> 00:00:03.780 You know you should be testing your code, right? How do you know whether it's well-tested? 00:00:03.780 --> 00:00:08.380 Are you testing the right things? If you're not using code coverage, chances are you're guessing. 00:00:08.380 --> 00:00:12.940 But you don't need to guess. Just grab coverage.py maintained by our guest this week, 00:00:12.940 --> 00:00:18.880 Ned Batchelder. This is Talk Python To Me, episode 178, recorded September 10th, 2018. 00:00:18.880 --> 00:00:38.940 Welcome to Talk Python To Me, a weekly podcast on Python, the language, the libraries, the ecosystem, and the personalities. 00:00:38.940 --> 00:00:43.080 This is your host, Michael Kennedy. Follow me on Twitter, where I'm @mkennedy. 00:00:43.080 --> 00:00:46.960 Keep up with the show and listen to past episodes at talkpython.fm, 00:00:47.060 --> 00:00:49.480 and follow the show on Twitter via at Talk Python. 00:00:49.480 --> 00:00:53.440 This episode is brought to you by Brilliant.org and Manning. 00:00:53.440 --> 00:00:57.280 Please check out what they're offering during their segments. It really helps support the show. 00:00:57.280 --> 00:00:59.120 Ned, welcome to Talk Python. 00:00:59.120 --> 00:01:01.100 Hi, thanks, Michael. It's great to be here. 00:01:01.100 --> 00:01:06.380 It's great to finally have you on the show. I cannot believe we are at episode 178, 00:01:06.380 --> 00:01:08.640 and you have not been a guest on the show. How did this happen? 00:01:08.640 --> 00:01:12.360 I know. You're doing something wrong over there. I don't know. You're doing something great over there. 00:01:12.360 --> 00:01:14.920 You've got to episode 178, which is astounding. 00:01:15.140 --> 00:01:19.500 Anyone who says, I'm going to do a thing and then does it 178 times, clearly is doing something right. 00:01:19.500 --> 00:01:22.160 Yeah, we're coming up on, I think on three years. 00:01:22.160 --> 00:01:25.560 Yeah, actually, maybe over three years. So I got to do some quick math. 00:01:25.560 --> 00:01:29.220 But yeah, it's been going for a while, and it's really fun. I'm just absolutely loving it. 00:01:29.220 --> 00:01:34.180 And we're going to dig into a project that you've been actually working on more than three years, right? 00:01:34.440 --> 00:01:40.880 Yes. So coverage.py is a project that I've been maintaining for 14 years, which seems crazy. 00:01:40.880 --> 00:01:44.340 That is really amazing. And kudos to you for doing that. That's great. 00:01:44.340 --> 00:01:47.500 I think what I realized about myself is that I'm very inertial. 00:01:47.500 --> 00:01:51.500 It's hard for me to start new things, and then it's also hard for me to stop old things. 00:01:51.500 --> 00:01:53.500 Once you get them rolling, they just keep going. 00:01:53.500 --> 00:01:53.720 That's right. 00:01:53.720 --> 00:01:54.520 That's right. 00:01:54.640 --> 00:01:56.560 It's not a bad trade at all. 00:01:56.560 --> 00:02:02.140 So before we get into all the details of code coverage and so on, let's just get a little background on you. 00:02:02.140 --> 00:02:03.380 How did you get into programming in Python? 00:02:03.380 --> 00:02:06.220 Well, so I've got kind of an unusual story that way. 00:02:06.220 --> 00:02:10.080 So I'm fairly old for the Python world. 00:02:10.080 --> 00:02:15.220 I'm 56 years old, and I got into programming because my mother was a software person, too. 00:02:15.220 --> 00:02:20.800 She was a programmer in the 1960s and 70s, 80s and 90s, I guess, until she retired. 00:02:20.800 --> 00:02:22.020 Back when programming was really hard. 00:02:22.020 --> 00:02:24.500 There was no internet, not many books. 00:02:25.280 --> 00:02:26.720 Programming was different back then. 00:02:26.720 --> 00:02:28.320 Yeah, there was definitely no internet. 00:02:28.320 --> 00:02:33.020 The books were in a big, huge three-ring binder at the other side of the room, et cetera, et cetera. 00:02:33.020 --> 00:02:36.220 But the cool thing was that she would bring home some of those books. 00:02:36.220 --> 00:02:42.540 So I remember as a kid looking through the IBM 360 programmer's manuals and puzzling over what this stuff might mean. 00:02:42.540 --> 00:02:45.440 So I sort of come by programming naturally. 00:02:45.440 --> 00:02:46.840 I've been doing it for a while. 00:02:46.840 --> 00:02:51.160 I joke that it's the only skill I've got, so it's a good thing that people will hire me to do it. 00:02:51.880 --> 00:02:56.400 And I got into Python probably in the year 2000, maybe 1999. 00:02:56.400 --> 00:03:01.340 I'd been working at Lotus on Lotus Notes, which is a collaboration environment. 00:03:01.340 --> 00:03:03.080 Was that in C++ before? 00:03:03.080 --> 00:03:05.060 Well, Lotus Notes is written in C. 00:03:05.060 --> 00:03:13.260 But the reason, the way I got to Python was that Lotus Notes, with its access controls and collaboration controls, someone said, oh, you should look at this thing called Zope. 00:03:13.260 --> 00:03:14.800 It also does stuff like that. 00:03:15.000 --> 00:03:17.700 And I looked at Zope, and I thought, you know, Zope's kind of cool. 00:03:17.700 --> 00:03:18.540 I don't really need that. 00:03:18.540 --> 00:03:21.120 But this Python thing it's written in seems kind of interesting. 00:03:21.120 --> 00:03:29.660 And so basically from that point on, when I had a choice of tools for writing some little tools or some scripting or some automation, I would reach for Python. 00:03:29.660 --> 00:03:31.780 And that's just grown and grown since then. 00:03:31.780 --> 00:03:34.980 Now, I guess I've been using it for 18 years or so. 00:03:34.980 --> 00:03:35.800 That's really cool. 00:03:35.800 --> 00:03:38.440 And Python itself has grown with you, right? 00:03:38.440 --> 00:03:41.360 I mean, Python of 18 years ago is not Python of 2018. 00:03:41.360 --> 00:03:42.380 Yeah, and it's funny. 00:03:42.380 --> 00:03:45.640 I don't even know what version of Python that was. 00:03:45.640 --> 00:03:47.860 It might have been a 1.x, I guess. 00:03:47.860 --> 00:03:50.240 Yeah, and Python has definitely grown. 00:03:50.240 --> 00:03:55.040 I feel like I sort of made a technology choice there, and it's worked out very well. 00:03:55.040 --> 00:04:02.240 I've watched Python grow into at least two new major niches since then, into web dev and now into data science. 00:04:02.240 --> 00:04:04.800 I feel much more comfortable in web dev. 00:04:04.800 --> 00:04:09.080 I feel a little bit like I'm getting left behind with the data science and machine learning. 00:04:09.080 --> 00:04:18.060 And even just hanging out where people ask questions about things, it's very clear that the center of interest is outside my expertise now. 00:04:18.060 --> 00:04:20.100 So I've got a lot to learn. 00:04:20.100 --> 00:04:27.180 It's interesting that you can be an expert in Python after 18 years and be a beginner at the things that people want to use Python for. 00:04:27.180 --> 00:04:28.240 Yeah, that's super interesting. 00:04:28.240 --> 00:04:34.900 Yeah, like if somebody said, hey, Michael, go do this plot with Matplotlib and get this data loaded up with Pandas. 00:04:34.900 --> 00:04:41.640 I'm pretty sure I could not do that without documentation or examples in front of me because I spend most of my time writing web and database code. 00:04:41.640 --> 00:04:44.740 Yeah, and I have done Matplotlib in a little bit of notebooks. 00:04:45.020 --> 00:04:48.360 And I'm, you know, I am like the typical, I'm on Stack Overflow. 00:04:48.360 --> 00:04:50.380 I'm just searching for stuff. 00:04:50.380 --> 00:04:51.360 I see a chunk of code. 00:04:51.360 --> 00:04:52.080 I don't know what it means. 00:04:52.080 --> 00:04:52.940 I paste it in. 00:04:52.940 --> 00:04:53.660 It seems to work. 00:04:53.660 --> 00:04:55.060 We're done, you know. 00:04:55.060 --> 00:04:58.480 And I would love to have a deeper foundational understanding. 00:04:58.480 --> 00:05:01.280 But I don't have day-to-day problems that need those tools. 00:05:01.560 --> 00:05:04.980 So there's not much chance for me to really get that learning. 00:05:04.980 --> 00:05:08.220 Yeah, I feel like it's sort of bimodal now. 00:05:08.220 --> 00:05:13.180 There's these two big areas that Python is really being used a lot in, at least. 00:05:13.180 --> 00:05:20.380 And, you know, did you catch Jake Vander Plaas' keynote at PyCon 2017 about Python being a mosaic? 00:05:20.380 --> 00:05:21.300 I didn't. 00:05:21.300 --> 00:05:25.600 2017 was the PyCon I missed of the last decade of PyCons. 00:05:25.600 --> 00:05:26.720 That was a good one. 00:05:27.100 --> 00:05:32.740 Well, basically it was, look, there's all these different ways and people are using Python and their goals. 00:05:32.740 --> 00:05:38.500 And their entire purpose of using Python may be very different than the person you're sitting next to. 00:05:38.500 --> 00:05:41.480 But if you learn to appreciate it, it just makes it richer. 00:05:41.480 --> 00:05:45.400 And I thought it was a really great way to sort of say, like, look at all these things people are doing. 00:05:45.400 --> 00:05:48.040 They have different motivations and whatnot. 00:05:48.040 --> 00:05:51.580 But it's just as valid Python in style. 00:05:51.580 --> 00:05:53.280 It's for a different use case. 00:05:53.280 --> 00:05:53.960 Right, exactly. 00:05:53.960 --> 00:05:55.580 Yeah, I feel like we're definitely there. 00:05:55.580 --> 00:06:00.200 Yeah, one of the things that I do is organize the Boston Python user group. 00:06:00.200 --> 00:06:02.100 And we have project nights every month. 00:06:02.100 --> 00:06:05.320 And we just get a big room with a lot of round tables. 00:06:05.320 --> 00:06:07.100 And we put labels on each table. 00:06:07.100 --> 00:06:09.580 So there's a table labeled beginners and a table labeled web. 00:06:09.580 --> 00:06:12.680 And then we get data and we get science and we get hardware. 00:06:12.680 --> 00:06:18.300 And it's just really interesting to see the variety of uses that people are putting Python to. 00:06:18.300 --> 00:06:22.340 And there was a woman who came last month, was sitting at the beginners table. 00:06:22.820 --> 00:06:25.820 And towards the end of the night, I was asking her more about what she wanted to do. 00:06:25.820 --> 00:06:27.220 And she mentioned biology. 00:06:27.220 --> 00:06:31.800 And I said, oh, you should have at the beginning of the night, you should have stood up and said, I'm doing biology. 00:06:31.800 --> 00:06:33.720 We could have found you some biologists to talk to. 00:06:33.720 --> 00:06:35.040 And she laughed like I was joking. 00:06:35.040 --> 00:06:40.520 But then I introduced her to the four or five biologists across the room who were doing biology with Python. 00:06:41.320 --> 00:06:49.100 So it's really a very rich ecosystem of expertise and individual domains, which is fascinating. 00:06:49.100 --> 00:06:56.800 One of the things I love about Python is a lot of people seem to come to it with another expertise, kind of like you were just saying, right? 00:06:56.800 --> 00:07:02.040 Like if you're a C++ developer, there's a good chance you may be a developer first, right? 00:07:02.040 --> 00:07:05.440 But if you're doing Python, you may be something else first that uses Python. 00:07:05.560 --> 00:07:08.020 And I think that just makes us a richer community. 00:07:08.020 --> 00:07:17.720 That's why I think it's doing so well in data science is that it's, for whatever reason, it's the kind of language and environment that those types of people can succeed in. 00:07:17.720 --> 00:07:18.580 Yeah, absolutely. 00:07:19.060 --> 00:07:21.020 So you mentioned the Boston user group. 00:07:21.020 --> 00:07:22.900 This is a global podcast. 00:07:22.900 --> 00:07:25.280 The internet doesn't have a zip code or whatever. 00:07:25.280 --> 00:07:31.040 But for people generally in the Northeast, like you want to just tell them really quickly about it so they can find it if they don't know? 00:07:31.040 --> 00:07:31.220 Sure. 00:07:31.220 --> 00:07:33.860 So the Boston Python user group is a big group. 00:07:33.860 --> 00:07:43.120 We run events twice each month, generally, a project night, which is basically a two and a half hour unstructured hackathon with some sorting by topic, like I just described. 00:07:43.120 --> 00:07:47.580 And then most months we also run a presentation night where we try to find people to give talks. 00:07:47.580 --> 00:07:49.400 We did lightning talks for August. 00:07:49.400 --> 00:07:56.620 I'm working on grooming a web scraping talk for September, and we're going to have science talks for November. 00:07:56.620 --> 00:07:58.300 And we're very friendly. 00:07:58.300 --> 00:07:59.400 We're big and open. 00:07:59.400 --> 00:08:01.820 We're on meetup.com or bostonpython.com. 00:08:01.820 --> 00:08:04.240 And if you're anywhere around, come and see us. 00:08:04.240 --> 00:08:08.400 We've had people travel as long as three hours to get to events. 00:08:08.400 --> 00:08:10.840 So all of New England is kind of in scope. 00:08:11.340 --> 00:08:12.220 Yeah, absolutely. 00:08:12.220 --> 00:08:14.940 Well, I guess it depends on the time as well, right? 00:08:14.940 --> 00:08:15.820 Rush hour and all that. 00:08:15.820 --> 00:08:18.560 Three hours could be not far away in certain parts of Boston. 00:08:18.560 --> 00:08:19.180 That's true. 00:08:19.180 --> 00:08:23.840 The really memorable one was the father and son who took a three-hour bus ride down from Maine. 00:08:23.840 --> 00:08:27.660 And that kid was 13, and he was one of the smartest people I've ever met. 00:08:27.660 --> 00:08:32.480 And they were going to leave and get home at like 2 in the morning based on the bus schedule to have attended. 00:08:32.480 --> 00:08:33.500 They only came once. 00:08:33.500 --> 00:08:36.040 And I mean, I don't blame them, but it was very impressive. 00:08:36.040 --> 00:08:36.760 No, that's cool. 00:08:36.760 --> 00:08:38.220 Is there any way to remotely attend? 00:08:38.220 --> 00:08:39.680 Any streaming options? 00:08:39.680 --> 00:08:46.840 We've never managed to be routine about videoing the presentations, which is really unfortunate. 00:08:46.840 --> 00:08:52.580 Because even in Boston, we have 8,000 people on the meetup group, and we can only fit 120 people in the room. 00:08:52.580 --> 00:08:53.780 And we always have a waiting list. 00:08:53.780 --> 00:08:55.820 So lots of people would like to see video. 00:08:55.820 --> 00:09:01.240 But we have just never managed to find the staff to make it a regular thing. 00:09:01.240 --> 00:09:05.900 It's almost got to be somebody's, their responsibility, their role to just do that, right? 00:09:05.900 --> 00:09:06.360 Absolutely. 00:09:06.360 --> 00:09:06.760 Yep. 00:09:06.760 --> 00:09:09.160 And they've got to show up every time, et cetera, et cetera. 00:09:09.160 --> 00:09:09.460 Yeah. 00:09:09.460 --> 00:09:14.580 So another thing that you do, and I'm also super passionate about, has to do with online education, right? 00:09:14.580 --> 00:09:14.980 Right. 00:09:15.060 --> 00:09:17.100 Yeah, my day job is at edX. 00:09:17.100 --> 00:09:23.820 EdX.org is the website that was founded by Harvard and MIT and puts university-level courses online. 00:09:23.820 --> 00:09:29.580 We've got, I don't know, 2,000, 3,000 courses from 130 different institutions at this point. 00:09:30.000 --> 00:09:35.700 And it's all Python and Django, and it's all open source, which is the thing that really appeals to me because I'm an open source guy. 00:09:35.700 --> 00:09:38.720 I actually work on the open source team here at edX. 00:09:38.720 --> 00:09:43.340 So we are encouraging and enabling other people to use our software to do online education. 00:09:43.340 --> 00:09:50.200 There's about 1,000 other sites besides edX.org that use OpenEdX to do their education, which is thrilling. 00:09:50.200 --> 00:09:58.640 Because as great as Harvard and MIT courses are, there's all sorts of other kinds of education that those institutions will never provide. 00:09:59.260 --> 00:10:11.900 We just recently discovered there was a website in Indonesia which has something like 150 different courses all very, very focused on specific skills that might lift someone out of poverty. 00:10:11.900 --> 00:10:19.440 You know, how to be a maid, how to do hairdressing, how to raise chickens, how to fix small engines, how to catch fish, like just all sorts of things. 00:10:19.440 --> 00:10:21.760 Super practical vocational type things. 00:10:21.760 --> 00:10:25.860 Super practical vocational in Indonesian for Indonesians. 00:10:25.860 --> 00:10:28.880 And edX.org, as many courses as we're going to get. 00:10:29.000 --> 00:10:30.940 We're never going to deliver those courses. 00:10:30.940 --> 00:10:38.840 So having the software be open source, you know, we give away education on edX.org and we give away the software to give away education to the rest of the world. 00:10:38.840 --> 00:10:43.440 And there's 1,000 sites out there that are using it, which is really, really gratifying. 00:10:43.440 --> 00:10:44.140 Oh, that's awesome. 00:10:44.140 --> 00:10:45.520 It sounds like a great project. 00:10:45.520 --> 00:10:47.820 And it's mostly Python and Django? 00:10:47.820 --> 00:10:48.380 Yeah. 00:10:48.380 --> 00:10:50.720 It's almost, I mean, JavaScript, of course, too. 00:10:50.720 --> 00:10:52.460 But yeah, it's all Python and Django. 00:10:52.460 --> 00:10:54.400 And it's almost all open source. 00:10:54.400 --> 00:10:55.200 Very cool. 00:10:55.480 --> 00:10:57.960 And we're hiring, if anyone, you know, tell them Ned sent you. 00:10:57.960 --> 00:10:58.840 There's a referral bonus. 00:10:58.840 --> 00:11:03.440 Do you guys have remote positions or it's got to be in Boston? 00:11:03.440 --> 00:11:06.360 The easiest thing to say is, let's say it's got to be in Boston. 00:11:06.360 --> 00:11:07.220 Yeah. 00:11:07.320 --> 00:11:11.020 We're not super good at remote, which is something I wish we could get better at. 00:11:11.020 --> 00:11:13.660 But that's the reality of the situation today. 00:11:13.660 --> 00:11:13.940 Yeah. 00:11:13.940 --> 00:11:19.620 So if people are listening and they want a cool Python job in the general Boston area or they're willing to get there. 00:11:19.620 --> 00:11:20.200 Yeah. 00:11:20.200 --> 00:11:20.400 Yeah. 00:11:20.400 --> 00:11:20.700 Awesome. 00:11:20.700 --> 00:11:22.140 Get in touch. 00:11:22.140 --> 00:11:22.940 Let's see what we can do. 00:11:22.940 --> 00:11:23.560 Yeah. 00:11:23.560 --> 00:11:24.240 That's really great. 00:11:24.240 --> 00:11:25.640 It sounds super fun. 00:11:25.720 --> 00:11:25.880 Okay. 00:11:25.880 --> 00:11:30.860 So let's talk about this brand new project that you just started called Coverage PY. 00:11:30.860 --> 00:11:31.360 That's right. 00:11:31.360 --> 00:11:33.880 This is a podcast from December of 2004. 00:11:33.880 --> 00:11:34.980 Exactly. 00:11:34.980 --> 00:11:36.140 All right. 00:11:36.140 --> 00:11:38.920 So let's talk about what is code coverage and what is this project? 00:11:38.920 --> 00:11:39.220 Okay. 00:11:39.220 --> 00:11:42.920 Well, and let me just start with one thing, which is I didn't actually start this project. 00:11:43.100 --> 00:11:45.480 This project was started by a guy named Gareth Reese. 00:11:45.480 --> 00:11:53.480 And back in 2004, I was working on a different Python thing and I wanted to use some code coverage on it. 00:11:53.480 --> 00:11:57.320 And I found this thing called Coverage.py and it worked almost exactly the way I wanted. 00:11:57.320 --> 00:12:03.220 And the way it didn't, I tried to make a change and I tried to get it to Gareth and Gareth didn't seem to be reachable. 00:12:03.220 --> 00:12:05.860 So I just sort of published it with my change. 00:12:05.860 --> 00:12:10.120 And 14 years later, I'm the maintainer of Coverage.py. 00:12:10.120 --> 00:12:12.800 That's how open source works, right? 00:12:12.800 --> 00:12:14.460 That's how open source works. 00:12:14.460 --> 00:12:14.660 Yeah. 00:12:14.660 --> 00:12:22.500 I'm mulling the idea of doing a lighting talk called Lies People Believe About Coverage.py, one of which is that I started it. 00:12:22.500 --> 00:12:23.520 Yeah. 00:12:23.520 --> 00:12:24.200 Okay. 00:12:24.200 --> 00:12:26.640 But to answer your question, so what is code coverage? 00:12:26.640 --> 00:12:33.200 So the idea of code coverage is you've got some product code that you've written, meaning the code you actually want to write. 00:12:33.200 --> 00:12:36.480 And then to make sure that that code works, you write some tests. 00:12:36.480 --> 00:12:40.620 And I'll give the entire audience the benefit of the doubt and say you have written tests. 00:12:41.460 --> 00:12:47.140 But now you need to know, are the tests actually doing their job, which is proving that your code works? 00:12:47.540 --> 00:12:59.020 And one way to test your tests essentially is to observe the tests running and see if all of the lines of your product code were executed by the tests. 00:12:59.020 --> 00:13:06.160 Because if there's a line of code that isn't executed when you run your entire test suite, then there's no way that line of code can be tested. 00:13:06.160 --> 00:13:08.120 The converse isn't true. 00:13:08.180 --> 00:13:11.520 If the line of code is run, it still might not be properly tested. 00:13:11.520 --> 00:13:14.100 But if it isn't run, then it's definitely not properly tested. 00:13:14.100 --> 00:13:15.060 Absolutely. 00:13:15.060 --> 00:13:18.160 If it was never executed, you know nothing about it. 00:13:18.160 --> 00:13:18.540 Absolutely. 00:13:18.540 --> 00:13:18.880 That's right. 00:13:18.880 --> 00:13:22.220 There's no way you know how that line of code works. 00:13:22.760 --> 00:13:34.040 So the code coverage in general is the automation of that process, which is it is a tool that can observe a program being run and can tell you what lines of code were run in the program. 00:13:34.040 --> 00:13:36.860 And notice in that sentence, I didn't say anything about tests. 00:13:36.860 --> 00:13:39.840 Coverage doesn't know anything about what a test is. 00:13:39.840 --> 00:13:44.980 It's just that typically the program you want to watch is your code while the tests are being run. 00:13:44.980 --> 00:13:48.800 But you could run coverage for any reason to know what parts were run or not. 00:13:48.800 --> 00:13:49.120 Right. 00:13:49.120 --> 00:13:53.540 This is typically most spoken about in terms of unit testing and other types of tests. 00:13:53.540 --> 00:13:57.640 But one example that comes to mind right away is I've got some app. 00:13:57.640 --> 00:14:07.720 It's been handed down from person to person and somehow it arrives in my lap and they're like, Michael, you've got to now add a feature or maintain this thing. 00:14:07.840 --> 00:14:09.800 And it's a big scrambled mess. 00:14:09.800 --> 00:14:11.440 And the person who knows all about it is gone. 00:14:11.440 --> 00:14:15.920 Maybe I just want to know when it does its job, does it ever even call this function? 00:14:15.920 --> 00:14:16.160 Right. 00:14:16.160 --> 00:14:22.140 Like there could be all sorts of code in there that is just nobody wanted to remove it because they didn't know for sure it was OK. 00:14:22.140 --> 00:14:25.680 But if you can run the coverage and say, actually, no, it's never executed. 00:14:25.680 --> 00:14:26.360 Let's delete it. 00:14:26.360 --> 00:14:26.760 That's right. 00:14:26.760 --> 00:14:27.560 That's right. 00:14:27.560 --> 00:14:31.040 So long as you are sure that you know how to fully exercise. 00:14:31.040 --> 00:14:31.460 Yes. 00:14:31.460 --> 00:14:32.360 Yeah. 00:14:32.360 --> 00:14:33.680 You got it. 00:14:33.680 --> 00:14:35.960 That is another another thing. 00:14:36.100 --> 00:14:51.100 But I've spent hours trying to understand what a particular function does and how it influences like a big program just to realize that actually the reason any changes I'm making to this section or try to make it do a thing have no effect because it's not being called. 00:14:51.100 --> 00:14:51.560 Right. 00:14:51.560 --> 00:14:51.760 Yeah. 00:14:51.760 --> 00:14:53.080 It's super frustrating. 00:14:53.080 --> 00:14:53.520 Right. 00:14:53.520 --> 00:14:54.760 So coverage can be used for that. 00:14:54.760 --> 00:15:07.760 But like you say, the 99.9% use case for any code coverage tool, including coverage.py, is for it to observe your test suite being run and then to tell you about your product code, which lines were run and which lines weren't. 00:15:07.760 --> 00:15:08.820 Right. 00:15:08.960 --> 00:15:12.380 And the idea being that the lines that weren't, that's what you focus in on. 00:15:12.380 --> 00:15:15.440 And you think about how can I write a test to make that line of code be run. 00:15:15.440 --> 00:15:17.460 And you'll gradually increase the coverage. 00:15:17.460 --> 00:15:18.460 And then you're right. 00:15:18.460 --> 00:15:19.940 Or you make a conscious decision. 00:15:19.940 --> 00:15:21.420 This part we don't care to test. 00:15:21.420 --> 00:15:21.860 Right. 00:15:21.860 --> 00:15:22.520 That's right. 00:15:22.520 --> 00:15:24.160 You can also decide that. 00:15:24.160 --> 00:15:24.640 Yes. 00:15:24.820 --> 00:15:31.720 But I think the important thing is, even if you're in that place, there is a core reason your application exists. 00:15:31.720 --> 00:15:35.480 There is a thing that it does and there's stuff that supports it doing that. 00:15:35.480 --> 00:15:35.620 Right. 00:15:35.620 --> 00:15:41.700 If you're writing a stock trading application, the stock decision engine had better have a good bit of coverage on it. 00:15:41.700 --> 00:15:42.580 Yes. 00:15:42.580 --> 00:15:43.960 Or you failed with your test. 00:15:43.960 --> 00:15:44.140 Right. 00:15:44.140 --> 00:15:48.300 Testing the login like crazy doesn't help the core engine do anything better. 00:15:48.300 --> 00:15:48.520 Right. 00:15:48.520 --> 00:15:52.480 You could decide you're going to increase coverage on the parts you care about. 00:15:53.060 --> 00:15:55.940 And coverage doesn't really have any opinions about this. 00:15:55.940 --> 00:15:59.140 It's designed to just tell you something about your code. 00:15:59.140 --> 00:16:06.720 I've found over the years I am drawn to projects that are all about helping developers understand their world better. 00:16:06.720 --> 00:16:09.280 And coverage is one of those ways. 00:16:09.280 --> 00:16:09.520 Right. 00:16:09.520 --> 00:16:13.680 You wrote something and you wrote something to test it and you thought it was testing it. 00:16:13.680 --> 00:16:14.960 And, oh, is it? 00:16:14.960 --> 00:16:16.800 Well, coverage can tell you whether it is. 00:16:16.800 --> 00:16:17.040 Yeah. 00:16:17.040 --> 00:16:19.340 I really love that, the way it works. 00:16:19.340 --> 00:16:22.140 So I guess maybe we could talk a little bit about how you run it. 00:16:22.420 --> 00:16:24.980 Like, is this something you put in continuous integration? 00:16:24.980 --> 00:16:26.540 Is it a command line tool? 00:16:26.540 --> 00:16:27.900 Like, where does it work? 00:16:27.900 --> 00:16:28.220 Right. 00:16:28.220 --> 00:16:34.140 So the simplest thing is it is a command line tool and you can run it in your continuous integration on Travis or something. 00:16:34.140 --> 00:16:35.880 Or you can just run it from the command line. 00:16:35.880 --> 00:16:40.920 The design is that the coverage command has a number of subcommands, one of which is run. 00:16:40.920 --> 00:16:46.800 And when you type coverage run, anything you could put after the word Python, you could put after coverage run. 00:16:46.800 --> 00:16:53.000 So if you used to run your program by saying Python prog.py, then you can say coverage run prog.py. 00:16:53.120 --> 00:16:56.680 And it will run it the same way that Python would have run it, but under observation. 00:16:56.680 --> 00:16:59.020 That will collect a bunch of data. 00:16:59.020 --> 00:17:02.600 And then if you type coverage report, it will give you a report. 00:17:02.600 --> 00:17:14.160 And for every line, every file that got executed, it will tell you things like how many statements there are, how many got executed, how many didn't get executed, and therefore what percentage of them were executed. 00:17:14.600 --> 00:17:16.120 And then a big total at the end. 00:17:16.120 --> 00:17:17.320 Yeah, that's really cool. 00:17:17.320 --> 00:17:23.180 So I could say like coverage-m unit test in some file or something like this? 00:17:23.180 --> 00:17:25.100 Yeah, coverage run-m. 00:17:25.100 --> 00:17:26.300 Yeah, okay, gotcha. 00:17:26.300 --> 00:17:27.700 Right, dash-m unit test. 00:17:27.700 --> 00:17:29.500 Yes, I'll just leave it at that, yes. 00:17:29.500 --> 00:17:32.480 And there's also a pytest plugin, right? 00:17:32.480 --> 00:17:34.640 There are plugins for pytest and for Nose. 00:17:34.640 --> 00:17:44.580 And I'll be perfectly honest with you, I'm not a huge fan of the plugins because it's just another bunch of code between you and me, sort of. 00:17:44.900 --> 00:17:52.740 Like, I don't understand exactly how those plugins work, so it's hard for me to vouch for them doing what you want them to do. 00:17:52.740 --> 00:17:55.140 But yes, there are plugins for pytest. 00:17:55.140 --> 00:18:09.220 So there's pytest-Cov, and when you install it, you now have dash-dash-cov, a few, about half dozen dash-dash-cov options to pytest to say things like, I want you to only look at these modules, or I want you to produce this kind of report at the end. 00:18:09.420 --> 00:18:20.120 So it's much more convenient in that you don't, you can get all of the coverage behavior in one pytest run, rather than having coverage run pytest, and then coverage producing a report as two separate commands. 00:18:20.120 --> 00:18:26.840 But like I said, there's a trade-off there in that if the plugin isn't doing what you want, then it's a little bit trickier. 00:18:26.840 --> 00:18:31.720 Now you've got to figure out, is it the plugin, or is it coverage, or is it my script that's doing it? 00:18:31.720 --> 00:18:33.140 It's one more variable in there. 00:18:33.440 --> 00:18:34.960 Yeah, it just makes it a little more complex. 00:18:34.960 --> 00:18:35.720 Okay, interesting. 00:18:35.720 --> 00:18:41.100 Another place that I really like to run coverage.py is from PyCharm. 00:18:41.100 --> 00:18:44.560 I don't know if you've ever seen the integration there, but that is just incredible. 00:18:44.560 --> 00:18:50.040 You right-click on some part of your code and say, run this with coverage, and then you get a report in PyCharm. 00:18:50.040 --> 00:18:55.800 But PyCharm, the editor itself, actually colors each line based on the coverage, which I think is a really nice touch. 00:18:56.040 --> 00:19:01.360 Yeah, PyCharm is really an amazing IDE, which I don't actually use because I'm old. 00:19:01.360 --> 00:19:09.480 There's also a Vim plugin, I believe, to do similar things to get the coverage data and to display it to you in Vim and probably for Emacs as well. 00:19:09.480 --> 00:19:10.140 Oh, that's cool. 00:19:10.140 --> 00:19:17.260 Yeah, the more convenient we can make it for people to see the information, the better off it's going to be, the tighter feedback loop you've got. 00:19:17.260 --> 00:19:22.680 And right there in the editor is the best place to see it because that's where you're going to have to be dealing with the code anyway. 00:19:23.200 --> 00:19:26.440 You're not trying to correlate some report back to some file. 00:19:26.440 --> 00:19:28.500 You just look at it like, oh, why is that red? 00:19:28.500 --> 00:19:29.200 That should be green. 00:19:29.200 --> 00:19:29.880 What's going on here? 00:19:29.880 --> 00:19:30.280 Right, exactly. 00:19:30.280 --> 00:19:37.820 And coverage will produce HTML reports that are colored red and green and actually have a little bit of interaction if you want to focus in on things. 00:19:37.820 --> 00:19:44.580 But if other IDEs or whatever can produce displays that make it more convenient for people, then more power to them. 00:19:45.140 --> 00:19:53.860 Coverage has an API, Python API, which is, I assume, what PyCharm is built on, although they could be also just doing subprocess launches and things like that. 00:19:53.860 --> 00:20:01.740 So, yeah, I'm happy to have people get access to the power of that coverage measurement, however they're most happy with. 00:20:01.980 --> 00:20:03.040 Yeah, that's really great. 00:20:03.040 --> 00:20:15.960 So, one of the things that I kind of hinted at before with the core trading engine and certainly with, say, like your test code, you probably don't care about looking at the analysis of the coverage of your test code. 00:20:15.960 --> 00:20:18.900 You would like to see the analysis of your code under test. 00:20:19.320 --> 00:20:22.640 So, how do you exclude some bits of code from being analyzed? 00:20:22.640 --> 00:20:23.140 All right. 00:20:23.140 --> 00:20:26.900 So, I'll answer your question first, and then I will challenge the premise of the question. 00:20:26.900 --> 00:20:27.400 Okay. 00:20:27.400 --> 00:20:27.940 Sure. 00:20:27.940 --> 00:20:29.120 Good. 00:20:29.120 --> 00:20:37.980 So, coverage gives – there's a bunch of controls that you can get used with coverage to, say, basically to focus its attention on the code you want. 00:20:37.980 --> 00:20:47.420 If you run coverage just the way I started with, it will tell you about a lot more than what you care about because it's going to tell you about every library you've imported, even if it's not your code. 00:20:47.880 --> 00:20:59.800 And that's partly because that's the way it used to be and partly because it's hard to know what counts as a third-party library versus your code if you're running your code in an installed setting and so on and so forth. 00:20:59.800 --> 00:21:05.020 So, there are controls to focus coverage in on the code you want. 00:21:05.020 --> 00:21:07.280 The simplest is the source option. 00:21:07.280 --> 00:21:12.960 And the source option says, I'm only interested in any source code that you find from this tree downwards. 00:21:12.960 --> 00:21:16.960 And, for instance, source equals dot is often a very good choice. 00:21:16.960 --> 00:21:19.840 choice because it means I'm in the current working directory. 00:21:19.840 --> 00:21:20.640 Here's my code. 00:21:20.640 --> 00:21:22.880 Don't tell me about any code you find anywhere else. 00:21:22.880 --> 00:21:30.860 But even more than that, you can say things like omit these file paths or only include these file paths. 00:21:30.860 --> 00:21:33.720 So, there's a lot of controls there to focus coverage in. 00:21:34.340 --> 00:21:41.440 And that's because an automated tool is great unless you're constantly knowing more than the automated tool tells you. 00:21:41.440 --> 00:21:51.280 Like, if the automated tool is like a noisy kindergartner yammering at you and you have to tell it, keep thinking to yourself, shut up about that. 00:21:51.360 --> 00:21:53.840 I know that's not what I'm concerned about. 00:21:53.840 --> 00:21:56.360 Then the tool is not useful. 00:21:56.360 --> 00:21:56.720 Yeah. 00:21:59.100 --> 00:22:02.420 This portion of Talk Python To Me is brought to you by Brilliant.org. 00:22:02.420 --> 00:22:09.620 Many of you have come to software development and data science through paths that did not include a full-on computer science or mathematic degree. 00:22:09.620 --> 00:22:14.240 Yet, in our technical field, you may find you need to learn exactly these topics. 00:22:14.240 --> 00:22:15.900 You could go back to university. 00:22:15.900 --> 00:22:19.820 But then again, this is the 21st century and we do have the internet. 00:22:19.820 --> 00:22:24.520 Why not take some engaging online courses to quickly get just the skills that you need? 00:22:24.520 --> 00:22:26.740 That's where Brilliant.org comes in. 00:22:26.740 --> 00:22:29.200 They believe that effective learning is active. 00:22:29.200 --> 00:22:33.920 So master the concepts you need by solving fun, challenging problems yourself. 00:22:33.920 --> 00:22:35.560 Get started today. 00:22:35.560 --> 00:22:39.600 Just visit talkpython.fm/brilliant and sign up for free. 00:22:39.600 --> 00:22:40.780 And don't wait either. 00:22:40.780 --> 00:22:45.120 If you decide to upgrade to a paid account for guided courses and more practice exercises, 00:22:45.120 --> 00:22:51.560 the first 200 people that sign up from Talk Python will get an extra 20% off an annual premium subscription. 00:22:51.560 --> 00:22:54.600 That's talkpython.fm/brilliant. 00:22:55.880 --> 00:22:57.340 People will stop using it, right? 00:22:57.340 --> 00:23:00.760 Like if it becomes too noisy and too annoying, then you're like, well, it would have been helpful. 00:23:00.760 --> 00:23:02.560 But it's just so much noise. 00:23:02.560 --> 00:23:03.200 Forget this thing. 00:23:03.200 --> 00:23:03.440 Right. 00:23:03.440 --> 00:23:08.300 Or there's actually useful information in it that they can't see or whatever. 00:23:08.620 --> 00:23:19.280 So it shouldn't be that the way you use coverage is you run it, you get a report, and then you skip over all the stuff you know you don't care about to hopefully see the stuff you do care about. 00:23:19.280 --> 00:23:30.740 So there's a lot of controls and coverage to let you be the smart one in the room and let it be the savant about the one thing that it is smarter than you about, which is what code got run. 00:23:31.260 --> 00:23:43.220 So in addition to being able to exclude or include file paths and modules, you can actually put comments in your code to tell coverage, this line isn't run, but I don't care. 00:23:43.220 --> 00:23:45.100 Like, don't tell me about this line anymore. 00:23:45.100 --> 00:23:47.240 I'm okay with it not being run. 00:23:47.240 --> 00:23:47.560 Yeah. 00:23:47.640 --> 00:23:50.440 One of the examples you gave was the repper method, dunderrepper. 00:23:50.440 --> 00:23:51.300 Right. 00:23:51.300 --> 00:23:56.800 Do you really want to write a test that instantiates an object and just prints it just to get that function? 00:23:56.800 --> 00:23:57.060 Right. 00:23:57.060 --> 00:23:57.420 Like, no. 00:23:57.420 --> 00:24:03.660 And I've got lots of dunderreppers in the coverage.py code, and they're only ever run when I'm debugging coverage.py. 00:24:03.660 --> 00:24:06.160 I'm in a debugger, and I want to see what that is. 00:24:06.160 --> 00:24:09.260 You want to see the string representation better than this type at that address. 00:24:09.260 --> 00:24:10.000 That's right. 00:24:10.000 --> 00:24:17.040 So I write a dunderrepper, and I don't want to be told that it's not being executed in my test suite because I don't run it in my test suite. 00:24:17.220 --> 00:24:32.300 So – and in addition to putting comments on lines, what you can actually do with coverage is there's a coverage.rc file to configure coverage, and you can actually specify a list of regexes, and any line that matches one of those regexes will be excluded from coverage measurement. 00:24:32.300 --> 00:24:34.200 And that's how the comment works. 00:24:34.200 --> 00:24:37.440 The comment is a regex pattern by default in that setting. 00:24:37.440 --> 00:24:44.320 But, for instance, when I run coverage, I put, like, def space dunderrepper as one of the regexes. 00:24:44.640 --> 00:24:49.780 And that will match all my reppers, and then they'll all be excluded from coverage, and I don't need to worry about them anymore. 00:24:49.780 --> 00:24:50.800 Yeah, that makes a lot of sense. 00:24:50.800 --> 00:24:58.380 So you could put the comment hash, pragmical, and no cover, but you might not want that in your code all over the place. 00:24:58.380 --> 00:24:58.900 Right. 00:24:58.900 --> 00:25:01.120 You'd have to remember to put it in and et cetera. 00:25:01.120 --> 00:25:01.520 Yeah. 00:25:02.240 --> 00:25:04.760 People who don't care about it, they're like, why is this in this code? 00:25:04.760 --> 00:25:06.980 Or, you know, they write their code and they don't add it. 00:25:06.980 --> 00:25:09.460 Yeah, it's just – sometimes it's better to have it separate. 00:25:09.460 --> 00:25:09.720 Yeah. 00:25:09.720 --> 00:25:10.060 Right. 00:25:10.160 --> 00:25:23.820 And what I actually have done, because I'm a little obsessive about it in coverage.py itself, is I have a half dozen or ten different comment syntaxes that I'll use to exclude lines from coverage because they're being excluded for different reasons. 00:25:24.260 --> 00:25:34.040 Like a line that only runs on Jython, I will exclude from coverage because I'm not doing coverage measurement under Jython, even though I want to have a little bit of Jython support in the code. 00:25:34.040 --> 00:25:36.480 And so I'll have a comment that says only Jython. 00:25:36.480 --> 00:25:38.740 And then I know why it's been excluded. 00:25:38.740 --> 00:25:40.340 That's cool. 00:25:40.600 --> 00:25:44.540 Because I can just make a list of a dozen regexes and it's all there. 00:25:44.540 --> 00:25:46.020 You put it into that configuration file. 00:25:46.020 --> 00:25:46.320 Yeah. 00:25:46.320 --> 00:25:46.580 Yeah. 00:25:46.580 --> 00:25:47.800 Nice. 00:25:47.800 --> 00:25:56.420 Another thing that I thought was interesting was sort of the converse of this is to explicitly include some files. 00:25:56.420 --> 00:26:09.860 Because if you say, look here, and you run this one particular file, it only looks at the actual files that were loaded, the modules that were loaded, not the stuff laying next to it that maybe should have been reported on but nobody ever touched. 00:26:09.860 --> 00:26:10.340 Right. 00:26:10.420 --> 00:26:19.340 And that was one of the failings of early versions of coverage.py was that the only thing it knew about your code is code that got run. 00:26:19.340 --> 00:26:26.560 And so, for instance, if there was a particular file in your source tree that was never executed at all, I mean, forget about lines not being executed. 00:26:26.560 --> 00:26:28.160 The entire file was never executed. 00:26:28.160 --> 00:26:32.140 It wouldn't show up in the coverage report because coverage had never heard about it. 00:26:32.140 --> 00:26:33.060 Right. 00:26:33.060 --> 00:26:37.580 If you run a line of code in a file, coverage knows about the file and then it can see all the lines that weren't run. 00:26:37.680 --> 00:26:41.080 But if you never ran any of them, coverage never heard of the file. 00:26:41.080 --> 00:26:43.420 So that feature is now in coverage.py. 00:26:43.420 --> 00:26:52.500 If you give it a source option, then it has a tree to search and it can look for all of the importable Python files that it never heard of and tell you about those 0% files. 00:26:52.500 --> 00:26:52.860 Right. 00:26:52.860 --> 00:26:56.820 That's kind of like the example of me saying, here's this method that was never run. 00:26:56.820 --> 00:26:59.780 And, like, not even the module is imported in the main bit of code. 00:26:59.880 --> 00:27:00.000 Right. 00:27:00.000 --> 00:27:00.340 Right. 00:27:00.340 --> 00:27:00.900 Right. 00:27:00.900 --> 00:27:13.020 And, by the way, there's a really cheap, low-tech way to do file-level coverage, which is you delete all your .pyc files and then you exercise all your code. 00:27:13.020 --> 00:27:16.800 And any .py file that doesn't have a .pyc file was never imported. 00:27:16.800 --> 00:27:17.340 Oh, right. 00:27:17.340 --> 00:27:17.620 Yeah. 00:27:17.620 --> 00:27:17.860 Okay. 00:27:17.860 --> 00:27:18.660 That's pretty interesting. 00:27:18.660 --> 00:27:19.660 Very low-tech. 00:27:19.660 --> 00:27:21.320 It's quite low-tech. 00:27:21.320 --> 00:27:23.360 I forgot to challenge the premise of your earlier question. 00:27:23.360 --> 00:27:23.540 Yeah. 00:27:23.540 --> 00:27:23.700 Yeah. 00:27:23.700 --> 00:27:24.680 Let's get back to that. 00:27:24.880 --> 00:27:39.260 Yeah, so you said you don't want to do coverage measurement of your tests, but it actually can be very useful to do coverage measurement of your tests, if only because the way test runners work, it's really easy to make two tests that accidentally have the same name. 00:27:39.260 --> 00:27:41.700 You know, oh, I like that test. 00:27:41.700 --> 00:27:42.780 I want to do one kind of like it. 00:27:42.780 --> 00:27:43.720 I'll copy it and I'll paste it. 00:27:43.720 --> 00:27:44.820 I'll forget to change the name. 00:27:44.820 --> 00:27:47.400 And now I actually only have one test. 00:27:47.400 --> 00:27:51.360 If I look in the code, it looks like there's two, but there's really only one. 00:27:51.360 --> 00:27:51.940 Yeah, yeah. 00:27:51.940 --> 00:27:53.580 It's so easy to do that. 00:27:53.700 --> 00:27:58.480 And so if you do coverage measurement of your test files also, then you'll see those cases. 00:27:58.480 --> 00:27:59.140 Interesting. 00:27:59.140 --> 00:27:59.580 Okay. 00:27:59.580 --> 00:28:01.660 I guess, yeah, I never really thought about that. 00:28:01.660 --> 00:28:02.320 That's pretty valid. 00:28:02.320 --> 00:28:12.040 You know, I was thinking more of like you might say like ignore a particular test and you don't want that to say break the build because it drops it below some percentage or something like that. 00:28:12.040 --> 00:28:18.640 But yeah, that's a very – because I feel test code is probably some of the most copy and pasted code there is. 00:28:18.640 --> 00:28:19.120 Exactly. 00:28:19.120 --> 00:28:19.700 Exactly. 00:28:19.700 --> 00:28:22.780 And you never actually use the function name directly. 00:28:22.940 --> 00:28:24.140 So you'd never know. 00:28:24.140 --> 00:28:24.960 Right. 00:28:24.960 --> 00:28:26.900 The name of the test function doesn't matter. 00:28:26.900 --> 00:28:28.980 Yeah, exactly. 00:28:28.980 --> 00:28:30.820 It's really easy to lose a test. 00:28:30.820 --> 00:28:30.920 All right. 00:28:30.920 --> 00:28:33.340 I accept defeat on this one. 00:28:33.340 --> 00:28:34.600 No, that's a really good point. 00:28:34.600 --> 00:28:35.380 All right. 00:28:35.380 --> 00:28:36.260 Well, spur one for me. 00:28:36.260 --> 00:28:44.380 The other thing, when people say they want to exclude their tests from coverage, often it's because they've set a goal for their coverage measurement. 00:28:44.380 --> 00:28:46.440 Like we need 75% coverage. 00:28:47.200 --> 00:28:50.260 And those goals are really completely artificial. 00:28:50.260 --> 00:28:52.100 Like how did you choose 75%? 00:28:52.100 --> 00:28:55.500 Like why – like a quarter of your code doesn't need to be tested. 00:28:55.500 --> 00:28:56.920 Why is that okay, right? 00:28:56.920 --> 00:29:04.660 So the way I look at it, the number – the coverage number has no meaning except that lower numbers are worse. 00:29:04.660 --> 00:29:06.640 That's the only meaning to the number. 00:29:06.800 --> 00:29:11.300 So if someone says like how much coverage should I have, there's no right answer to that. 00:29:11.300 --> 00:29:11.820 Yeah. 00:29:12.120 --> 00:29:18.480 You know, I guess probably your test code has a pretty high coverage rate relative to your other codes. 00:29:18.480 --> 00:29:18.660 Yes. 00:29:18.660 --> 00:29:20.360 You're only helping yourself in that number. 00:29:20.360 --> 00:29:21.400 That's right. 00:29:21.400 --> 00:29:21.840 That's right. 00:29:21.840 --> 00:29:22.600 You can gain the system. 00:29:22.600 --> 00:29:23.700 Exactly. 00:29:23.700 --> 00:29:27.340 I guess one of the reasons I was thinking about excluding it is I just don't want to see it in the report. 00:29:27.340 --> 00:29:29.640 Like I don't need to see the test coverage. 00:29:29.640 --> 00:29:33.460 But your example of this copy and paste error actually is pretty valid, I think. 00:29:33.460 --> 00:29:39.640 Well, another option in the coverage.py reports is to exclude files that are 100%. 00:29:40.160 --> 00:29:41.020 Right. 00:29:41.020 --> 00:29:43.880 Which, again, lets you focus in on where the problems are. 00:29:43.880 --> 00:29:46.500 Like you don't need to think about files that have 100% coverage. 00:29:46.500 --> 00:29:52.140 What you need to think about is the ones that are missing some coverage because you need to go and look at those lines and write some tests for those files. 00:29:52.140 --> 00:29:58.920 So if all your tests have 100% coverage, then include them and exclude the 100% files from the report. 00:29:58.920 --> 00:29:59.240 Right. 00:29:59.240 --> 00:30:01.740 And hopefully that does a smaller list. 00:30:01.740 --> 00:30:02.140 Yeah. 00:30:02.140 --> 00:30:03.760 Yeah. 00:30:03.760 --> 00:30:04.080 Yeah. 00:30:04.080 --> 00:30:09.000 You can always use the pragma stuff on, say, your test code to say, 00:30:09.120 --> 00:30:12.460 Well, these three parts I'm having a hard time getting to run for whatever reason. 00:30:12.460 --> 00:30:14.800 And just, you know, tell it to not report on that. 00:30:14.800 --> 00:30:16.900 It'll hit it to 100 and it drops out of the list, right? 00:30:16.900 --> 00:30:17.160 Right. 00:30:17.160 --> 00:30:17.640 Exactly. 00:30:17.640 --> 00:30:18.220 Yeah. 00:30:18.260 --> 00:30:27.180 The other thing about test code I find is that in a full, mature test suite, you've got a significant amount of engineering happening in your test. 00:30:27.180 --> 00:30:30.800 If not in the tests themselves, in the helpers that you have written for your tests. 00:30:30.800 --> 00:30:31.340 Yeah. 00:30:31.680 --> 00:30:40.140 And not that there's code in there that isn't being run and should be run, but there might be code in there that isn't being run and therefore you can delete it. 00:30:40.140 --> 00:30:40.400 Right. 00:30:40.400 --> 00:30:40.600 Yeah. 00:30:40.600 --> 00:30:48.540 That same conversation of like, let's, I do feel like a lot of times people treat their test code with less, what's the right word? 00:30:48.600 --> 00:30:50.760 Kind of professionalism or attention. 00:30:50.760 --> 00:30:52.640 They're like, well, this is test code. 00:30:52.640 --> 00:30:57.080 So it doesn't matter that this big block of code was repeated 100 times. 00:30:57.080 --> 00:30:59.000 Why would I ever extract the method to that? 00:30:59.080 --> 00:31:02.700 Like there's just less attention to the architecture and patterns there. 00:31:02.700 --> 00:31:04.140 And I feel like that would help. 00:31:04.140 --> 00:31:04.480 Right. 00:31:04.480 --> 00:31:10.400 And so I agree with you that repetition should be removed from tests as it is from elsewhere. 00:31:10.400 --> 00:31:18.220 But I've also heard people feel passionately that tests should be repetitive, that each test should be readable all by itself. 00:31:18.220 --> 00:31:22.980 So I'll, you know, give them the benefit of the doubt that maybe that's what they want. 00:31:22.980 --> 00:31:24.720 And that's fine too. 00:31:24.720 --> 00:31:25.680 Yeah, sure. 00:31:25.700 --> 00:31:29.420 And if that's a conscious decision, then that's fine, I think. 00:31:29.420 --> 00:31:35.920 But a lot of people just do it because, well, they wrote one test and then they copied it and then they edited it and they copied it and they edited it. 00:31:35.920 --> 00:31:36.380 You know what I mean? 00:31:36.380 --> 00:31:37.180 Yeah, exactly. 00:31:37.180 --> 00:31:37.460 Right. 00:31:37.460 --> 00:31:40.820 And they don't have time to make the tests nice because who cares about the tests? 00:31:40.820 --> 00:31:41.540 Yeah, exactly. 00:31:41.540 --> 00:31:50.540 Until you change the thing under test and all of a sudden it's so hard to get it to run because you had poor decision making around writing the test. 00:31:50.540 --> 00:31:54.200 Then you claim unit testing is too hard because I don't want to do it and so on. 00:31:55.440 --> 00:32:01.080 Yeah, writing tests is real engineering with different problems than writing your product code. 00:32:01.080 --> 00:32:06.100 And those problems need to be paid attention to, which that's probably a whole other episode. 00:32:06.100 --> 00:32:19.140 Yeah, I've certainly heard people make statements like that they don't really understand, you know, things like object oriented programming and other proper design patterns until they started writing tests and trying to make their code more flexible. 00:32:19.140 --> 00:32:28.020 Like, how do I actually get in between the data access layer and my middle tier logic and test that without having a database and things like that? 00:32:28.080 --> 00:32:29.160 Yeah, they really make you think. 00:32:29.160 --> 00:32:32.100 Yeah, they do because it's a second use of your code. 00:32:32.100 --> 00:32:37.000 And your code is going to be way better designed if you consider more than one use of the code. 00:32:37.000 --> 00:32:38.080 Yeah, absolutely. 00:32:38.080 --> 00:32:39.980 Testability is a great topic. 00:32:39.980 --> 00:32:42.100 Yeah, I actually, yeah, I totally agree. 00:32:42.100 --> 00:32:47.660 And I think that, what you just said right there is one of the main reasons to test is the architecture that comes out of it. 00:32:48.220 --> 00:32:50.680 All right, so you spoke a little bit about these config files. 00:32:50.680 --> 00:32:55.980 And in these config files, I can put regular expressions, which will, you know, limit these sections. 00:32:55.980 --> 00:32:59.600 Oh, and one point I did want to make really quick about that. 00:32:59.600 --> 00:33:07.000 When you put one of those comments, if you put that on like a branch or some kind of structure, like a class, like everything underneath that will be blocked. 00:33:07.000 --> 00:33:09.240 You don't have to like put that on 100 lines, right? 00:33:09.240 --> 00:33:10.660 You put it at the sort of root node. 00:33:10.660 --> 00:33:11.400 Yeah, exactly. 00:33:11.400 --> 00:33:18.200 And it's, so it's basically if you put it on a line with a colon, then that entire clause is excluded. 00:33:18.200 --> 00:33:23.720 The one thing that sounds like it might be a generalization of that that doesn't work is to put one of those comments at the top of a file. 00:33:23.720 --> 00:33:25.540 It doesn't exclude the entire file. 00:33:25.540 --> 00:33:27.520 There's a request to do that. 00:33:27.520 --> 00:33:30.060 That seems like a good idea, but we've never, haven't gotten to that yet. 00:33:30.060 --> 00:33:30.640 Okay, cool. 00:33:30.640 --> 00:33:34.240 So, but back to config files, there's more stuff that you can put in there. 00:33:34.240 --> 00:33:38.740 Like what are useful things that are people are doing there beyond just exclusion? 00:33:38.740 --> 00:33:40.460 Yeah, so there's some basic options. 00:33:40.640 --> 00:33:46.700 Like one of the things that coverage.py can do is to measure not just which statements got executed, but which branches got taken. 00:33:46.700 --> 00:33:53.680 So, for instance, if you write an if statement, it can tell you whether both the true case and the false case were executed. 00:33:53.680 --> 00:34:01.540 And you might say, well, I can tell that by looking at the code in the true case and the code in the else clause, but not every if statement has an else clause. 00:34:01.860 --> 00:34:12.520 So, if you have an if with a statement in it, but there's no else clause, you can't tell just by looking at individual lines that are executed whether the false case of the if was executed. 00:34:12.520 --> 00:34:12.960 Right. 00:34:12.960 --> 00:34:15.320 Did you effectively skip that if block? 00:34:15.320 --> 00:34:15.880 Right. 00:34:15.960 --> 00:34:18.640 Have you ever actually skipped that statement in the if? 00:34:18.640 --> 00:34:19.340 Yeah. 00:34:19.340 --> 00:34:21.680 And so, and branch coverage is what can do that. 00:34:21.680 --> 00:34:25.820 And so, that's one of the options you can set in the config file as well as on the command line. 00:34:25.920 --> 00:34:39.100 I mean, basically, just to go back to the plugins question, the original reason I wrote support for the coverage RC was because I was adding features to coverage and people were using the test runner plugins as their UI. 00:34:39.320 --> 00:34:45.160 So, I didn't have any way to give them to try new features and coverage until the plugins were updated. 00:34:45.160 --> 00:34:46.080 I see. 00:34:46.080 --> 00:34:53.840 So, I added support for an RC file so you could control coverage even under a plugin from a thing that I could actually add features to. 00:34:53.840 --> 00:34:58.080 So, all of the features of coverage are controllable from the RC file. 00:34:58.080 --> 00:35:05.860 A big thing that gets put in there, one of the complicated scenarios for coverage is if you have tests that run in parallel. 00:35:06.560 --> 00:35:15.320 Either because you're running them on separate machines or you're just running them in separate processes or because you ran the 2.7 tests separately from the 3.6 tests. 00:35:15.320 --> 00:35:19.280 And then you want to take all that data and combine it back together to get one coverage report. 00:35:19.280 --> 00:35:25.840 And under those scenarios, there's often cases where, oh, on my CI system, the code was in this directory. 00:35:25.840 --> 00:35:30.100 But back home where I'm going to write the report, it's all in this directory. 00:35:30.100 --> 00:35:35.220 And coverage has to somehow know that those two directories are the same in some way, that they're equivalent. 00:35:35.860 --> 00:35:40.640 And those file path mappings go into the coverage RC file, for instance. 00:35:40.640 --> 00:35:41.200 I see. 00:35:41.200 --> 00:35:41.960 Okay, that's cool. 00:35:41.960 --> 00:35:47.020 So, that during the combination process, it can remap file names to get everything to make sense. 00:35:47.020 --> 00:35:48.060 Yeah, that makes a lot of sense. 00:35:48.060 --> 00:35:56.740 So, speaking of stuff in parallel, what's the story around like threading, async.io, multiprocessing, or even if I just want to start a subprocess? 00:35:56.740 --> 00:35:57.620 Yeah. 00:35:57.620 --> 00:35:58.680 Does it have support for that? 00:35:58.680 --> 00:36:00.060 Do I have to do things special for that? 00:36:00.200 --> 00:36:02.000 Yeah, so that gets complicated, too. 00:36:02.000 --> 00:36:08.300 So, there are four kinds of concurrency that coverage.py supports right out of the box. 00:36:08.300 --> 00:36:12.300 Threading, multiprocessing, gevent, and green. 00:36:12.300 --> 00:36:14.220 I forget which one of the other ones. 00:36:14.220 --> 00:36:14.980 I forget what it's called. 00:36:14.980 --> 00:36:22.240 And there's code in coverage.py that's specifically designed to see that that's happening and do the right thing. 00:36:22.340 --> 00:36:28.680 You have to tell coverage which one you're going to use, but once you do that in your coverage RC, it knows how to do that. 00:36:28.680 --> 00:36:41.880 If you're running your own subprocesses, it gets a little bit trickier because you've got Python code in your main process, and then you're going to spawn a subprocess, which is a whole new Python process that's sort of jumped outside of what coverage is watching. 00:36:41.880 --> 00:36:46.440 There's a little bit of support for coverage getting started on that subprocess. 00:36:46.920 --> 00:36:49.940 You have to do some manual setup, and that's covered in the docs. 00:36:49.940 --> 00:36:58.920 I would like to make that more automatic, but it feels a little intrusive to sort of have coverage start on every Python process you ever start in the future. 00:36:58.920 --> 00:37:05.380 I'd rather be conservative about that than suddenly be in the middle of something where people didn't expect it. 00:37:05.380 --> 00:37:08.800 So, we try to support all those different concurrencies. 00:37:09.380 --> 00:37:16.100 I just got a report that QT threads don't work, which doesn't surprise me because they're C threads, and so how do I get involved in that? 00:37:16.100 --> 00:37:18.580 Right, it doesn't carry that process across, yeah. 00:37:18.580 --> 00:37:19.940 Right, and it gets very fiddly. 00:37:19.940 --> 00:37:31.000 Some of the worst code in coverage.py is in the way we support those kinds of concurrencies because we need to get involved at the very beginning of a thread or the very beginning of a process, 00:37:31.000 --> 00:37:36.820 and there isn't always support for me to just say, hey, next time you start a process, why don't you run a little bit of my code first? 00:37:37.940 --> 00:37:39.380 No, that's not your process. 00:37:39.380 --> 00:37:42.140 Yeah, we do some very invasive things there. 00:37:42.140 --> 00:37:44.920 If you're interested, go and look at multiproc.py. 00:37:44.920 --> 00:37:46.020 Yeah, how interesting. 00:37:46.020 --> 00:37:51.540 This portion of Talk Python To Me is brought to you by Manning. 00:37:51.540 --> 00:37:58.860 Python has become one of the most essential programming languages in the world with a power, flexibility, and support that others can only dream of, 00:37:58.860 --> 00:38:01.360 but it can be tough learning it when you're just starting out. 00:38:01.360 --> 00:38:04.080 Luckily, there's an easy way to get involved. 00:38:04.600 --> 00:38:08.580 Written by MIT lecturer Annabelle and published by Manning Publications, 00:38:08.580 --> 00:38:12.720 Git Programming, learn to code with Python, is the perfect way to get started with Python. 00:38:12.720 --> 00:38:20.140 Anna's experience as a teacher of Python really shines through as you get hands-on with the language without being drowned in confusing jargon or theory. 00:38:20.720 --> 00:38:26.160 Filled with practical examples and step-by-step lessons, Git Programming is perfect for people who want to get started with Python. 00:38:26.160 --> 00:38:32.140 Take advantage of a Talk Python exclusive 40% discount on Git Programming by Annabelle. 00:38:32.140 --> 00:38:37.640 Just visit talkpython.fm/manning and use the code belltalkpy. 00:38:37.640 --> 00:38:40.440 That's belltalkpy, all one word. 00:38:41.840 --> 00:38:46.620 One thing that the documentation talks about is this thing called sitecustomize.py. 00:38:46.620 --> 00:38:47.700 Right, exactly. 00:38:47.700 --> 00:38:49.160 I had never even heard of this. 00:38:49.160 --> 00:38:50.480 You're like, oh, you just put it in here. 00:38:50.480 --> 00:38:51.820 I'm like, what is this? 00:38:51.820 --> 00:38:53.060 Tell them about this. 00:38:53.060 --> 00:38:56.100 It's all sort of the start of the initialization of Python itself, right? 00:38:56.100 --> 00:38:57.000 Right, exactly. 00:38:57.000 --> 00:39:04.940 So this is about how do, if you're going to run a Python program, how can I make it so that my code runs before your program even starts? 00:39:05.340 --> 00:39:12.200 Right, because if you're going to run a subprocess and it's a Python program, I want coverage to start before your code starts. 00:39:12.200 --> 00:39:18.200 One way to do that is for you to change your code so that instead of launching your Python program, you launch it with coverage. 00:39:18.200 --> 00:39:20.000 But that gets very invasive. 00:39:20.000 --> 00:39:21.120 No one wants to do that. 00:39:21.120 --> 00:39:26.420 You're not going to run your product by launching coverage, so your test would have to be different. 00:39:26.420 --> 00:39:26.800 Probably not. 00:39:26.800 --> 00:39:28.280 Product works. 00:39:28.280 --> 00:39:28.700 Yeah. 00:39:28.700 --> 00:39:34.640 So I looked around for ways to have code run before your main program. 00:39:34.640 --> 00:39:38.000 And there's no sort of built-in support for it. 00:39:38.000 --> 00:39:44.120 Perl actually has a command line switch that says run this program, but first run something else. 00:39:44.120 --> 00:39:45.460 Python doesn't. 00:39:45.460 --> 00:39:46.820 So there are two ways to do it. 00:39:46.820 --> 00:39:52.040 One is that when you run a program in Python, if it finds a file called site, customize.py, it will run it. 00:39:52.040 --> 00:39:54.860 And I don't know exactly what that file is for. 00:39:54.860 --> 00:39:59.680 It sounds like the kind of thing that would be recommended against these days. 00:39:59.680 --> 00:40:00.920 Seems weird. 00:40:00.920 --> 00:40:02.560 Like if you need something for your program, right? 00:40:02.560 --> 00:40:05.420 Don't put it into a weird file in your site packages. 00:40:05.420 --> 00:40:07.360 Somehow put that in your main program. 00:40:07.360 --> 00:40:08.640 But it's there. 00:40:08.640 --> 00:40:13.740 So one of the ways to get coverage to run before your subprocess starts is by putting something inside 00:40:13.740 --> 00:40:14.640 customize.py. 00:40:14.640 --> 00:40:19.640 And can you make that just like a file alongside your working directory, the top level of your 00:40:19.640 --> 00:40:20.040 working directory? 00:40:20.040 --> 00:40:21.080 Or has it got to go somewhere else? 00:40:21.180 --> 00:40:22.900 Honestly, I'm not quite sure. 00:40:22.900 --> 00:40:30.560 When I have to do this, I use the second technique for doing it because changing a file is scarier 00:40:30.560 --> 00:40:32.360 than creating a new file. 00:40:32.360 --> 00:40:38.100 And so the second way to do it only involves creating a new file, even though it's perhaps 00:40:38.100 --> 00:40:42.920 an even more obscure way to get code to run before the start of Python. 00:40:42.920 --> 00:40:44.960 Now, we don't have to go through all the details. 00:40:44.960 --> 00:40:50.300 But if you go into your site packages directory and you look for .pth files, path files, they 00:40:50.300 --> 00:40:56.900 have this very bizarre semantic, which is if a line in that file starts with the word import, 00:40:56.900 --> 00:41:01.140 then it will execute that line, even if the line has way more stuff than just an import 00:41:01.140 --> 00:41:01.480 statement. 00:41:01.480 --> 00:41:01.940 Weird. 00:41:01.940 --> 00:41:03.720 Yeah, it's super weird. 00:41:03.720 --> 00:41:04.440 It's super weird. 00:41:04.440 --> 00:41:05.400 And you can name it anything? 00:41:05.400 --> 00:41:06.060 Just like? 00:41:06.060 --> 00:41:07.340 Yeah, anything .pth. 00:41:07.340 --> 00:41:07.920 Yeah. 00:41:07.920 --> 00:41:11.680 This is all part of how site packages and the path gets set up. 00:41:11.680 --> 00:41:13.460 Just a little bit of backstory. 00:41:13.460 --> 00:41:17.820 Working at Coverage is fascinating because just programming technology is interesting, 00:41:17.820 --> 00:41:22.740 but you also get to discover all sorts of really weird, dark corners of the Python world. 00:41:22.740 --> 00:41:29.520 And I've gotten in the habit of every time a new alpha of CPython is announced, I get it 00:41:29.520 --> 00:41:33.220 as soon as I can, and I build it, and I run the Coverage test suite. 00:41:33.220 --> 00:41:38.280 And some of the core developers are used to, like, we better get the alpha out there so that 00:41:38.280 --> 00:41:40.400 Ned can run the Coverage test suite and tell us what we broke. 00:41:40.840 --> 00:41:44.700 So I think 3.6 RC3 was because of me. 00:41:44.700 --> 00:41:45.240 Oh, wow. 00:41:45.240 --> 00:41:47.840 Because of a bug I reported from the Coverage.py test suite. 00:41:47.840 --> 00:41:52.500 Yeah, there's a lot of weird, dark corners and a lot of kind of gross hacks to make things 00:41:52.500 --> 00:41:52.860 work. 00:41:52.860 --> 00:41:53.740 It's very difficult. 00:41:53.740 --> 00:41:56.520 So for instance, I said Coverage run works just like Python. 00:41:56.520 --> 00:42:02.200 Well, it's not easy to make a program that will run your Python program just the way Python 00:42:02.200 --> 00:42:03.320 runs your Python program. 00:42:03.540 --> 00:42:08.900 And there's some extensive tests in the Coverage.py test suite that actually do run the two side 00:42:08.900 --> 00:42:12.800 by side and then do a bunch of comparisons of the environment in which the code finds itself 00:42:12.800 --> 00:42:15.900 to try to assert that it does run it the same way. 00:42:15.900 --> 00:42:16.180 Yeah. 00:42:16.180 --> 00:42:19.060 I don't envy the job of keeping that compatibility working. 00:42:19.060 --> 00:42:20.440 Yeah. 00:42:20.440 --> 00:42:22.940 Well, luckily, we have a good test suite. 00:42:22.940 --> 00:42:23.400 Yeah. 00:42:23.400 --> 00:42:26.140 Well, I bet it has good code coverage as well. 00:42:26.260 --> 00:42:26.420 Yeah. 00:42:26.420 --> 00:42:31.100 Well, the sad thing is that there's code inside Coverage.py that because it's at the very 00:42:31.100 --> 00:42:35.260 heart of how coverage measurement gets done cannot itself be coverage measured. 00:42:35.260 --> 00:42:38.420 So my code coverage is at about 94%. 00:42:38.420 --> 00:42:39.460 That's still pretty good. 00:42:39.460 --> 00:42:40.660 But yeah, that's still pretty good. 00:42:40.660 --> 00:42:41.500 That's quite ironic. 00:42:41.500 --> 00:42:42.600 It is ironic. 00:42:42.600 --> 00:42:45.540 You'd think like me of all people, I'd have 100% test coverage. 00:42:45.540 --> 00:42:49.000 But and I've thought about really weird, hacky ways of trying to get at that stuff. 00:42:49.000 --> 00:42:50.060 But it's just not worth it. 00:42:50.060 --> 00:42:51.460 Yeah, I totally hear. 00:42:51.460 --> 00:42:55.040 So you spoke a little bit about the complexities and challenges of making it work. 00:42:55.040 --> 00:42:57.720 Let's dive in a little bit to how this actually works, right? 00:42:57.720 --> 00:42:58.460 Yeah. 00:42:58.460 --> 00:43:05.660 It seems like magic that I type your command line thing and then numbers all spit out of, 00:43:05.660 --> 00:43:07.940 well, here's exactly how your code ran. 00:43:07.940 --> 00:43:08.740 Like, how does that work? 00:43:08.740 --> 00:43:09.080 Yeah. 00:43:09.080 --> 00:43:13.540 So at the very heart of it is a feature of CPython called the trace function. 00:43:13.540 --> 00:43:16.820 So if you look in the sys module, there's a function called set trace. 00:43:16.820 --> 00:43:22.040 And you give it a function and it will call your function for every line of Python that gets 00:43:22.040 --> 00:43:22.460 executed. 00:43:23.460 --> 00:43:25.960 And that function can do whatever it wants. 00:43:25.960 --> 00:43:31.620 It is the basis for debuggers, for profilers, and for code measurement tools. 00:43:31.620 --> 00:43:36.560 And for some other things like tools that will let you run your program and have it just print 00:43:36.560 --> 00:43:37.940 every line of code that gets run. 00:43:37.940 --> 00:43:41.980 So you can sort of get a like a global log of everything that happened. 00:43:41.980 --> 00:43:42.280 Right. 00:43:42.280 --> 00:43:47.320 Some of those like sort of inspectors that will print out like an execution of your code as 00:43:47.320 --> 00:43:49.120 it runs, like the actual lines that are running. 00:43:49.120 --> 00:43:50.040 Yeah, those are pretty neat. 00:43:50.140 --> 00:43:50.580 Yeah, exactly. 00:43:50.580 --> 00:43:50.840 Yeah. 00:43:50.840 --> 00:43:52.780 Doug Hellman wrote a cool one called Smiley. 00:43:52.780 --> 00:43:53.380 Smiley. 00:43:53.380 --> 00:43:53.740 Okay. 00:43:53.740 --> 00:43:54.540 It does some cool things. 00:43:54.540 --> 00:43:54.840 Yeah. 00:43:54.840 --> 00:43:56.020 It spies on your code. 00:43:56.020 --> 00:43:59.320 So at the heart of it is that trace function. 00:43:59.320 --> 00:44:03.680 And for instance, if you've ever had to break into a debugger from your code and you type 00:44:03.680 --> 00:44:08.620 import PDB, PDB.setTrace, the reason it's called .setTrace instead of what it should be called, 00:44:08.620 --> 00:44:15.740 you know, break into the debugger, is because that's the function where PDB calls setTrace to get its trace 00:44:15.740 --> 00:44:20.060 function in place so that it will get told as lines get executed and then you're debugging. 00:44:20.060 --> 00:44:22.760 So that's the core CPython feature. 00:44:22.760 --> 00:44:28.620 And you can go and write, you know, the 10 lines of code that do interesting stuff with the trace function. 00:44:28.620 --> 00:44:29.640 It's actually pretty simple. 00:44:29.640 --> 00:44:32.620 Coverage.py has way more lines of code than that. 00:44:32.620 --> 00:44:35.140 But, you know, at its heart, that's what it's doing. 00:44:35.340 --> 00:44:38.060 It sets a trace function and then it runs your code. 00:44:38.060 --> 00:44:44.300 And as its trace function gets invoked with, you know, this line got executed, this line get executed, etc., etc., 00:44:44.300 --> 00:44:47.340 it records all that data and dumps it into a data file. 00:44:47.340 --> 00:44:50.800 And that's the run and out phase of Coverage.py. 00:44:50.800 --> 00:44:54.840 All that raw data then gets picked up when you say Coverage report. 00:44:54.840 --> 00:44:56.200 It reads that data. 00:44:56.200 --> 00:45:00.560 It looks at your source to analyze the source to figure out what could have been run. 00:45:00.560 --> 00:45:03.160 Right in that first phase, all we hear is what got run. 00:45:03.380 --> 00:45:06.300 The analysis phase is let's look at the source code. 00:45:06.300 --> 00:45:07.340 How many lines are there? 00:45:07.340 --> 00:45:08.180 What could get run? 00:45:08.180 --> 00:45:11.640 And then essentially it just does a big set subtraction. 00:45:11.640 --> 00:45:13.000 Here's the lines that could have been run. 00:45:13.000 --> 00:45:14.920 There are the lines that did get run. 00:45:14.920 --> 00:45:17.200 What's left over are the lines that didn't get run. 00:45:17.200 --> 00:45:18.180 I mean, conceptually. 00:45:18.180 --> 00:45:27.820 One thing that you did talk about in how it works is in the documentation is that you actually use the PYC files as part of this, right? 00:45:27.820 --> 00:45:32.180 You've got to look in there for the line numbers and stuff, which is kind of interesting. 00:45:32.180 --> 00:45:42.340 And that's one of the things that's actually changed a number of times over the course of Coverage.py's life is, well, how do we know what could have been executed? 00:45:42.340 --> 00:45:46.760 The question of what did get executed is kind of straightforward because we get it from the trace function. 00:45:46.760 --> 00:45:48.000 There's not much we can do there. 00:45:48.000 --> 00:45:50.040 I mean, there's a lot to do there. 00:45:50.040 --> 00:45:53.160 But there's essentially only one way to know what got run. 00:45:53.160 --> 00:45:56.620 Figuring out what could have been run, there's a bunch of different ways. 00:45:56.620 --> 00:46:05.420 As you mentioned, .pyc files compiled into them are a line number table that tells Python which lines are executable. 00:46:05.420 --> 00:46:10.060 And that's how Python actually decides when a next line has been run. 00:46:10.060 --> 00:46:12.040 The trace function, it's executing bytecodes. 00:46:12.040 --> 00:46:17.080 It calls the trace function when the line number table says this bytecode is the first bytecode on a new line. 00:46:17.080 --> 00:46:28.580 Right, because if I write like print, you know, quote, ned, comma, quote, batch elder, that actually is like multiple steps of the instruction. 00:46:28.580 --> 00:46:36.380 If I disassembled that, that would be like load the first string onto the stack, load the second string, execute the function, get the return value, all that type of stuff, right? 00:46:36.440 --> 00:46:36.860 That's right. 00:46:36.860 --> 00:46:39.080 So the trace function only gets called for lines. 00:46:39.080 --> 00:46:43.840 And so we're looking at the same information the Python interpreter is for deciding where the lines are. 00:46:43.840 --> 00:46:48.740 It gets more involved in that because, for instance, for branch coverage, we have to decide where the branches are. 00:46:48.740 --> 00:46:52.560 And we do that by analyzing the abstract syntax tree. 00:46:52.560 --> 00:46:58.960 We used to do it by analyzing the bytecode back when I thought, oh, well, the bytecode will have all the ifs and jumps I need. 00:46:58.960 --> 00:47:00.820 But it doesn't, and it never did. 00:47:01.360 --> 00:47:06.380 And when the async keywords got added, they were completely different than everything else. 00:47:06.380 --> 00:47:09.220 And I said, OK, fine, we're getting rid of all this bytecode analysis. 00:47:09.220 --> 00:47:13.320 And I'm going to do an AST analysis because I understand the AST when I don't understand the bytecode. 00:47:13.320 --> 00:47:13.820 All right. 00:47:13.820 --> 00:47:15.860 So it does support async and await? 00:47:15.860 --> 00:47:16.220 Yeah. 00:47:16.220 --> 00:47:17.420 You know, I have to be honest. 00:47:17.420 --> 00:47:20.380 When that first came out, I didn't know anything about async and await. 00:47:20.380 --> 00:47:23.400 I didn't understand how to write programs that use them. 00:47:24.060 --> 00:47:31.780 And I sort of took the low-tech, lazy maintainers approach, which is I'll wait for the bug reports to come in about how it doesn't work for those things. 00:47:31.780 --> 00:47:33.500 And no one has ever said it doesn't work. 00:47:33.500 --> 00:47:34.400 So I guess it works. 00:47:34.400 --> 00:47:35.220 Yeah. 00:47:35.220 --> 00:47:35.600 All right. 00:47:35.600 --> 00:47:36.440 That'll do it. 00:47:36.440 --> 00:47:37.120 Yeah. 00:47:37.120 --> 00:47:37.920 Cool. 00:47:37.920 --> 00:47:46.100 So I guess one question is calling functions in Python is not one of the fastest things that Python does relative to, say, other languages. 00:47:46.240 --> 00:47:52.500 So if you're calling this set trace function on every single line, how does that not just stop the execution, basically? 00:47:52.500 --> 00:47:52.760 Yeah. 00:47:52.760 --> 00:48:01.820 So, well, so the really simple answer is that if you write a Python function and set it as your trace function, then, yeah, you're going to have a significant performance hit. 00:48:01.820 --> 00:48:05.840 Because it's going to call a Python function for every line of your code. 00:48:05.840 --> 00:48:11.900 And if that Python function does anything interesting at all, then it's going to do a lot of work for every line of your code. 00:48:11.900 --> 00:48:15.380 Inside Coverage, we actually have two implementations of the trace function. 00:48:15.380 --> 00:48:17.100 One in Python and one in C. 00:48:17.100 --> 00:48:19.980 And the one that's in C is there to make it go faster. 00:48:19.980 --> 00:48:25.860 So there's a bunch of work to try to do as little work as possible on each call. 00:48:25.860 --> 00:48:37.040 So, for instance, when we get told that we're calling a new function, we do a bunch of work to figure out is this function or is this file, rather, a file that we're interested in at all. 00:48:37.040 --> 00:48:39.680 Have you already figured out how much coverage it has? 00:48:39.680 --> 00:48:41.700 Like if it's 100, right, then don't do it again. 00:48:41.700 --> 00:48:42.100 Well, we don't know. 00:48:42.100 --> 00:48:44.140 At this point, we're in the run phase. 00:48:44.260 --> 00:48:46.420 We have no idea about numbers like that. 00:48:46.420 --> 00:48:48.860 All we know is this file is getting executed. 00:48:49.260 --> 00:48:53.100 Is our current configuration telling us that that file is interesting or it's not interesting? 00:48:53.100 --> 00:49:03.620 And if it's not interesting, then we try to quickly get out of that function and set some bookkeeping so that the function doesn't get called again until we return from the function, for instance. 00:49:03.820 --> 00:49:07.940 So there's a lot of work to try to make the trace function as fast as possible. 00:49:07.940 --> 00:49:13.060 Because especially for mature test suites for the types of people that are running coverage. 00:49:13.820 --> 00:49:18.600 The one thing I can tell you about all of those test suites is that developers wish they ran faster. 00:49:18.600 --> 00:49:20.240 Like I don't need to know anything about your test suite. 00:49:20.240 --> 00:49:23.000 If you've got a lot of tests, you want them to run faster. 00:49:23.000 --> 00:49:26.700 So the last thing I want to do is make them go even slower. 00:49:27.220 --> 00:49:35.180 And I don't know what the typical overhead is of coverage on a test suite because it's going to vary so much depending on what else those tests are doing. 00:49:35.180 --> 00:49:38.540 But the C code is there to try to make it go as fast as possible. 00:49:38.540 --> 00:49:39.020 I see. 00:49:39.020 --> 00:49:40.140 Yeah, that makes a lot of sense. 00:49:40.280 --> 00:49:45.440 The pure Python implementation is there for, for instance, PyPy, which can't run C extensions. 00:49:45.440 --> 00:49:45.980 Right. 00:49:45.980 --> 00:49:52.200 But after a while, it'll start to speed up as they maybe consider it has to be jitted after like a thousand lines or something. 00:49:52.200 --> 00:49:53.120 I guess so. 00:49:53.120 --> 00:49:54.200 I haven't. 00:49:54.200 --> 00:49:54.680 I don't know. 00:49:54.680 --> 00:49:57.920 PyPy is still under the magic category for me. 00:49:57.920 --> 00:50:02.500 And I don't know whether a trace function will actually get the jit applied. 00:50:02.500 --> 00:50:07.860 That's the thing I am used to is that a trace function is not run like regular code. 00:50:08.360 --> 00:50:13.980 And I don't know if PyPy can jit a trace function, for instance. 00:50:13.980 --> 00:50:15.720 I have no idea either. 00:50:15.720 --> 00:50:16.800 Yeah, I don't know. 00:50:16.800 --> 00:50:17.480 Interesting. 00:50:17.480 --> 00:50:20.340 There are a few speed ups in there specifically for PyPy. 00:50:20.340 --> 00:50:25.340 Basically, when a PyPy developer tells me, hey, why don't you say this magic thing that you don't understand? 00:50:25.340 --> 00:50:26.620 And PyPy will go a little bit faster. 00:50:26.620 --> 00:50:27.280 And I'm like, great. 00:50:27.280 --> 00:50:28.080 Sure, I'll put it in. 00:50:28.080 --> 00:50:32.220 Yeah, PyPy is interesting. 00:50:32.220 --> 00:50:36.820 So I guess, you know, that's a good segue over to which runtimes are supported. 00:50:36.820 --> 00:50:40.400 Like, you mentioned jiton, you mentioned PyPy, obviously cpython. 00:50:40.400 --> 00:50:42.420 What versions of cpython? 00:50:42.420 --> 00:50:42.740 What else? 00:50:42.740 --> 00:50:47.880 Yeah, so the current released version of coverage.py is 4.5.1. 00:50:47.880 --> 00:50:53.800 And that supports Python 2.6 and Python 3.3 and up. 00:50:53.800 --> 00:50:57.920 So 2627, 33, 34, 35, 36, 37. 00:50:58.340 --> 00:51:03.120 The tip of the repo now, I'm already working on coverage 5.0 alphas. 00:51:03.120 --> 00:51:06.360 And I've dropped support for 2.6 and for 3.3. 00:51:06.360 --> 00:51:09.020 But the released version can do 2.6. 00:51:09.020 --> 00:51:16.040 For a while, I was very proud of supporting 2.3 through 3.3, which is kind of an interesting trick. 00:51:16.680 --> 00:51:18.100 But I don't have to do that anymore. 00:51:18.100 --> 00:51:21.040 So PyPy, both 2 and 3, are supported. 00:51:21.040 --> 00:51:24.080 I have run tests on jiton and IronPython. 00:51:24.080 --> 00:51:28.960 They can't do all of it because they can't do the analysis phase because they don't have AST. 00:51:28.960 --> 00:51:33.960 But you can run your code under jiton and IronPython for the measurement phase. 00:51:34.200 --> 00:51:36.860 And then you use cpython to do the analysis phase. 00:51:36.860 --> 00:51:37.320 I see. 00:51:37.320 --> 00:51:38.700 I don't know how well that works. 00:51:38.700 --> 00:51:40.280 I haven't heard a lot from people about it. 00:51:40.280 --> 00:51:44.080 I don't know how much those jiton and IronPython are getting used these days. 00:51:44.080 --> 00:51:45.740 There's another one called python.net. 00:51:45.740 --> 00:51:47.600 I actually haven't used it for anything. 00:51:47.600 --> 00:51:49.460 But I think it's a successor type of thing. 00:51:49.460 --> 00:51:51.660 A little bit different than IronPython, but similar. 00:51:51.660 --> 00:51:52.280 Is it? 00:51:52.280 --> 00:51:53.600 I need to look into this. 00:51:53.600 --> 00:51:55.480 I need more complication in my life. 00:51:55.480 --> 00:51:58.520 Yeah, I haven't done a lot with it. 00:51:58.520 --> 00:52:04.660 But python.net seems like a more maintained, different take on what IronPython was. 00:52:04.660 --> 00:52:07.040 But I pretty much have exhausted my knowledge of it now. 00:52:07.040 --> 00:52:07.400 All right. 00:52:07.400 --> 00:52:07.920 Cool. 00:52:07.920 --> 00:52:11.980 My question earlier was, what are your plans on supporting old versions? 00:52:11.980 --> 00:52:17.120 So if cpython stops supporting 2.7, are you going to stop supporting it? 00:52:17.120 --> 00:52:19.920 Yeah, that's certainly going to happen soon enough. 00:52:19.920 --> 00:52:23.120 Or say 3.4, what's the timing? 00:52:23.120 --> 00:52:24.980 Do you try to stay farther behind? 00:52:24.980 --> 00:52:26.620 What's the thing? 00:52:26.620 --> 00:52:29.320 Like I said at the top of the show, I'm pretty inertial. 00:52:29.320 --> 00:52:31.700 So I tend to support things for longer. 00:52:31.700 --> 00:52:37.760 So my philosophy about coverage.py development is that I want coverage.py to be available 00:52:37.760 --> 00:52:42.580 for anyone who's trying to make their code work better for Python. 00:52:42.580 --> 00:52:46.080 And the environments they care about, I will care about. 00:52:46.080 --> 00:52:54.080 So for instance, I first ported coverage.py to 3.0 in 2009, which was pretty early. 00:52:54.080 --> 00:52:55.740 And it was pretty rough around the edges. 00:52:55.740 --> 00:52:59.880 But I thought, you know, if people are going to start porting their libraries to Python 3, 00:52:59.880 --> 00:53:04.180 it'd be super useful if coverage.py were there to help them understand how their tests were doing. 00:53:04.180 --> 00:53:06.380 So I tried very hard to get ahead of it. 00:53:06.380 --> 00:53:10.340 And for that reason also, coverage.py has no installation prerequisites. 00:53:10.340 --> 00:53:17.060 Because if I prerequire a library, then that library has to get into a new environment before coverage.py can. 00:53:17.720 --> 00:53:19.380 And then we've got a chicken and egg problem. 00:53:19.380 --> 00:53:20.240 Yeah. 00:53:20.360 --> 00:53:25.200 It also gives me a principled reason to have written my own HTML template engine, which is fun. 00:53:26.320 --> 00:53:28.920 That sounds like a pretty challenging thing. 00:53:28.920 --> 00:53:30.400 Also challenging. 00:53:30.400 --> 00:53:36.060 Another runtime, if you want to call it that, I guess, that seems interesting is Cython. 00:53:36.060 --> 00:53:37.800 Does this have any... 00:53:37.800 --> 00:53:40.420 But it seems to me my first guess is it wouldn't work with Cython. 00:53:40.420 --> 00:53:41.780 But what's the story there? 00:53:41.780 --> 00:53:43.200 I should really know more about Cython. 00:53:43.200 --> 00:53:47.320 And in fact, it's been proposed to me that instead of writing C code for the trace function, 00:53:47.320 --> 00:53:54.180 I should write Cython for the trace function, which I've just been resistant to because, like I said, I'm inertial. 00:53:54.180 --> 00:53:57.540 So my understanding of Cython is that it compiles to C code. 00:53:57.540 --> 00:53:59.520 And then when you're running, you're running C code. 00:53:59.520 --> 00:54:04.820 So you need a C coverage measurement tool in order to understand what's happening there. 00:54:04.820 --> 00:54:05.120 Yeah. 00:54:05.120 --> 00:54:10.400 It compiles to a C file, which then compiles, like at least on Mac, to a .so file. 00:54:10.400 --> 00:54:12.040 And the .so file is running. 00:54:12.040 --> 00:54:12.340 Yeah. 00:54:12.340 --> 00:54:12.820 Right. 00:54:12.820 --> 00:54:15.160 Coverage.py won't be able to see any of that execution. 00:54:15.160 --> 00:54:17.800 A C coverage tool could. 00:54:17.800 --> 00:54:24.120 But then you'd have to figure out how to map that back to the Cython code that you're actually interested in. 00:54:24.120 --> 00:54:25.940 Which actually brings up an interesting point. 00:54:25.940 --> 00:54:39.540 One of the things that coverage.py has gotten in its 4.0 releases is plugins so that you can do coverage measurement of things that are not directly Python but result in Python. 00:54:39.540 --> 00:54:46.780 So, for instance, there's a Django coverage plugin that can measure the coverage of your Django templates. 00:54:46.780 --> 00:54:51.520 Because Django templates themselves have if statements and for loops in them. 00:54:51.520 --> 00:54:53.260 So that's code. 00:54:53.260 --> 00:54:55.580 You'd like to know if there's coverage there. 00:54:55.580 --> 00:55:04.580 And so there's a plugin, the Django coverage plugin, that works with coverage.py to basically understand how Django executes the templates. 00:55:04.580 --> 00:55:12.980 And take the raw Python information about the template execution and map it back to the lines in the Django template. 00:55:12.980 --> 00:55:18.300 So that you can get sort of a red-green Christmas tree of your Django template. 00:55:18.300 --> 00:55:18.860 Oh, that's cool. 00:55:19.020 --> 00:55:19.800 Yeah, it's very cool. 00:55:19.800 --> 00:55:24.240 And it was getting a little stale, but I've just gotten a new maintainer to take it over. 00:55:24.240 --> 00:55:26.880 And so they just had a release today, actually. 00:55:26.880 --> 00:55:27.640 Oh, great. 00:55:27.640 --> 00:55:27.880 Yeah. 00:55:27.880 --> 00:55:28.260 All right. 00:55:28.260 --> 00:55:28.760 That's interesting. 00:55:28.760 --> 00:55:33.400 It seems to me like there's probably enough information there, but that it's really not super straightforward. 00:55:33.400 --> 00:55:34.620 For Cython. 00:55:34.620 --> 00:55:36.820 But maybe plugins would do it. 00:55:36.980 --> 00:55:37.060 Yeah. 00:55:37.060 --> 00:55:41.520 I mean, the C file is a huge, massive thing relative to the size of the Python code. 00:55:41.520 --> 00:55:44.640 But I do believe it has line numbers that map back to the Python. 00:55:44.640 --> 00:55:46.680 So it's possible, I guess. 00:55:46.680 --> 00:55:51.460 The reason that reminded me of the plugins was because I wanted to also make a Mako plugin. 00:55:51.460 --> 00:55:53.760 So Mako is another HTML templating engine. 00:55:53.760 --> 00:55:56.860 And Mako and Django work completely differently. 00:55:56.860 --> 00:56:01.340 Mako actually compiles your template to a Python file and then runs the Python file. 00:56:01.500 --> 00:56:06.940 Whereas Django templates, there's Python code that's actually running on an abstract syntax 00:56:06.940 --> 00:56:07.880 tree of the template. 00:56:07.880 --> 00:56:08.540 It's sort of. 00:56:08.540 --> 00:56:12.920 So in that sense, Mako is kind of compiled to Python and Django is kind of interpreted in 00:56:12.920 --> 00:56:13.260 Python. 00:56:13.260 --> 00:56:16.180 And so the plugin technologies had to be completely different. 00:56:16.180 --> 00:56:23.300 And the Mako template has pointers back to the lines in the template, but they're a little 00:56:23.300 --> 00:56:23.960 bit inaccurate. 00:56:23.960 --> 00:56:29.880 And so I was frustrated at not being able to sort of close that loop because of a limitation 00:56:29.880 --> 00:56:30.500 in Mako. 00:56:30.740 --> 00:56:33.500 And Mako seems kind of unmaintained at this point. 00:56:33.500 --> 00:56:34.680 So it kind of didn't go anywhere. 00:56:34.680 --> 00:56:35.040 Yeah. 00:56:35.040 --> 00:56:35.500 Yeah. 00:56:35.500 --> 00:56:35.820 Interesting. 00:56:35.820 --> 00:56:37.100 It's cool that those are there, though. 00:56:37.100 --> 00:56:38.300 People want to use them. 00:56:38.300 --> 00:56:38.700 Yeah. 00:56:38.700 --> 00:56:39.180 Yeah. 00:56:39.180 --> 00:56:40.100 It's very gratifying. 00:56:40.100 --> 00:56:40.500 Yeah. 00:56:40.500 --> 00:56:44.000 So last question before we run out of time, because that's where we are. 00:56:44.000 --> 00:56:44.240 OK. 00:56:44.240 --> 00:56:44.880 What's next? 00:56:44.880 --> 00:56:45.860 What are the upcoming features? 00:56:45.860 --> 00:56:46.260 Right. 00:56:46.260 --> 00:56:48.100 So I mentioned that I've got some alpha. 00:56:48.100 --> 00:56:50.100 I've already released 5.0 alpha 2. 00:56:50.100 --> 00:56:56.800 The big feature that's coming up is something that I colloquially call who tests what, which is 00:56:56.800 --> 00:57:00.700 instead of just telling me that that line got run, tell me which tests actually 00:57:00.700 --> 00:57:01.380 ran that line. 00:57:01.380 --> 00:57:04.080 And people are interested in this for all sorts of reasons. 00:57:04.880 --> 00:57:09.060 But it can require some significant changes to the core of coverage.py. 00:57:09.060 --> 00:57:14.040 It's going to present some challenges in that if you have 1,000 tests, then you have to collect 00:57:14.040 --> 00:57:18.940 roughly 1,000 times as much data now because you're essentially – it's as much data as 00:57:18.940 --> 00:57:21.860 if you did a separate coverage run for every single one of your tests. 00:57:22.560 --> 00:57:26.940 And there are some tools out there that kind of do this already by doing exactly that, running 00:57:26.940 --> 00:57:29.920 coverage independently around each test. 00:57:29.920 --> 00:57:32.420 But coverage will do it differently because it's all bundled together. 00:57:32.420 --> 00:57:38.440 So there's an alpha out already now, alpha 2, which has switched the data format from a 00:57:38.440 --> 00:57:39.960 JSON file to a SQLite database. 00:57:40.780 --> 00:57:45.080 And I needed the SQLite database because if I'm going to dump 1,000 times more data, I want to 00:57:45.080 --> 00:57:46.180 dribble it into a database. 00:57:46.180 --> 00:57:49.340 And I want to give you a database you can query because I don't know how people are going to use 00:57:49.340 --> 00:57:49.740 the data. 00:57:49.740 --> 00:57:51.480 Do indexes on it and all that stuff. 00:57:51.480 --> 00:57:56.260 I have no idea how to report on this data because I'm not going to make an HTML file that for every 00:57:56.260 --> 00:58:00.300 line of your source code has the names of the 200 tests that ran it. 00:58:00.300 --> 00:58:00.920 Yeah. 00:58:01.160 --> 00:58:04.360 So we need to get this out there and into people's hands and start seeing what people 00:58:04.360 --> 00:58:04.820 do with it. 00:58:04.820 --> 00:58:06.700 That's the big feature that's coming up. 00:58:06.700 --> 00:58:07.100 That's cool. 00:58:07.100 --> 00:58:11.000 One of the most interesting things I think you could do with that, just listening to you 00:58:11.000 --> 00:58:15.680 describe it, would be if I have 1,000 tests and I change a little bit of my Python code and 00:58:15.680 --> 00:58:21.300 only three of those tests should actually interact with those lines, theoretically, I could just 00:58:21.300 --> 00:58:26.380 run those three tests to retest it, not all 1,000, which would be dramatically awesome. 00:58:26.380 --> 00:58:26.760 Right. 00:58:26.760 --> 00:58:27.180 Exactly. 00:58:27.180 --> 00:58:30.860 There are also some tools out there that do that now, too. 00:58:30.860 --> 00:58:36.120 Kirk Strausser has one, and the name of the tool escapes me at the moment, but he's got 00:58:36.120 --> 00:58:41.280 his own trace function that essentially does a mini coverage measurement of his own to get 00:58:41.280 --> 00:58:42.100 at that information. 00:58:42.100 --> 00:58:48.320 The thing I'd be interested to experiment with that information is, what if my code is over 00:58:48.320 --> 00:58:48.660 tested? 00:58:48.660 --> 00:58:55.480 If my test suite is taking too long, maybe that's because I've got 100 tests all getting 00:58:55.480 --> 00:58:58.900 at the same information, and if I could reduce it to 10 tests, then it would take a tenth 00:58:58.900 --> 00:58:59.440 of the time. 00:59:00.080 --> 00:59:04.980 And I honestly have no idea whether who tests what will give me the information that I'd 00:59:04.980 --> 00:59:07.060 need for that, but it'd be interesting to play around with. 00:59:07.060 --> 00:59:08.120 Yeah, it sounds really positive. 00:59:08.120 --> 00:59:10.640 Yeah, I'm looking forward to having the data, at least. 00:59:10.640 --> 00:59:11.400 Yeah, cool. 00:59:11.400 --> 00:59:15.300 All right, let me ask you the two final questions before you get out of here. 00:59:15.300 --> 00:59:15.980 Okay. 00:59:15.980 --> 00:59:17.520 You hinted at this at the beginning. 00:59:17.520 --> 00:59:20.220 If you're going to write some Python code, what editor do you use? 00:59:20.420 --> 00:59:20.920 I use Vim. 00:59:20.920 --> 00:59:26.620 I mentioned earlier that I am old, and I do not use Vim because I'm old. 00:59:26.620 --> 00:59:28.460 I've only been using Vim for about 10 years. 00:59:28.460 --> 00:59:34.360 So I got to it late in life, but it really suits my low-tech mentality, I think. 00:59:34.360 --> 00:59:34.860 Beautiful. 00:59:34.860 --> 00:59:44.640 And beyond coverage.py, what is a notable Python PyPI package that maybe people haven't heard 00:59:44.640 --> 00:59:46.480 of being like, this is awesome, you should know about this? 00:59:46.480 --> 00:59:47.920 Yeah, so I'll tell you the one. 00:59:47.920 --> 00:59:51.080 So this is the very last thing that happened to me at PyCon this year. 00:59:51.080 --> 00:59:55.760 I was literally almost dragging my suitcase out of the convention center to go catch a 00:59:55.760 --> 01:00:02.740 plane when I stopped by the PyLint sprint and heard about a package called checkmanifest. 01:00:03.260 --> 01:00:08.760 And it only does one little thing, but it's a thing that no one really cares about well 01:00:08.760 --> 01:00:09.840 enough to get it right themselves. 01:00:09.840 --> 01:00:14.880 So it's great to have a helper, which is it tells you whether the manifest.in that you wrote 01:00:14.880 --> 01:00:19.380 for your setup.py has all of the files from your working tree or not. 01:00:19.380 --> 01:00:19.780 Okay. 01:00:19.780 --> 01:00:24.180 So really sort of a check on your package before you ship it off. 01:00:24.180 --> 01:00:24.720 Exactly. 01:00:24.720 --> 01:00:27.760 And packaging is one of those things that everyone hates. 01:00:27.760 --> 01:00:29.320 It's no one's first love. 01:00:29.320 --> 01:00:30.780 No one wants to think about it. 01:00:30.780 --> 01:00:31.580 It's very confusing. 01:00:31.580 --> 01:00:32.780 What's a manifest.in? 01:00:32.780 --> 01:00:34.360 Why is that different than package data? 01:00:34.360 --> 01:00:35.180 I don't get it. 01:00:35.180 --> 01:00:38.720 And so checkmanifest, it just does one little thing and it's beautiful. 01:00:38.720 --> 01:00:40.600 And I had never heard of it before. 01:00:40.600 --> 01:00:43.040 And it seems like people should be screaming it from the rooftops. 01:00:43.040 --> 01:00:43.640 Yeah, awesome. 01:00:43.640 --> 01:00:45.280 Well, that's exactly the thing I'm looking for. 01:00:45.280 --> 01:00:46.040 Thanks for sharing that. 01:00:46.040 --> 01:00:46.560 That's right. 01:00:46.560 --> 01:00:52.340 A second library is tqdm, tqdm, which is a progress bar library, which is very cool. 01:00:52.340 --> 01:00:53.720 Yeah, I really like that one. 01:00:53.720 --> 01:00:56.980 I've been using those types of progress bars lately and they're pretty cool. 01:00:56.980 --> 01:00:57.580 Yeah. 01:00:57.580 --> 01:00:57.780 Nice. 01:00:57.780 --> 01:00:58.720 Okay. 01:00:58.720 --> 01:00:59.540 Final call to action. 01:00:59.540 --> 01:01:00.860 People are excited about coverage. 01:01:00.860 --> 01:01:02.560 Maybe even they're excited about edX. 01:01:02.560 --> 01:01:03.720 What do you want to leave folks with? 01:01:03.720 --> 01:01:04.620 Read about coverage. 01:01:04.620 --> 01:01:07.500 I've got some docs that I think are good, but that's because I wrote them. 01:01:07.500 --> 01:01:08.880 So I can confirm that. 01:01:08.880 --> 01:01:09.580 They were very good. 01:01:09.580 --> 01:01:13.200 I went through the docs to do a lot of research for this show and they were at the right level. 01:01:13.200 --> 01:01:15.140 What I wanted to know, but not so much. 01:01:15.140 --> 01:01:16.180 I couldn't get through them. 01:01:16.180 --> 01:01:16.680 It was perfect. 01:01:16.680 --> 01:01:17.060 Good. 01:01:17.060 --> 01:01:17.720 That's good to hear. 01:01:18.200 --> 01:01:19.200 I hang out. 01:01:19.200 --> 01:01:21.580 I hang out on the Python IRC channel and I love to see people there. 01:01:21.580 --> 01:01:24.700 I think it's a great way to connect with people. 01:01:24.700 --> 01:01:27.640 I like to think of it as a nice IRC channel. 01:01:27.640 --> 01:01:32.140 So if you've been to IRC and didn't like it in the past, try the Python IRC channel on Freenode. 01:01:32.940 --> 01:01:34.000 It's openedx.com. 01:01:34.000 --> 01:01:35.340 Openedx.org. 01:01:35.340 --> 01:01:37.400 I'm nedbatt on Twitter. 01:01:37.400 --> 01:01:38.400 You can follow me. 01:01:38.400 --> 01:01:41.060 I've got a blog that I've been running for far too long. 01:01:41.060 --> 01:01:44.920 If you want to read what I thought about 16 years ago, get in touch. 01:01:44.920 --> 01:01:46.140 You know, I like hearing from people. 01:01:46.140 --> 01:01:46.840 Yeah, that's awesome. 01:01:46.840 --> 01:01:48.780 The internet is written in ink, right? 01:01:48.780 --> 01:01:49.640 All that stuff's still there. 01:01:49.640 --> 01:01:52.460 So I definitely find your blog interesting. 01:01:52.460 --> 01:01:55.580 There's some topics in there I would love to have you back on to talk about. 01:01:55.580 --> 01:01:58.060 But for this one, we're going to have to just leave it here, I think. 01:01:58.060 --> 01:01:59.140 So thanks for being on the show, Ned. 01:01:59.140 --> 01:01:59.600 Sure. 01:01:59.600 --> 01:02:00.280 Thank you, Michael. 01:02:00.280 --> 01:02:00.800 This is great. 01:02:00.800 --> 01:02:01.180 You bet. 01:02:01.180 --> 01:02:01.440 Bye. 01:02:01.440 --> 01:02:01.660 Bye. 01:02:01.660 --> 01:02:05.700 This has been another episode of Talk Python To Me. 01:02:05.700 --> 01:02:10.700 Our guest on this episode has been Ned Batchelder, and it's been brought to you by Brilliant.org 01:02:10.700 --> 01:02:11.320 and Manning. 01:02:11.320 --> 01:02:17.080 Brilliant.org wants to help you level up your math and science through fun, guided problem 01:02:17.080 --> 01:02:17.400 solving. 01:02:17.400 --> 01:02:21.080 Get started for free at talkpython.fm/brilliant. 01:02:21.080 --> 01:02:24.820 Learning Python doesn't have to be overwhelming or intimidating. 01:02:25.340 --> 01:02:28.060 Check out Get Programming by Annabelle from Manning. 01:02:28.060 --> 01:02:33.980 Just visit talkpython.fm/manning and use the code belltalkpy to get 40% off. 01:02:33.980 --> 01:02:35.860 Want to level up your Python? 01:02:35.860 --> 01:02:41.040 If you're just getting started, try my Python jumpstart by building 10 apps or our brand new 01:02:41.040 --> 01:02:42.900 100 days of code in Python. 01:02:42.900 --> 01:02:46.320 And if you're interested in more than one course, be sure to check out the Everything 01:02:46.320 --> 01:02:46.700 Bundle. 01:02:46.700 --> 01:02:48.960 It's like a subscription that never expires. 01:02:48.960 --> 01:02:51.160 Be sure to subscribe to the show. 01:02:51.160 --> 01:02:53.360 Open your favorite podcatcher and search for Python. 01:02:53.360 --> 01:02:54.600 We should be right at the top. 01:02:54.980 --> 01:03:00.380 You can also find the iTunes feed at /itunes, Google Play feed at /play, and 01:03:00.380 --> 01:03:03.900 direct RSS feed at /rss on talkpython.fm. 01:03:04.240 --> 01:03:05.780 This is your host, Michael Kennedy. 01:03:05.780 --> 01:03:07.140 Thanks so much for listening. 01:03:07.140 --> 01:03:08.220 I really appreciate it. 01:03:08.220 --> 01:03:10.180 Now get out there and write some Python code. 01:03:10.180 --> 01:03:40.160 I'll see you guys next time.