« Return to show page
Transcript for Episode #156:
Python History and Perspectives
0:00 Michael Kennedy: Learning about programming libraries and languages is useful and interesting, but sometimes knowing why certain decisions were made or the history leading up to some change or a package being created gives you a deeper understanding. And that's what this episode is all about. You'll meet Mike Driscoll, who runs PyDev of the Week. He also just published a really relevant book, Python Interviews: Discussions With Python Experts. This is Talk Python to Me, Episode 156. Recorded March 21st 2018. Welcome to Talk Python to Me, a weekly podcast on Python, the language, the libraries, the ecosystem, and the personalities. This is your host, Michael Kennedy. Follow me on Twitter where I'm @mkennedy, keep up with the show and listen to past episodes at talkpython.fm, and follow the show on Twitter via @talkpython. Before we get to our conversation I want to tell you about a project I've been working on for the past three months. Do you remember Bob and Julian from Episode 140, where we talked about the 100 Days of Code challenges, and their projects around it? Well, since then we've teamed up and created a new course. 100 Days of Code in Python. This course has 18 hours of video lessons and around 80 hours of guided exercise. That's 100 hours, one for each day. If you're considering doing one of these 100 Days of Code challenges, please have a look at our course over at talkpython.fm/100days. Now, let's get on to the interview. Mike, welcome to Talk Python.
1:38 Mike Driscoll: Thanks for having me, Michael.
1:39 Michael Kennedy: Yeah, it's great to have you here. I've been a fan of your Python Dev of the Week project that you've had going for a long time, and things like that. How long has that been running?
1:49 Mike Driscoll: I think it's been going on now for almost three years.
1:51 Michael Kennedy: Yeah, that's that's pretty good, that's like 156 interviews or something on that scale, right? That's cool.
1:56 Mike Driscoll: Yep yep, that's correct.
1:59 Michael Kennedy: Yeah nice, awesome, we're going to talk about some of the interviews you did, not necessarily for that project but somewhat in the same vein as that, for this really cool book that you put together called Python Interviews. Which I think is a great idea. But before we get to it let's start with your story. How'd you get into programming in Python?
2:22 Mike Driscoll: I started out with your traditional languages in school, like C++ and COBOL. I know COBOL isn't really that interesting anymore, but that was what I was learning back then.
2:32 Michael Kennedy: There was a part of time where it was really good consulting, right? People really needed COBOL to help the transition, right?
2:38 Mike Driscoll: Yeah yeah, I do know a contractor who makes really good money doing COBOL for the government. There's definitely jobs still out there for you people who want to learn ancient dead languages.
2:48 Michael Kennedy: I guess so. All right, so you started with those languages, and you studied Comp Sci in college?
2:53 Mike Driscoll: Yeah, I went through Computer Science, then I transitioned to MIS, then I graduated, and the dot-com bubble burst, and there were no jobs.
3:02 Michael Kennedy: Oh, that hurts. And probably while you were in college you're like, oh my god, the jobs are so hot, I can't wait to get out there and like work with a place with like volleyball and massages and all the excesses of the day.
3:16 Mike Driscoll: Yeah, it sounded like it was going to be an awesome opportunity and then I get out and I'm like wow, I can't get a job anywhere because everyone has to have five years experience or more.
3:24 Michael Kennedy: Oh, that's rough, so what'd you do?
3:25 Mike Driscoll: I actually worked kind of as, I worked as a so-called webmaster doing FrontPage based websites for an auction company.
3:35 Michael Kennedy: FrontPage, the product, right? The thing that looked a little like Word but made websites?
3:38 Mike Driscoll: Yeah, the Microsoft product that's now dead, but yeah it created templates, and it was kind of pathetic.
3:44 Michael Kennedy: Yeah, it was a really interesting take on how you make the Web, and of course it was pretty typical of how Microsoft thought at the time. They don't do that anymore, either.
3:53 Mike Driscoll: No, they don't. Anyway, I did that and then the local county actually was hiring and for some reason they ended up choosing me. And they said we're transitioning away from Microsoft because every time we upgrade our VBA applications, like Office, it breaks all of our apps, and we want to go to open source. And the language that my boss had chosen at the time was Python. And so he said you can learn Python or, you know, you won't be able to work here. So I picked up Python on the job and I got really good at it and basically fell in love with language.
4:29 Michael Kennedy: That's really cool, what was your first thought when you got that news?
4:32 Mike Driscoll: Well I was pretty nervous because I was like, I had studied all this stuff in college and now I'm going to do a completely different language, which is very different than what they're teaching in college. And they're just like, whoa, transition shift. You know?
4:45 Michael Kennedy: Yeah.
4:46 Mike Driscoll: It was a little trial-by-fire, but it worked out in the end.
4:50 Michael Kennedy: Yeah, that's awesome. And then after that you pretty much have been just sold on Python.
4:54 Mike Driscoll: Yeah, I'm definitely a Python advocate.
4:57 Michael Kennedy: Yeah, of course, that's awesome. So that's how you got into it, how about today, what do you do day-to-day?
5:02 Mike Driscoll: Currently I'm working for an AG company that creates tools to automate and track what you're doing when you're planting and when you're harvesting. And it can basically plant your corn or your beans or whatever down to the centimeter, depending on your GPS quality.
5:23 Michael Kennedy: Wow.
5:23 Mike Driscoll: And what's cool about that. Yeah, I know. When I got into this, I didn't know that tech had reached the farm world. It's amazing what they are doing now in agriculture and in food produce.
5:36 Michael Kennedy: Yeah one of things I think that's super interesting is just how diverse the uses of Python and programming are. I suspect a lot of people when they think of farming they probably think of large farms, they probably think of like John Deere tractors and stuff doing their thing. But they probably don't think deep IoT, and a lot of code, and automation and all that kind of stuff.
6:01 Mike Driscoll: Yeah, yeah. I know I didn't.
6:06 Michael Kennedy: I think there's actually some really cool stuff in IoT in farming, for sure.
6:09 Mike Driscoll: Oh yeah yeah, definitely. They're getting more and more connected, so that you can track what you're doing on your iPad, and the display in the cab, and on your phone, too. And everyone in the business or your relatives can kind of see how you're doing, basically.
6:25 Michael Kennedy: Yeah, that's cool. There's a guy who was listening to this podcast and sent me a picture of it being played or something while he was plowing a field in his John Deere tractor. He's like, yeah I'm studying up while I'm working on the farm. Wow, that's really cool. So tell us about the app that you work on.
6:41 Mike Driscoll: What I actually work on is the testing of the applications that we work on for the agricultural company. So I write Python that uses a tool called Squish that connects to the C++ embedded application that we've written, and it'll actually instrument it and be able to press any buttons or grab any information out of it. And it allows us to test all of our different configurations and do the buttons exist when they're supposed to or not because we have permissions related to the type of owner or operator who's using the tool, stuff like that. Basically it gets pretty complicated. We have probably 300 or 400 tests just for the UI, not counting all the other unit tests and other things underneath the covers.
7:25 Michael Kennedy: That sounds pretty awesome, sounds like a fun project. So let's talk about some of your books. We talked about the Python Dev of the Week project, which is on the web, but you've written a number of books, right?
7:39 Mike Driscoll: Yeah, my audience for my blog had been encouraging me for a couple of years to write a book, and I finally took them up on that and wrote Python 101, which was pretty well-received. I did a Kickstarter for it to help and get some publicity and just to see if there was any interest in it. And that was by far the book that got the most interest of any of them that I've written. And people are still buying it and reading it, and telling me that they liked it for the most part.
8:08 Michael Kennedy: That's really awesome. I definitely think these, if you look at what people are interested in, it seems like it might be most amazing to write some really advanced book, but it's really the foundational stuff that actually most people are interested in. Like if you wrote the internals of asyncio, even though it might sound cooler than Python 101, it wouldn't be the same, right?
8:31 Mike Driscoll: Yeah, I've noticed that with the more specific topics you get the smaller the audience chunk is, because there're just less people that are going to be interested in that specific topic.
8:41 Michael Kennedy: Yeah, for sure. I think launching it on Kickstarter was a really good idea. I launched my training company on Kickstarter as well, and it was really surprising, but it really took a lot of the edge off. After a week you're like, OK, there's enough interest in this for me to pursue this project. It's not just me thinking of an idea and going, I hope it's good, we'll find out in six months. But you put it up there, people commit to it right away and give you feedback, and it actually guides a little bit of what you might build, right?
9:14 Mike Driscoll: Yeah, that's actually why I like Kickstarter, is that I got to work with people. My early readers got to have feedback early on in the process, and I was able to actually take some of their ideas and be like, this is awesome, let's add this to the book, or let's add that to the book. Or this actually isn't a good idea to talk about, nobody's interested in this topic.
9:33 Michael Kennedy: Yeah yeah, that's really really cool. So then you also did Python 201 Intermediate Python, right? Was that also on Kickstarter?
9:40 Mike Driscoll: Yes, it definitely was, and it got a lot of interest as well, although maybe not as much as of course the first one. But a lot of people thought it was really interesting and they took it as a good stepping stone to the next level, I think.
9:54 Michael Kennedy: Yeah, that's cool. One you did that really sounds super interesting to me these days is wxPython Cookbook and Recipes?
10:02 Mike Driscoll: Yeah, wxPython Cookbook I've been working with that community practically from the beginning of my time starting with Python and I thought, you know I've written a lot of recipes on my blog, I should clean those up and write some new ones and create a book around it. And so that's kind of what I did. And then Apress came along and said hey we like that book, we want to compete with Packt. Could we license it and give it our brand. You can call it recipes instead of cookbook. And so--
10:30 Michael Kennedy: That's interesting.
10:32 Mike Driscoll: That's kind of how that ended up happening.
10:34 Michael Kennedy: Yeah does this cover the wxPython Phoenix, or does it predate that, and how different are those?
10:42 Mike Driscoll: Yeah, it does cover Phoenix, which is now wxPython 4, actually. I tried to do both 3 and 4 when I wrote the book itself, and then when Apress came along I kind of updated it, so most of it is just for 4 but I did leave a little bit of legacy code in there to kind of show the difference. The big things that have changed is that Robin Dunn, the creator of wxPython, switched from Swig to Sip, and so the layer between Python and C++ is thinner, and it makes doing Python 2 and 3 a lot easier for him. But otherwise it's just like, he deprecated certain naming conventions, and just little things that have bothered him for who knows how long. Probably a decade, and now he's finally gotten it nicer.
11:33 Michael Kennedy: Yeah, that's cool. All right, I want to talk about your main book and really dig into that one, but you have one that also was on Kickstarter, I know because I backed it. And I don't know if, I think the Kickstarter is done, right, called ReportLab? Want to tell us about that?
11:49 Mike Driscoll: Yes, that Kickstarter is over.
11:51 Michael Kennedy: OK, cool. How did it go?
11:52 Mike Driscoll: I thought it went OK. I thought it might be a little bit more interest than the wxPython one, because my ReportLab articles on my blog get way more hits than my wxPython ones do. So I just thought ReportLab's going to be more interesting than wxPython is, at least book-wise. But oh well, it's been really good, I've gotten good feedback, I've got good people telling me get some good ideas. And the book itself is coming together really really well. And since a lot of it's brand new content I feel really positive about it because usually when I'm writing everything from scratch it takes a lot longer and they're just kind of like coming right out really smoothly for me.
12:36 Michael Kennedy: Oh yeah, that's really great. And so this basically teaches you how to do PDF processing in Python, right?
12:43 Mike Driscoll: Yeah yeah, ReportLab is for creating PDFs and basically laying out the page in your PDF. I also have a part of the book that's going to talk about some other Python related PDF tools, like PyPDF 2, and PDF Miner, that I'll show you how to export data from your PDF or split your PDF into the multiple pages or join multiple PDFs together. Different things like that that ReportLab doesn't really do natively.
13:11 Michael Kennedy: Yeah, that's really cool, I definitely want to study this because I need to create some PDFs for various things on my site, so I'm excited for this to come out. When's it coming out?
13:19 Mike Driscoll: It'll come out in June or maybe even May if I'm lucky.
13:22 Michael Kennedy: Yeah, that's awesome. All right, so let's talk about your book that you do have out, this is probably the most recent one published, right? The one called Python Interviews: Discussions with Python Experts?
13:32 Mike Driscoll: Yes, that one was released earlier this month actually.
13:35 Michael Kennedy: Yeah, nice. And this one is published through Packt, whereas your original wxPython cookbook, and your other ones as well, were they self-published?
13:45 Mike Driscoll: Yes, that's correct.
13:46 Michael Kennedy: Interesting, OK. So give us the quick trade off between self-publishing and say working with Packt or someone else?
13:54 Mike Driscoll: The main thing I like about self-publishing is I know when I'm going to release, I know the name of the book, I have control over the quality of the book, and I can also see how it's selling at any moment in time because Amazon and Leanpub, the two primary places I sell my book, actually have graphs and they can tell you in real time how many units have sold in a day. Whereas if you work with a publisher you have to wait quarterly for updates. So you have no idea if the book is selling well or not. And you have no idea how much of a cut you're going to get either, depending on the sales that were done during that time period.
14:35 Michael Kennedy: Yeah, interesting, OK. So this one is with Packt, and I think it's a really interesting idea. So you went through and you interviewed 20 notable experts in the field. I'll just read off some, Mike Bayer, Brett Cannon, Nick Coghlan. You had Jessica McKellar, Carol Willing, Al Sweigart, Jake Vanderplas, and a bunch more. All those people I named by the way have been on the podcast so I feel like you've done this sort of interesting book equivalent of a podcast. Like I'm going to go inside and get the stories of these people behind these major projects, right?
15:11 Mike Driscoll: Yeah, yeah.
15:12 Michael Kennedy: Nice, so I guess, let's start at kind of a high level. One thing I wanted to ask you is what's your takeaway from all the time you spent interviewing these people. What perceptions or thoughts maybe changed about ideas that you had going in that afterwards you're like, oh no actually this is totally different than what I thought it might be.
15:33 Mike Driscoll: I think the biggest takeaway for me is how how big asyncio has gotten, and asynchronous programming in general. A lot of the people I talked to were excited about it, and excited to have it added to the standard library. I think I had a couple people who were cautious about asyncio, and they thought its implementation wasn't as good as it should be. Or they just didn't think it was a good idea in general, but overall I got a positive feedback loop from that, that hey this is really good. Yeah, it needs some improvement. We're going to make Python use it in a way that is really easy to use.
16:14 Michael Kennedy: Yeah, I feel like if you are in a situation or a position where asyncio will really solve the problem it's such a great thing to have it. But a lot of people write code that has no use for it and it just seems confusing, it adds complexity, and why is this thing here. But if you happen to need it then you definitely definitely can benefit from it.
16:40 Mike Driscoll: Yeah, I totally agree with that.
16:42 Michael Kennedy: I think the one area that we could use some more help on to really jumpstart this is really the major web frameworks, if they could somehow be natively async capable.
16:55 Mike Driscoll: Yeah, I know Glyph was trying to do that, make async native, even though it is technically the async socket library already. He's going to start using the Python async library in the internals of Twisted.
17:10 Michael Kennedy: That's pretty awesome, because that way you can chain your async stuff with the lower-level async stuff in a more continuous way.
17:18 Mike Driscoll: Yeah, I thought that was a really neat idea.
17:21 Michael Kennedy: Yeah, pretty cool, OK other takeaways?
17:23 Mike Driscoll: The other takeaway, and I kind of noticed this at PyCon last year, is that machine learning and data science has gotten really big with Python. And so there was a lot of excitement about that and how Python is just kicking butt in that sphere, in that business area. And it's bringing in a lot of new people that don't really know Python at all or programming in general, and they're getting introduced to Python, which I think is really cool.
17:49 Michael Kennedy: Yeah, I think it really has opened up a whole 'nother aspect and you're right it is a big theme in the book that people have talked about. So what I want to do is just go through and we'll pick out a couple of the folks you interviewed, and some of the things that I found notable in your conversation with them from the book. And maybe we could just touch on those a little bit. So there's 20 people here, and we can't go through all 20 of them, I don't think, unfortunately. So people listening, if I don't cover cover your thing, it's not that I didn't like it, I ran out of time, so time and space. So let's start with one of the core developers I really like, Brett Cannon. So Brett he lives sort of in my neighborhood, if you think globally and define a sufficiently large neighborhood. Sort of the Pacific Northwest area, anyway. And one of the things that he touched on and I thought was pretty interesting was the idea of, can you know it all, even within the Python sphere. Can you know all of Python? Should this even be a goal? And of course there's the Python and the runtime, but then there's also all the libraries and external stuff, right?
19:00 Mike Driscoll: Yeah.
19:01 Michael Kennedy: Talking about that he said something to the effect of, Guido famously was once asked at Google, on a scale of 1 to 10, how well do you know Python? He said 8.
19:10 Mike Driscoll: Yeah, I've noticed that Google has that in part of their interview processes, how well do you know something. 1 is you don't know it at all. 10 is you wrote the book on it, or you wrote the language. So I find it really interesting that Guido himself is an 8. But I totally agree with that. While I consider myself to be really knowledgeable in Python there are libraries that I've never used in the standard library, yet.
19:34 Michael Kennedy: Yeah, same here, yeah. I think it's really interesting, and at first you might think, well how is that possible? He created Python. But he's not the only one, there are tons of people working on major changes and optimizations and whole new aspects of Python that maybe he just looked at the PEP and said that's a good idea, send us a PR.
19:54 Mike Driscoll: Yeah yeah, I totally agree with that, because there's a lot of people who've worked on it. And I think even Brett would say he doesn't know it all even though he did claim I think during the interview that he had touched pretty much everything in the Python internals at one point or another.
20:08 Michael Kennedy: Right, but how much of that sticks?
20:10 Mike Driscoll: Yeah, I know. I still have to look up stuff.
20:14 Michael Kennedy: Yeah, of course, like distributed cognition, we have the Internet. So another thing that he touched on that's really important right now and is going to be increasingly important over the next year and a half is moving to Python 3. He said I hope people realize that you can transition to Python 3 gradually. It doesn't have to be abrupt or painful. And he referenced the Instagram thing that was actually at the keynote at, what was that, 2017 PyCon, 2017 I think it was. Definitely in Portland.
20:44 Mike Driscoll: Yeah, I was actually doing this at work last year, I was trying to transition our automated test framework from Python 2 to Python 3, and it really wasn't that bad. I got the code working but I wasn't able to get all the tooling to work because of issues with our vendor. But it really isn't that painful, I thought it was actually fairly easy and we have, you know, probably 10,000 lines of code or more just in that one testing framework.
21:13 Michael Kennedy: Yeah, I think to me, when I look at the changes, I'm like, none of these are major. They're really really not, the one place where it gets really kind of tricky is at the network layer where you're trying to switch to Unicode versus bytes and strings and all that, but other than that boundary, it's really not that big of a deal. I think part of the major challenge for people is, we have this huge thing, it's working, please don't touch it.
21:40 Mike Driscoll: Yeah yeah, I agree. The issue that I think we found the most painful was writing binary files out, and we were using byte strings and we had to convert one way or the other, and that was a little bit trickier in Python 3. When I wasn't expecting it.
21:57 Michael Kennedy: This portion of Talk Python to Me is brought to you by Linode. Are you looking for bulletproof hosting that's fast, simple, and incredibly affordable? Look past that book store and check out Linode at talkpython.fm/linode. That's L-I-N-O-D-E. Plans start at just five dollars a month for a dedicated server with a gig of RAM. They have ten data centers across the globe, so no matter where you are there's a data center near you. Whether you want to run your Python web app, host a private git server or file server, you'll get native SSDs on all the machines. A newly upgraded 200 gigabit network. 24/7 friendly support, even on holidays. And a 7 day money-back guarantee. Do you need a little help with your infrastructure? They even offer professional services to help you get started with architecture, migrations, and more. Get a dedicated server for free for the next 4 months. Just visit talkpython.fm/linode. So another thing that Brett talked about that was pretty interesting was the mythical Python 4. Will there be a Python 4, what's going to be in Python 4, will it be as painful as two to three? He had some interesting ideas. Obviously it's not going to be any sort of thing like going from Python 2 to Python 3, but he talked about some things like switching to a tracing garbage collector instead of reference counting, to I think help with things like parallelism and stuff. And this parallelism is another theme that actually came up a lot in the book. Basically taking advantage of multi-core processors.
23:26 Mike Driscoll: Yeah, there was several people who are bringing up that topic, and how Python is usually given a black eye about being too slow in a lot of cases, when in actuality if you use it correctly you can get a lot of speed out of Python. Or write your stuff in Python and then switch to PyPy or some other kind of thing like that, and you can quadruple your output or your speed with just little tiny changes like that. But yeah, Brett was definitely talking about that, and I think Barry Warsaw was talking about removing the GIL. So yeah, there was a lot of different back-and-forth about how we could make parallelism in Python better.
24:07 Michael Kennedy: Yeah, absolutely. So let's switch on to another one of your interviewees, Steve Holden. One of the ideas that I thought was really interesting, and I touched on it recently in another episode, and I can't remember who I was talking to about it, which is really unfortunate, but it's the idea that the forward progress and overall maintenance of Python and the ecosystem and whatnot continues with no professional organizers. You've got Microsoft completely controlling something like C#, and anything that happens, there might be community input, but there's this giant company whose job it is to decide what the next steps are. Whereas with Python I guess there's just none of that. There's just us.
24:57 Mike Driscoll: Yeah yeah, the only thing that's really behind Python is the Python Software Foundation. And it just kind of guides the community into how things should work.
25:08 Michael Kennedy: Yeah, they're there more to support things than it is to specify directions. The PSF doesn't go, we need these three features in this library upgraded, and this thing done, for Python 3.7.
25:22 Mike Driscoll: Yeah, yeah they just provide funding and protect the Python brand, they don't really say we need X, Y, and Z.
25:29 Michael Kennedy: Exactly, yeah they actually, they handle the corporate details and chasing people down if they mess up with the copyright, and funky stuff like that, which is a bit of an odd job, it struck me as odd when I first heard it, but if someone's got to do it, they're the people to do it.
25:48 Mike Driscoll: Yeah, someone's got to do it. You don't want people using the Python name inappropriately.
26:44 Mike Driscoll: Yeah, I feel like we don't really need anything brand spanking new in Python anymore. There was some controversy over adding f-strings because we already had two or three ways to do string interpolation already. I think f-strings are kind of cool, but did I need them to do my job, no. So, I can see where Steve's coming from with that.
27:04 Michael Kennedy: Yeah, I feel the same way, actually. The one language feature that comes to mind, I'm like yeah, that really does something different is the async and await keywords.
27:13 Mike Driscoll: Yeah that one is definitely new, and well like I said, a little bit painful, especially with the first implementation. It's getting better and it definitely does something new and different in the language.
27:25 Michael Kennedy: Right right, whereas the fifth way to format a string is certainly debatable about whether it's needed. There's not a fifth way to asynchronously work with IO. That is part of the language, I mean they were obviously libraries.
27:38 Mike Driscoll: Yeah I kind of wonder sometimes though why they didn't continue working on the futures library because it seems to be kind of just kind of died off now.
27:45 Michael Kennedy: Oh yeah, that's a really good point. Maybe put some trial stuff, float some trial things, maybe even an external package that you can bring in and try and see what sticks.
27:54 Mike Driscoll: Yeah, yeah.
27:54 Michael Kennedy: That'd be pretty cool. So another thing that Steve talked about was the excitement of Python being the main programming language for people coming in in education. And I think this touches back to his previous comment as well, which is hopefully this will keep Python in a space where it's relatively comprehensible and these easily accessible tools for building rich applications will be available to people for the next 20 years at least. So really having a, not losing the ability of people to get into Python easily.
28:30 Mike Driscoll: Yeah, I think that's really awesome that Python has gotten a hold in the education sphere so much. I think I read earlier this year that MIT, all of its intro stuff is in Python now.
28:43 Michael Kennedy: Oh, that's sweet.
28:44 Mike Driscoll: And I think Iowa State University, which is where I did some of my studies, they're starting to use Python now in some of their intro courses. And I just think it's really cool that it's spreading like that. People have asked me if they can use my book as kind of a trial run in their classes, and I might like, go for it, see if it works, if it doesn't let me know. So it's just really cool what's going on with Python and education.
29:54 Michael Kennedy: Yeah, it just seems like they just went so far off the deep end, of going oh we're going to do all these things and bring all this stuff in. And there's also not consistently one way to do things. It just seems unnecessarily complicated. I think one of the powers of Python is it starts super easy. You can have a really complicated, complex architecture, but it's not required. Where it almost is these days to use a lot of the web stuff.
30:19 Mike Driscoll: Yeah, Python definitely has a nice, easy learning curve until you hit some of the more advanced topics. Which is what you'd expect.
30:26 Michael Kennedy: It's ideal, yeah. So let's move on to another interviewee, Carol Willing. She ran the PSF for a while, and she's done a bunch of cool stuff. And I think the first thing that I noticed from your conversation with her was the concept of Python as a mosaic. And this was really laid out super clearly by Jake Vanderplas at PyCon 2017 at his keynote. Where it's just these people are coming from all these different disciplines, and you touched on this as well at the beginning, that all these different ways and types of programming and areas of programming are coming into Python, and probably making it stronger for giving it these different perspectives.
31:06 Mike Driscoll: Yeah yeah, I've noticed especially, like I said, at the last PyCon you had data scientists, astrophysicists, people who had a course for middle school and teenagers to learn Python at PyCon. And then of course you had the people like Glyph, who does Twisted. You have all these asynchronous people, and people from Instagram and the business side, and it's like wow, you have all these disciplines, and they're all using the same fun language to do really cool things.
31:34 Michael Kennedy: Right, exactly, you had Instagram talking about how they are running their whole thing on Python. But you also had Jake Vanderplas going, these three major telescopes that are being launched in the next couple years, here's their API, it's in Python. The juxtaposition was amazing.
31:52 Mike Driscoll: And it's free, you can go read their code.
31:53 Michael Kennedy: Exactly, yeah, it's really cool. And just touching on astronomy for a minute. What's really cool is they had another language, it's still around I'm sure, for astrophysics called IDL, I believe. And that was a commercial language, you want to even compile your code, you pay money to get the compiler and the license just to work on that. And here you have, oh yeah no, we're just going to put it on GitHub and it runs on Python and now we're talking to space. Pretty cool.
32:21 Mike Driscoll: Yeah, I think a lot of the data scientists are getting away from that because they have Mathematica and some of those other math languages of sorts. And they had to pay big big bucks to do their job. When now they can use open source and share it with the world.
32:38 Michael Kennedy: Yeah, it's super amazing. I came from a math background and I worked at this company where we had to do wavelet decomposition of time series stuff. And that originally was in MATLAB and we were converting it over to a proper programming language. But just to get the little extension to MATLAB so that we could do that I believe was $2,000 a person. Just for the extension for the wavelet decomp, not the whole thing, just that little bit. It's like wow, that is intense.
33:07 Mike Driscoll: Yeah, we have some tools in the agriculture field that are very similar to that, where you have to spend a lot of money to compile from one language to another language, or do exactly what you just said because we also use MATLAB as well.
33:21 Michael Kennedy: Yeah, but we have things like SageMath, Jupyter, there's a lot of stuff that's coming along to disrupt that. I think those days are somewhat numbered, except for maybe really niche stuff, which probably is for the better.
33:34 Mike Driscoll: Yeah, totally.
33:35 Michael Kennedy: So another thing that Carol touched on I thought was cool is what I'm going to call Python for all. She has a data science background and does a lot of stuff with Jupyter. And so she talked about the rise of Python in education. But also the rise of Chromebooks in education, where you can't really install anything on your Chromebook. And so providing every student some kind of web account into something they can do Python on, and give them the same tools and the same experience. Which I think is really empowering.
34:11 Mike Driscoll: Yeah, I think if we had a couple more places where you could actually do Python in the browser, that would be really really powerful. I know there are a little bit, and it's pretty limited right now. But I think we're getting there to the point where we can I should start saving the Python scripts to the cloud and running them in the cloud, in such a way that a student could learn how to do it. And that would be really awesome.
34:33 Michael Kennedy: I think it is really awesome and I think that at that level the notebook-type answers are really really interesting, so Azure has I think free notebooks. Google has a notebook solution. JetBrains just came out with Datalore. All these are hosted online. You log in, you save your stuff. You run, and it runs there. So those are pretty cool. Even on the Google one, you could even hit, you know how you shift-enter in notebooks to run it, there's a modifier key to run this cell on the GPU. And that's it, you just use a different hotkey to run on the GPU, I was like wow, that's quite different.
35:14 Mike Driscoll: Yeah, I hadn't even heard of that.
35:16 Michael Kennedy: I covered it on Python Bytes a little while ago, I can't remember the name, I'll try to put in the show notes. I think some of these things are coming along. You have Python Anywhere, Python Everywhere, sorry guys. Actually if you go to python.org, I didn't realize this for the longest time, if you go to python.org there's those little screenshots of text and there's a little prompt-looking button. If you press that it'll actually open up a remote terminal at Python Anywhere, and you can just start typing.
35:42 Mike Driscoll: I knew it had a terminal, but I didn't realize Python Anywhere was underneath it.
35:46 Michael Kennedy: So pretty cool, right? But I think we need to be aware that so much of education at the lower levels is running on Chromebooks, and you can't pip install anything on the Chromebooks. You don't get a terminal, you get basically flavors of Chrome. So having a really good way to bring people on there is quite cool. You know another thing that comes to mind is Anvil, anvil.works. They basically let you create web apps, and the whole IDE is in the browser, so you can pretty much do that from Chromebooks as well. It's kind of cool.
36:17 Mike Driscoll: That is really neat.
36:19 Michael Kennedy: So another thing that Carol touched on was the community, and so did Brett, it was basically, I came for the language, I stayed for the community. I think that's one of Python's superpowers, how about you?
36:29 Mike Driscoll: Oh yeah, that's definitely what kept me going with Python. I have questions, and I ask them. And you don't usually get into a flame war, you usually get people saying hey, this is how you do it. Or ask the question better. They're not like, that's a dumb question. It's an actual real answers most of the time. And I really appreciate that they're so helpful, and sometimes you can give you some sample code to point you in the right direction.
36:55 Michael Kennedy: Yeah, that's really cool. Also I think you know you go to the Python conferences and they just feel different than a standard tech conference. They feel I don't know more representative of the general population, people seem quite open, and the whole community seems to really carefully nurture that aspect more than other communities I'd been part of.
37:18 Mike Driscoll: Well I really liked about it is that you'll sit down at a table with a bunch of strangers, and you'll be surrounded by people who are just starting Python, or people who are working for major companies. Like I sat at a table with Apple and Pixar, and major school, university, and I'm like wow, I wasn't expecting this a with these kind of people today. And they're all friendly.
37:41 Michael Kennedy: Yeah, it's really really cool, definitely. So, opposite of superpowers, maybe one of Python weaknesses, and I've been on a bit of a rant about this as well lately, but Carol talks about, you know, we need a better story for mobile, and I'll throw desktop development in there as well. There's Kivy, which does let you create apps, but it's not the same experience as Storyboards and Xcode stuff, or the IDE for Kotlin for Android, things like that.
38:12 Mike Driscoll: Yeah, Python doesn't have much besides Kivy, and I think Toga is the other one that I've heard about, but haven't actually tried. But yeah, there's basically nothing besides those two and maybe some like underlying libraries that Kivy relies upon, and it'd be awesome--
38:29 Michael Kennedy: Does that shock you a little bit?
38:30 Mike Driscoll: Yeah, it does, and it kind of bugs me that that happened. And I think Alex Martelli said that Guido had tried to talk to the people at Google when he was working there, to get them to include Python.
38:42 Michael Kennedy: Right right right, Andy Rubin, yeah.
38:44 Mike Driscoll: And the guy was like, we don't need more than one language for Android.
38:47 Michael Kennedy: Java is amazing, so we'll do that.
38:48 Mike Driscoll: And I'm just so disappointed that Google decided to do that.
38:53 Michael Kennedy: Yeah, that's really is unfortunate. It seems strange to me that there's such a hot development space and there's just not really an answer for it. And I think it's a couple of things. I think it's a little bit of the myth of boiling the frog slowly in the hot water, type of thing. It's even hard to just take your Python application and give it to anybody like say on Linux or Windows or whatever and go here, double-click this. That's not an easy thing in Python. And it should be. So if it's hard to even move your code from one place to another, to package it and compile it for another platform, it's just like well, we haven't solved this first step, so we're not going to go on the one-mile journey to get to mobile.
39:43 Mike Driscoll: Yeah, that's something, and we didn't really talk about it during the interview process, but that's something I think would be awesome in Python 4 is, let's have that built into the library so we can actually export the application to any EXE, or Linux tarball, or whatever you need to install on Linux. Because every flavor of Linux is different.
40:02 Michael Kennedy: That would be a game-changer. If you could just go python --build, this folder, with this entry point. And then you get a thing that you hand out and there's no dependency on Python, there's no dependency on the libraries, it's just packaged up and it's ready to go. I know there's PyInstaller and certain things, but there's just, all those things feel 85, 90% done, but then there's this weird glitch on this one framework or in this one platform you like. Why do some some of the windows show when I say no console? Stuff like that makes me crazy.
40:34 Mike Driscoll: Yeah, those things are pretty aggravating. I think the Windows executables that they create are probably better than, I've had better luck with that than I've had with any of the other platforms so far.
40:45 Michael Kennedy: I agree, but it's just it seems so fundamental, that I want to build a program and give it to someone else. Why is that not one of the first things that is super focused on. I want to build a program and distribute it.
40:58 Mike Driscoll: I know, it bugs the heck out of me. When I first started learning Python, I was, how do I do this? It took a while, it took a lot of trial and error to get it to work right.
41:07 Michael Kennedy: Yeah, you can do it, it just has lots of sharp edges, though. If I can put in a vote for what is a major feature in the next version of Python, whether that's 4, or 3.8, 3.9, or whatever, make this packaging thing just absolutely a non-issue. A binary comes out, you hand it off, it runs. Maybe you got a builder for each platform, I don't care, that's fine. But something you can send out, right?
41:34 Mike Driscoll: Yeah, I totally agree with that. That'd be wonderful.
41:37 Michael Kennedy: All right, two votes I'm sure it's going to happen now. Fingers crossed anyway. You already spoke about Glyph, let's talk about Glyph. So one of the things he talked about, and this was also touched on by Guido in the PyCascades keynote, which was in this past January, the transition of Python 2 to Python 3, as it was managed by the core developers, and the PSF, and just generally. And I think the main takeaway was they didn't realize how popular the thing they had created was and how much code was already in place. So basically how much would have to be migrated.
42:20 Mike Driscoll: Yeah, there's a lot of legacy Python 2 code. I've been doing Python since 2.4, I think? And Python 3 came out like two or three years after I started doing Python. And I've never worked, in my 12 years of doing Python, in a company that uses Python 3 regularly. Everything uses Python 2.
42:39 Michael Kennedy: So what do you think about the 2020 end-of-life date, do you think that's going to make a big difference for you guys?
42:45 Mike Driscoll: If I'm still at my company I'm going to make sure we're already on 3. But there are a lot of companies where I can see where they're like, it's going to cost a lot of money, and we're just going to keep running this code until we have to for security reasons or some other monetary reason.
43:00 Michael Kennedy: I wonder how well-managed that's going to be versus how headline driven it's going to be. So for example, I think the biggest distributed denial-of-service attack that's ever been launched was because a bunch of memcached servers were left on the Internet unprotected. And they had this vulnerability where you could send one you UDP packet at it and it would basically dump the memory contents at an address you pointed at. So you can get a 50,000 multiplier of your bandwidth against, there's just like thousands of these on the Internet that somebody pointed at GitHub and went after. Somehow GitHub survived actually, but anyway, if there's something like that and you're like, oh you just do this to all the Python 2 things running, and you own it, or you look at Equifax, or something like that, right? One of these really bad sort of headline driven things, that would possibly get people's attention. Above the developer level.
43:58 Mike Driscoll: Yeah, I think that's what's going to be, unless they can just kind of sneak it in, a lot of developers are going to be stuck with Python 2 for a long time. And it's a good language, but they really need to move to Python 3 if they can.
44:12 Michael Kennedy: Yeah, absolutely, and it's increasingly there's a lot of things that are inaccessible to Python 2 developers, API Star, Django 2, there's just a bunch of things that are coming along that are only Python 3. Which is an interesting twist because the story used to be told exactly in reverse while you stay on Python 2.
44:30 Mike Driscoll: Yeah, we're finally at the point where Python 3 is getting all the tools and all the tooling, and it's not always getting backported to 2.7. So, if you want to use the new cool stuff that the cool kids are using, you got to go to Python 3 right now.
44:45 Michael Kennedy: Yeah, for sure. So another thing that Glyph touched on was a little bit of a similar idea as that you and I were just talking about with packaging. One of his main thoughts of where he thinks Python should go is to get away from using pip as a tool for installing apps for users. And also touched on just the GUI, like there's not really any built-in way to create a Python GUI. Be it mobile or be it desktop. I don't know what else to add to that, but we've kind of, just one more voice, maybe he's a third vote for us in this package story.
45:21 Mike Driscoll: Yeah, TkInter, it's still kind of ugly, but I will say it has gotten a lot better than it was when I first started using it.
45:28 Michael Kennedy: Yeah, I've definitely seen a few examples recently, like that's actually not as bad as I remember it. But it doesn't feel like wow, this is an amazing UI framework that I just can't wait to build my apps on. It seems like, well I guess I could live with that, and it's probably not where you want it to be.
45:44 Mike Driscoll: What's funny is they added a new library called ttk to it and there was like no advertising, no nothing, so I didn't even know it had improved a lot.
45:52 Michael Kennedy: Yeah, I didn't either.
45:53 Mike Driscoll: I don't think a lot of people realize that.
45:55 Michael Kennedy: Yeah, the only way I found out was I complained about it and somebody said hey, I made some pretty good looking apps with it and send me some GitHub repos. I'm like, those are good looking apps, really. Oh, interesting. Another person you spoke to was Doug Hellman, who does the Python Module of the Week. And he's been doing that for 10 years. That's a pretty interesting series.
46:15 Mike Driscoll: His website is pretty good, I used to use it a lot for when I needed to look up something.
46:21 Michael Kennedy: Yep, it's definitely cool. So you know another thing, Doug mostly works on cloud and OpenStack and things like that. But just keeping with the theme, the main takeaway from your interview with him was that we need better packaging. We still need another way. But his take was that this is not directly related to Python itself but it'll get into all the tools, like Setuptools, Twine, Wheel, Pip, Warehouse and so on.
46:45 Mike Driscoll: Yeah, it would be really be nice if we had a common base framework to just make all those tools work the same way. And we'd have a unified way of distributing and packaging our applications.
46:59 Michael Kennedy: Yeah definitely, we could have an entire show on that, but I don't really like to complain without providing the answer, at least a suggested, like this is bad, here's a possible solution. But yeah I think the packaging thing would really make a huge huge difference.
47:18 Mike Driscoll: I would just going to say, I keep hearing good things about Briefcase, but I've not used it myself, so that's something people could look at I guess or at least contribute to if they thought that would be helpful.
47:28 Michael Kennedy: Yeah absolutely, it seems like something built-in that you just know you have, and it just works, I don't know.
47:37 Mike Driscoll: We need some advocates in the core developers to do that.
47:40 Michael Kennedy: Well, and that's why I said it's kind of like a boiling-the-frog problem. By the time you get into, where you're in a position you could make these kinds of changes, I feel a lot of people are like, well we're good enough, I got a requirements.txt, or hey didn't we just create pip file, didn't that solve... I feel like the pain, by the time you get far enough into it the pain is not high enough anymore. But if you go and teach a little one-week workshop at a high school, and the kids say, well how do I share this with my friends? Well, you can't share this with your friends, I'm sorry. I mean that's almost the answer, not quite. Can you tell your friend to go download Python from python.org, change the path, pip install the stuff, that is a large step if that's a 12-year-old kid. Right?
48:26 Mike Driscoll: Yeah.
48:28 Michael Kennedy: And that just seems not amazing. OK, so let's speak about amazing stuff though. Another person that you interviewed that had tons of interesting things to say was Alex Martelli.
48:37 Mike Driscoll: Yeah, Alex was awesome.
48:39 Michael Kennedy: Yeah he really mentioned the Stack Overflow thing, that talked about Python's incredible growth. Said the forecast is that Python will become the most popular programming language, with one of the most active developers groups behind it, by early 2019. That's pretty awesome.
48:59 Mike Driscoll: Yeah, I've been amazed, just looking at the statistics about Python over the last two or three years, and how it keeps skyrocketing higher and higher. That brings back the data science part, I think that is driving a lot of that interest is we have a really big chunk of that now. And if everyone wanted to do machine learning and AI and neural networks, just use Python, it's easy. And when you need to speed it up you can drop back down to a low-level language.
50:50 Mike Driscoll: I could totally see that. I mean, once you've used Python, why would I use another language unless I have to?
50:56 Michael Kennedy: OK, so let's see, I think one of the things, you're kind of getting short on time here. But one of the things I want to talk about that Alex focused on was what I'm going to classify as Python as a competitive advantage. I guess there's a couple of things. He ends up talking to these big Internet companies and they seem to use Java or some other core language, Go maybe, I don't know, as their main language. But they're acquiring these other companies, they're buying these little startups that are getting ahead and creating amazing things, he said a lot of them are buying these Python-based startups, and that's coming back in, and they're happening to sort of absorb Python into them. But he said what's really interesting about that is an example with say YouTube. So before I read your book I didn't know there was a thing called Google Video. Had you heard of that before?
51:51 Mike Driscoll: Yeah, I actually had. I used it briefly.
51:54 Michael Kennedy: Yeah, so Google Video was this thing that was a YouTube competitor or something to that effect. And Alex was at Google at the time, I think he still is. Anyway he said Google Video kept losing the feature battle to this tiny startup just a few miles away. Each time this little startup unveiled a new, successful feature that the customers liked a lot, our engineers would scramble to pick up something similar in a few months. They said they were basically amazed that these 20 guys were outclassing hundreds of engineers at Google. And of course he said well, the reason they were so productive was they were using Python and Google Video was using C++. And that's the thing that became YouTube.
52:36 Mike Driscoll: I became the advocate at my current job that we might not be able to use Python in the embedded application side, but we could do so much prototyping and other things with Python to help us figure out what works and what doesn't work before we go to the embedded side, that we would have be so much more productive, we'd have so many more new features or products. And it's just a slow, uphill battle when you have a lot of C++ people there.
53:00 Michael Kennedy: Well, and the people writing the embedded code they've probably been doing C++ for many many years, and why would they change, etc. etc., right?
53:07 Mike Driscoll: Yeah.
53:08 Michael Kennedy: Have you guys looked at MicroPython, is that relevant to you?
53:11 Mike Driscoll: I don't know, it might be might be relevant to the module side, I don't think it would be for our actual main bread-and-butter display, because that thing has to run really really fast with the number of operations it works with.
53:23 Michael Kennedy: OK, interesting. All right, I think there's still so many more things we could cover, I'm going to do one more, which I think is a really interesting and positive way to leave it. You also interviewed Barry Warsaw, who's been around the Python space for a long time. And one of the things that he talked about was Python's strength. And he said I think one of the real strengths of Python as a language is that it scales. And he's not talking about, well on this Linux server I add this many requests to it, it gets this sort of scale curve or whatever. He says, I'm talking about the human scale. So for example you can have one person write a little script on their laptop and solve a problem, and Python is good for that. You can have a little startup or open-source project with 10 to 15 people working on it and that's great. And you can have hundreds of people working on large projects, and it's still great. And it's sort of this idea of the elasticity of the architecture needed to build Python apps. They can start really small but they can stretch, without breaking the language, to things like YouTube.
54:26 Mike Driscoll: Yeah, that's something I've always liked about Python is you can start small and keep adding more and more to it. And while there's a little bit of hesitancy on my part for the new typing module, I can see that that would be really helpful for people who are used to statically typed languages and how that might help them get into the Python sphere, and get used to the language quickly. We just have to be careful they don't bring in too many of their idioms into the Python part.
54:56 Michael Kennedy: Yeah, it's going to be an interesting marriage. You know, the more I play with the type annotation stuff the more I like it, but I find I don't cover everything with it, I cover just like a certain certain few functions, and then the types sort of flow to the rest of it. And I'm pretty positive on it these days.
55:13 Mike Driscoll: I think it has ups and downs.
55:15 Michael Kennedy: Yeah, for sure. So I guess we'll leave it there. You know, I found your book to be really interesting. There's a lot of folks we didn't get a cover, and there's just a bunch of little insights like this, so hopefully people will check it out. Where can they find your book? It's on the Packt site I'm sure, it's on Amazon, things like that?
55:32 Mike Driscoll: Yeah, it's on Packt publishing, and Amazon, yeah.
55:35 Michael Kennedy: All right, so well done, I thought it was a good book, and I definitely enjoyed the insights that you got out of it.
55:41 Mike Driscoll: Thank you, I'm glad you enjoyed it.
55:43 Michael Kennedy: Yeah, how long did it take to write?
55:44 Mike Driscoll: I did the interviews over about nine months. And then you have to do editing after that, so we'll call it around 12, just to even it out. Even though it didn't take quite 12.
55:55 Michael Kennedy: It's a pretty serious project, though. Took some time.
55:58 Mike Driscoll: Yeah, it took a long time.
56:00 Michael Kennedy: Yeah, cool. Well hopefully you're proud of it, I think it came out good.
56:03 Mike Driscoll: Thank you.
56:03 Michael Kennedy: Yeah, you're welcome. I think we have time for the final two questions. So if you're going to write some Python code, what editor do you open up?
56:11 Mike Driscoll: I usually use Wingware's IDE. I prefer that 'cause I found their debugger to be the most useful for live debugging. But I also use PyCharm because we have to use it at work for the automated test stuff, and I can see why people like it. So I'm a little torn at this point.
56:30 Michael Kennedy: You want some kind of offspring of the two.
56:33 Mike Driscoll: Yeah, pretty much.
56:34 Michael Kennedy: Yeah, someday, who knows. Very cool, yeah I'm definitely have a fan of those as well. And then notable PyPI package or packages.
56:44 Mike Driscoll: So for me, I've been using ReportLab a lot because I'm writing a book on it, but I also used it a lot on my previous job. And it's just incredible what you can create with ReportLab. Pretty much any document you see or even this book, the Python interview book, I could probably create with ReportLab with a little bit of time. You can just create templates and it'll whip through that and generate whatever you want.
57:06 Michael Kennedy: That's awesome, yeah definitely checking it out.
57:07 Mike Driscoll: Yeah, it's cool. I like wxPython, which I've already mentioned. It's great for creating UIs across all three major platforms with very little or no changes across platforms. I've written code that works on all three without having to do anything. Just running the code.
57:23 Michael Kennedy: I definitely think it has the best overall experience for the cross-platform Python-based apps. It looks like it belongs on those platforms and it feels modern and nice.
57:36 Mike Driscoll: Yeah, it works really really well in my opinion.
57:38 Michael Kennedy: Very cool. All right Mike, well thank you so much for taking the time, and also for doing all these interviews with everyone. It was great to chat with you.
57:46 Mike Driscoll: It's good to chat with you, too. Thanks for having me.
57:49 Michael Kennedy: This has been another episode of Talk Python to Me. Our guest has been Mike Driscoll, and this episode has been brought to you by Linode and us over at Talk Python Training. Linode is bulletproof hosting for whatever you're building with Python. Get four months free at talkpython.fm/linode. That's L-I-N-O-D-E. Want to level up your Python? If you're just getting started try my Python Jumpstart by Building 10 Apps, or our brand new 100 Days of Code in Python. And if you're interested in more than one course, be sure to check out the Everything Bundle. It's like a subscription that never expires. Be sure to subscribe to the show. Open your favorite podcatcher and search for Python, we should be right at the top. You can also find the iTunes feed at /itunes, Google Play feed at /play, and direct RSS feed at /rss on talkpython.fm. This is your host, Michael Kennedy. Thanks so much for listening, I really appreciate it. Now get out there and write some Python code.