#6: Requests, PyCon, and Python's future Transcript

00:00 [music]

00:35 Hello and welcome to Talk Python To Me, a weekly podcast on Python- the language, the libraries, the ecosystem and the personalities. This is your host, Michael Kennedy. Follow me on Twitter where I am at @mkennedy, and keep up with the show and listen to past episodes at talkpythontome.com. This episode we'll be talking to Kenneth Rietz about Request the package, API design, PyCon and more.

01:00 Hello, Michael here. I've got a couple of news items for you. First, thank you for everyone who contacted or mentioned the show on Twitter where we are at @talkpython. I really appreciate the feedback and suggestions. It's also a nice way to let others know about the show and it makes a big difference for us.

01:17 Next, If you listen to the show and you have not yet subscribed- please do. Open up your favorite podcatcher and search for Python. The show should be right near the top. If it isn't, send me a note and I'll try to fix that. How about a T-shirt? I think I have a really cool design for a Talk Python themed T-shirt. If there is enough interest I'll actually create it. Visit talkpythontome.com, look in the footer and click "T-shirt" or just visit talkpythontome.com/home/shirt and press the button.

01:49 Finally, I want to say thank you to Tobias Macey and Chris Patti. In some crazy coincidence the Python community was without a proper podcast for almost a year, then Tobias and Chris and I independently set about creating new podcast and with all the set up and scheduling and million little items it takes to get a podcast off the ground, we ended up releasing in the very same week- amazing.

02:14 Check out their show called "Podcast__init " at podcastinit.com. Now, let's get onto speaking with the amazing Kenneth Rietz.

02:24 Kenneth, welcome to the show.

02:26 Thank you for having me Michael.

02:29 Yeah, this is great. You've built some amazing open source packages and I'm glad we have got the chance to share that with the world.

02:36 Absolutely, I'm really glad to be here.

02:38 Cool. So, before we get into sort of modern day, you know what are you working on and so on, I think people find it really interesting to see how everyone has come to Python and how it started in this whole community. So maybe you could just tell me a bit about how you got started in Python and what is your story?

02:56 I started doing Python when I was in undergrad, I went to George Mason University and they had a Python class and I had already been in a lot of programming languages throughout my life; you know, since I was a young kid I was learning like Basic, and C and play with Pascal and stuff like that, and Python really attracted me as a language. And, so I dropped out of school shortly after and just kind of started doing a full time and- here I am.

03:20 That's awesome. So you are really glad you just took Python and ran with it, and it is working out well?

03:26 Yeah, absolutely. I was doing PHP for a while and I did a little bit of Java and all kinds of stuff, but, Python is like really resonated deeply with me and what I am doing.

03:35 Yeah, that's cool. So the reason I reached out to you, the sort of I think I know you for if you will, is your Request Packages which is sort of ATP client library for Python. Maybe start- I suspect a lot of people know about Request but not everybody, so maybe you can give me like the 30 second elevator pitch, like what's wrong with urllib, why did you create this Request package, and go with that?

03:58 Yes. The Request is called "HTTP for humans", and originally when I created it it was not called that, it was actually called "HTTP for Python that doesn't suck". It was a very- much more negative title, and it kind of evolved as I was going along and I decided I didn't want to be putting down another projects and I wanted to be building up the community, so I called it "HTTP for humans".

04:19 And effectively the pitch is that if you are, you know, the internet is basically the way we communicate, you know, it's like, everything that we do, usually goes over the internet, when we are building tools today, the entire world revolves around the internet. And so, making calls to the internet, with Python should be as simple as the print statement; and if you use the built in tools that come with Python it's tremendously difficult. So I built Request to make it really great. And I believe I succeeded.

04:49 Yeah, I think you did a really fine job and whenever I needed some sort of HTTP client for Python I definitely used Request. I mean, if I think what I have to write, in order to do an authenticative request with urllib it's ridiculous...

05:05 Default password or a password manager with the default realm?

05:07 Yes, exactly. Specify the domain or whatever...

05:11 Take an opener and install the opener and then- it's a nightmare.

05:14 It's a total nightmare. So yeah, it's definitely a breath of fresh air.

05:18 Thank you.

05:19 So, you know, building libraries like that, you probably spend a lot of time thinking about APIs- and what makes a good API?

05:26 Yeah, Request is kind of like- when people ask me what my favorite libraries are I usually answer Request is my favorite one. Because it is. I've put a lot of thought in design into the way it works and it is kind of like my personal like totem of good design. But, there is a lot of- I am very opinionated about certain things in Python. Actually PyCon- Raymond Hettinger just did a great talk at PyCon, called "Beyond PEP 8 " and he kind of got into a lot of these things. He talked basically about like taking the basic data model in Python and you know, like you can make an object behave in any number of ways and using properties and stuff and that gets a really good starting place for someone that wants to know how to build good Python code.

06:13 Yeah, Ok that's really cool and you know, all those shows are recorded and we'll talk about that a little bit later, so I recommend people go and check them out. And just for the listeners who maybe are catching us later, or got to the recording a few weeks later, this is April 21st and PyCon was like a week ago?

06:30 Yeah, I just got back two or three days ago from PyCon.

06:34 Yeah, it was last week, wasn't it? That's right.

06:36 Yeah, I stayed actually too long, so… It was good, I was there for like 12 days in total, it was ridiculous.

06:42 Amazing, amazing. So before we get to that, you know, can you kind of say a few comments about what you think makes a good API design? I mean, obviously Request has it, but what do you think like the essential ingredients are?

06:56 Well, so I have a litmus test that I pitch people and it is basically that if you have to look at the documentation every time you use the module then you should either find a new module or build a new module.

07:06 That's a good plan.

07:08 Yeah, it's like surprisingly difficult when you are using code like if you have to- and I am not saying like if you pick it up for the first time in months then you are looking at the docs, there is certain code bases where like you just have the docs and you are looking at them more than you are looking at the code. And to me that's not, you know, that's a testament to a unergonomic design.

07:28 Yeah.

07:29 Or a really hard problem space. One of the two.

07:31 Right, it could be a super hard problem space but...

07:35 ... but even in those cases there is- you can build a layered API and you can have a nice human layers as well as the low level one.

07:43 Absolutely, just because your library might do a hundred things there is probably two or three very common use cases, maybe there should be some like higher level abstraction that you use for that?

07:53 Precisely.

07:54 And then you dig down, you know I am thinking of things like- we just spoke to Mike Bayer from SQLAlchemy, which was a great conversation, and with SQLAlchemy you've got two, literally two layers, there is like the SQLAlchemy Core which is purely about data access and there is the ORM which is kind of like the "Hey let's keep it a little bit easier but you have maybe less control over the performance."

08:20 It's funny you mentioned that because I am actually- while I work on new project there is like SQLAlchemy for humans. But we will see how it goes.

08:28 Yeah, excellent, excellent. So before we move off of Request, let me ask you just a couple of more questions: are there like really notable users or sort of like famous use case that you can talk about?

08:41 Yeah, well it's basically, it's quite ubiquitous, basically anyone doing Python and HTTP they are using Requests; the one that I am the most proud of is Amazon uses it for their internal API tools, so everything that runs UC2 and S3 and all AWS, every single API call goes through Requests which is astonishing, because it means like it is powering the entire internet.

09:04 Yeah, in some way you are like the layer beneath so much of the internet, that's really cool.

09:09 Yeah, I'm really proud of that.

09:11 Speaking of APIs, one of the things that really puts external pressure on APIs is the whole Async programming model. So you know, traditionally with things like Twisted but more modern with Async IO, does that, does Request have any support for that and what do you think about APIs in general adapting to these, Async stories?

09:34 So, the old way, sorry, like the kind of the current best practice way of doing it is using Gevent and what that does is Monkey patches the entire standard library and it adds basically callbacks, throughout every piece of blocking code in the standard library, in theory, I mean that's what it is designed to do, it doesn't do a perfect job but it gets the job done most of the time.

09:58 And then, what you do is you have an event loop and you say like, instead of like, you have a function that's like go get HTTP Request with requests and the blocks, you know, it's like a very blocky process, but if you do a Gevent you just say like run a hundred of these and it does them all concurrently, and it comes back, and that works really well.

10:16 In Python 3 I think in 3.2 they added it, they added this new library called Async IO and it's Guido's used to be called Tulip and it is the new asynchronous way of interacting with, you know doing asynchronous work specially over the network and it's still very new, it is what is considered to be a provisional package just in the standard library which means that it is not officially done. So there is like a small core that's still being iterated on and then there is going to be like an official HTTP client after that.

10:50 And I don't necessarily think that will replace the usage of requests over these blocking style services but it is too early to tell I would say, but right now you can use the Gevent and it works really well.

11:03 Yeah, ok, that's really cool. I don't think every API or every use case needs to use Async IO, I mean, it's one of these special cases scenarios, I'm running a web server and I want to support 20 000 requests on this machine, right, maybe you need to think about Async programming, but if you are just trying to do like a basic web scraper then maybe you do maybe you don't, it depends on what are you up to.

11:26 Yeah, I mean you can always scale as long as your performance is predictable, and unfortunately when you are using something like Gevent it makes a lot less predictable, but, you know, it depends on what you are doing really, it's one of those things you just have to try and see if it works for your project. A lot of times your code isn't compatible because of difference in dependency because that always blocks and it's kind of like this big hair ball you are getting yourself into. But, you know, if you have the right type of code and you use Gevent you can come close to like node performance benchmarks and go, it's really fantastic.

11:59 Yeah, that's really great.

11:59 [music]

12:08 Michael here, Thank you so much for listening to and spreading the word about "Talk Python To Me" I'm still looking to line up stable corporate sponsorships, but I wanted to tell you about a community based campaign I'm launching to allow listeners to directly support the show. We are running a Patreon campaign. Consider supporting us today at patreon.com/mkennedy and thanks for listening.

12:08 [music]

12:35 Ok, so, like we were saying before, you just came back from PyCon, and it amazing, you spent 12 days there, you've been like completely immersed, what are your thought?

12:44 The conference itself?

12:46 Yeah just the whole experience and what was the vibe of Python this year, and so on.

12:51 Well, so, PyCon is kind of like my way of measuring the year, every year. Like, you know, every year most people like think of the New Year as the start of the new year, but for me like PyCon is kind of like where I sit down and think about my life and all that stuff. And it's great, because I go to all these tech conferences all the time for work and speak at these conferences and events and I hang out with all these people, everywhere, and I get to see almost all of them at PyCon.

13:16 So for me it's the very social event, it's very work oriented but it's also very personal, and it's just kind of like, you know, this most intense week of my life. And I had a really great PyCon, I would say this was my best PyCon so far. The energy of the whole thing was really good, there were some things that were a little different this year even from last year, which was also in Montreal.

13:40 I don't know, I liked it though. It was good. To me, I personally spent a lot more time interacting one on one with people, instead of like doing these big group talks like what normally happens when you have like 20 people in the circle, and everyone is talking. I used to do that constantly, and then this year, I spent a lot more time like spending time with the people I wanted to see and that seemed to be really beneficial to me.

14:03 Yeah, that's really excellent. Some of my favorite conferences I've hardly gone to any of the talks, it's just you kind of go there and you spend your time with the people and the environment and you know maybe see the talks, maybe you don't.

14:13 Exactly. At PyCon that's known as "The hallway track".

14:16 Yeah, awesome. How many years you've been going?

14:19 Excuse me. Let's see, my first year was 2011, so I guess this is my fourth PyCon.

14:26 Excellent, excellent. Yeah, I couldn't go this year, my wife was also traveling and we have small kids and we would have been in trouble if we both left town, so...

14:33 Well, there is always next year.

14:34 Yeah it is coming to my hometown so it is looking good.

14:36 Yeah that will be great. Brandon Rhodes is coming this year so I am really excited to see what he does to it.

14:43 Yeah, that will be fantastic. So, can you tell me some of your- speaking of not going to the talks and then asking you questions, can you tell me what are some of your favorite sessions or topics that were covered at PyCon?

14:54 Well, I- didn't go to any talks.

14:58 Awesome, so you really enjoyed the hallway tracks, that's fantastic.

15:01 I have almost a no talk policy, because I go to a lot of conferences and they all kind of blur together eventually, but, and especially at PyCon, there are all already up online like you can watch them now, and so I just kind of I do that; and I am there to see the people. But I have watched a few already, and I really enjoyed Jacob Kaplan- Moss who is one of the runners of the Django project and a co-worker of mine did a really great keynote that I enjoyed, about kind of diversity in tech problem in that space; he went up on stage and and it was like "Everybody, I'm a mediocre programmer..." and I was like "That's pretty cool." And I was thinking like if I was to do that talk, I think I would do the opposite, I'm like "I'm an amazing programmer, and so are all of you" But, yeah, it was a good talk. So that's the only one I think I really set down and watched. Although, I watched Raymond's talk as well Beyond PEP 8 and that was I think fantastic, because people- I've never seen it so well solidified before how to write good Python code, so I recommend checking that out as well.

16:01 Ok, yeah, those both sound really awesome. So, because I couldn't go, I kind of you know, You Tube was my connection to PyCon this year, so I am like, all right, let me go through and sort of make a playlist for myself, at least just read in the description seems like kind of really valuable stuff, not to disparage anybody else's stuff that's not on the list of whatever, but these are just the ones that jumped at me, right? So, on You Tube I made a playlist you can find it at bit.ly/pycon2015mk all lower case. And I think how many videos I have there- I've got 29 videos apparently.

16:42 Nice.

16:43 Yeah, and so some of them that I thought are pretty cool, there was one called Bytes inside the machine, inside the CPython interpreter, and that's by Allison Captur, and I thought that was really cool because a lot of time you spend time so high in sort of the Python code like looking down and actually what happens when you run it. I thought that was pretty cool. Andrew Baker did one on the Demystifying Docker...

17:09 OOh yeah...

17:09 I think Docker is going to be massive. It's already kind of massive, but I think it's just at the beginning, you know.

17:15 I agree.

17:17 Yeah, and so I think understanding Docker is going to be really important, if you are doing anything involving like a server, which is pretty much everyone, not everyone, there is a plenty of people that just run scripts on their machines, for the most part, I think people need to understand that, so that's pretty cool, and then, Greg Ward did one on "How to write reusable code" which sounds a little bit like Beyond PEP 8.

17:39 Yeah...

17:40 And, let's see, a few more. Ryan Kelly did one on "PyPy.js"

17:44 Oh, I didn't hear about that. That sounds amazing.

17:46 Yeah, doesn't that sound cool? Like, you know...

17:50 Is this a Javascript interpreter...?

17:55 I don't remember exactly, I think...

17:58 I suspect it is. Because PyPy is in addition to being a Python interpreter it's also a platform for building other interpreted languages.

18:05 Right. It may well be. So maybe it's running, actually I should stop talking because I now know I've not told you as much as I know about this, other than watching, because that one I haven't seen. You know, I'd like get your thoughts on one that Guido did, just the topic, not because I know neither of us has seen it yet, is "Type hints"

18:22 Oh yes, I talked to Guido about Type Hints.

18:24 What do you think about type hints?

18:26 Ok, I don't care about them. It's interesting, so it seems like those are being built to servers at particular market that I am very disconnected from and I think that the people that are involved in the core development at Python are connected to and it seems to be like these people that are like building compilers and like, they are like taking Python code and turning it into something else, like optimization and stuff, and I think that that's the idea behind type hints. I don't think that's like a A use case,

18:56 It's definitely a use case. Another one that jumps to mind is if you are building editors like the Python guys,

19:02 Yeah, and that's seems really odd to me to build language features around that, but I'm not like, I'm not in that space, so I don't know what the real motivation is but it just seems a little odd to ma and everyone I talked to also doesn't care about typing hints. But it's also one of those things I don't think you'll need to care about if you are not going to use them probably

19:25 Yeah, I suspect that's right. You know, if you are coming from Java or CSharp or some other type full language, that might convince you to make the leap, you know, whereas if you have been doing Python for ten years you are like "We don't need this", you know.

19:38 Yeah, I am definitely in a different camp than Guido is, I am very much, I love Python 2.7 and I feel like I could use it for the rest of my life; I'm planning on like you know, fully embracing three when everyone else does. And so, there is a lot of, you know, like all this new development on these type hints and stuff doesn't sound as exciting to me as like making sure that we have a way to like ship stand alone binaries, I feel that will be a really great use of our time.

20:04 I'm not saying Guido should not be working on this, like, you should work on whatever you want to work on, and I'm really excited that he is putting the work into this, but I hope other people don't stop working on other things because of it and I don't think they will. That's not happening. I went to the language summit this year, and we had a really great conversation about the next year in Python and there was a lot of talk Guide did one on Type Hints, there was a lot of really great stuff that kind of came up.

20:28 Ok, that's cool. Can you talk about what else was at the language summit?

20:31 Let me think, I was only there for the second half, I did a talk on Requests because everyone is really interested in adding it to the standard library so I did a talk about the pros and cons of that and basically stated that Requests is critical piece of infrastructure for the Python community, because PEP relies on it and it's like the most popular Python package there is. So, and I have security updates, you know, when there is like a CVE that released for HTTP in OpenSSL and stuff, we have to make changes.

21:06 Python has a 16 months release cycle and we've released a version my maintainers in like 12 hours, so there is a big difference there. So if Request was a stable library it would be I think irresponsible. So we decided together as a group that Request is going to be officially recommended in the Python documentation instead of going into the standard library, which seems like a good compromise.

21:33 Right, that makes a lot of sense, and you know, HTTP being such a critical but vulnerable layer

21:42 And amorphic layer as well.

21:42 A- what? sorry?

21:44 Amorphic layer as well, it changes over time.

21:46 Yeah, absolutely. It's definitely changing and if that needs to be fixed, it needs to be fixed now, not in a year, so that's really interesting.

21:55 Yeah, I think that was a good takeaway.

21:57 It seems like a good takeaway... Cool. Well, I guess one more than I thought was worth calling out and maybe we'll do a show on this one is a guy named David Beazley, he did one on "Modules and packages".

22:09 Oh, very nice.

22:10 I did like a 3 hour workshop on it. And you think what is there to know about modules and packages for 3 hours, but actually there is a lot of really interesting stuff about creating maintainable sub-imports when you are building large packages and all sorts of cool stuff that I think is actually a lot of people with 22:28 that's also on that list of videos.

22:30 Yeah, David Beazley always does amazing work, he is kind of crazy.

22:36 Yeah, he was really good. That was fantastic. So what else would you like to get out into the community about Python and just let people know while you got a chance?

22:47 Let me think... I've been very concerned for a long time about the adoption of Python 3 and I still am, and we had some discussion about that at PyCon, as well and it's interesting, it seems though on certain platforms like where I work at Heroku we have about 10% in the last month or so deploy rate, for Python 3 which is really surprising and it seems to me that- and it's also if you look at the numbers Python 3 is starting to get a little bit more usage, you know, it's still extremely minimal and far behind what we want but the needle is moving.

23:19 And it's really fascinating because the only thing that I can tell has changed, is that Django when they made their last release they officially, they just used Python 3 by default. And that seems to have tremendous impact on people, on users of language in general, just having literally the quick start for Django in all the documentation is just Python 3 by default. So I think that's really interesting I think there is a lot to learn about that and just kind of think about that, and just be like you know, maybe if I share code samples to people they should be Python 3 not Python 2.

23:52 You know, maybe that will help make a big difference, you know, if we just like shift our default mode maybe they'll get us over this hump. Because I am concerned about the future of the language, I think we have a big marketing problem, I think that there is this project just being built no one wants, and no one is using it, and I think that it's just all about getting on the same page.

24:13 Yeah, I think that's really good idea, even if you are just going to blog something and put up a code snippet. If you started making that Python 3 more frequently, that would probably send some kind of signal to the world, you know.

24:22 Yeah, it's just so interesting to see, I mean, like the needle is actually moving and it is because Django changed the docs to use Python 3 and it is just, it blows my mind, that's not, it can't be that easy.

24:37 Well, nobody would have you know, thought that would make the difference. But, you know, sometimes things are unpredictable, right?

24:43 Exactly, yeah. So I am all in favor of a- You know, I've been for a long time if you talked to me in private I would always say that I am a Python 2 user for life, and I do kind of feel that way, but, the reality is that the Python community is valuable enough to me that I don't care what I am using as long as we are all using the same thing. So I think that we should all just kind of embrace the change and just see what happens.

25:08 You know, I think when you look at the actual code, it's sort of like modern Python 2 versus Python 3, there is not that much of a difference, it's kind of the libraries or the packages that you want to take along and it doesn't support Python 3 well this project isn't using Python 3...

25:22 It definitely depends on the style of project you are doing too, so if you are doing something that deals with bytes as text which I do a lot because I worked with HTTP, then Python 3 is the worst language in human history.

25:36 Yeah, it's true, because the string stuff is really pretty different, right?

25:41 Yeah, so if you have a byte string and you can't do anything with it anymore, it's just like a byte array of bytes, of integers. And so if you can't do like formatting and stuff, I think they added formatting back actually I think in 3.4, but they didn't add the .format method, they did the percentage the modules overwrite which is ridiculous, but so you can format strings again; but still you can't work with them very easily as text. And to me, Python 3 makes a lot of sense in a world where everything is unicode but we don't live in that world. Because like when you are sending stuff over the internet when you are writing it to disc, when you are sending an email it is all actually bytes, and so I really enjoyed the flexibility the Python 2 had, and that was my favorite part of the language and they kind of took that away from me, so I am deincentivized to use Python 3, personally, and if you look at like Armin Ronacher the guy who wrote Flask he wrote a really detailed blog post about the same problem. It sucks, but at the same time like if you- usually when you are interacting with that stuff you are dealing with libraries, we deal with it because we are library authors so if we solve the problem than it is ok for everybody else-

26:55 Right you guys are basically the firewall to that problem, right, you live literally at the network layer both of you and by the way we are going to have him on and do a show on Flask and other topics as well, pretty soon.

27:07 I am looking forward to listening that.

27:08 Yeah, but I think you probably feel it worse than a lot of people because you live ta the network boundary.

27:14 Yeah, to me adding Python 3 to Request was like one of the most painful things I've ever done, because I'm programmer, but definitely one of the most frustrating, if not the most frustrating. But I learned a lot from it too, it got better in some ways, like in Python 2 it was admittedly unclear if you were dealing with bytes or unicode. So, before in Request if you got back an HTTP response, and if I could decode it to unicode it would just be unicode, the content of it would be like response.content, and if it was the unicode to be unicode, the bytes to be bytes and they would like change.

27:52 And then when I went to Python 3 that API doesn't work anymore, it's not possible, so you had to split them up; so now there is .content which is always bytes and there is .text which is always unicode and that is a superior API design and something I should have done before. And using Python 3 helped me design better. So I do think it's a good- I think there is a lot of benefits to use Python 3, I think that to be a lot less data loss errors that people have but at the same time it's like really fucking frustrating.

28:24 Yeah, it makes a lot of sense. So, how do you think Python 3 would look different if it was 90% use case, do you think there would be different pressure applied to it or would it just be slightly more polished?

28:38 Do you mean if I- so like with Request I say that-

28:42 Well, imagine like we could snap our fingers and tomorrow like 90% of Python projects that are active today are using Python 3.

28:49 Got ya. Well, with that change what would the world look like-

28:52 Do you think Python 3 would be changed by that, or do you think people would just adapt, like are there things in Python 3 that just haven't had enough, you know, we needed this way pressure put upon it-

29:04 Oh, I see what you are saying. I feel like it is possible for us to have a 90% adoption rate now, it just takes a lot of time, and really the problem is that Python 3 right now isn't attractive enough to incentivize people to rewrite all their code to work with Python 3. If you have existing code you have to change it in order to get it to work. And, so Python 3 I do think to get to that point we either need a lot of time or we need to incentivize users by making something really attractive, that looks really nice. And there's been some talks of like what if we removed the GIL would that be attractive enough, you know, and stuff like that. There is a lot of different options and none of them are very clear at this point, but-

29:48 Right, that's a really interesting idea. And I do think you need like "the killer reason" to force a lot of people to go to the trouble of actually changing code, it is one thing to say we will do new projects in Python 3 but how many new projects do people do versus like we have had this whole infrastructure for ten years and now-

30:05 It's so funny, I went to go do that recently and it was so frustrating... I think I was using Flask, I don't think it has, they support Python 3 but I don't know if it is like proper Python 3 support. There was something I was running into, and it was just like- I wanted to throw my computer out the window. It's like- everything works great with Python 2, why should I have to deal with this? So I don't, and that's what everybody does, because they don't. So, it'll be interesting to see what happens, I have faith that the community will you know, whatever happens is intended to happen, that will be for the best.

30:35 Right.

30:38 Yeah.

30:38 I agree, I think it's just a little unhealthy that there is such like the fore progress is kind to be put on the language on Python 3 but a lot of the real work is still being done in Python 2, and it just seems like that can't be a really positive thing.

30:51 Yeah, and some of the biggest problem is that we stand up and preach you know PEP 20 all the time and one of the best ones in there is that should be one and only obvious way to do it. And you go to Python.org, and you wanna go download Python and you have to pick- Python 2 or Python 3, and it is like what now? It's confusing for new users, and so you have a lot of new users that are using Python 3 and they are actually really adamant about Python 3, so if you go onto like Reddit or Hacker News and you see anything about Python 2, you will have all these people that are like bashing and being like "why aren't you using Python 3, man, that's so old." And it's like "who are these people?" because it's like I go to conferences and I meet all the Python developers that go to conferences and those people are not at the conferences. It took me a while to figure it out, it's the new people, because when you are new, and you are learning the code and it's like your hobby, you are like online all day, like typing on Hacker News or Reddit.

31:48 You are not busy. You can go complain.

31:50 Yeah, exactly, so the people who work for companies, they use it too, you know. The new kids are starting to use Python 3, and I would like for that to be the same community and not 2 different communities, and it appears as though it is time there are two different communities, so I am just, I just want us to be unified instead of separated.

32:08 Yeah, well that makes sense, I mean, if you are starting a new project you have the flexibility, but if you are working on older technology that was written before Python 3 even came out, there is maybe no choice- that's what you are going to be using. A lot of the companies I visited there is still Python 2 as well of course...

32:27 Yeah, I don't know any major companies that are using Python 3 at all, but I'm sure there are some.

32:32 Yeah, there is a few I have talked to but I can't- I don't want to say because I might get it wrong and conflict them with other companies I spoke to, and so...

32:41 There is a few but you are right, it is not a large portion.

32:44 Yeah. But I am hopeful that everything will unfold as it is supposed to. Did you guys talk about that at the Language Summit?

32:51 It was not brought up as much as you would think. It was more- so there was one great talk, it was by Jacob again, and he went up on stage and he was kind of just showing some numbers that he has access to, different adoption rates and it was interesting, so basically what he said was that, you know, if we are to treat this like- No it was Glyph, Glyph did a talk, sorry.

33:18 Jacob did a great talk on this same subject but Glyph did a fantastic talk- the author of Twisted. And he got up there and he was saying "Look at this problem space that we have. We all knew that it's a problem, right", and everyone agrees, and he moves forward and he was like, ok, so the problem is that we don't have anything to measure to see if we are improving on this problem, so the first thing that we need to do is decide what does-when this problem is solved what does that look like, right.

33:44 So to do that you have to decide the measure something, so the ideal would be like 20%, so example metric would be like 50% of Python downloads of package downloads, or like majority of the downloads from Python.org, for Python versions would be Python 3 like you know, let's pick something and try to optimize it. Basically. And that was very much like what he was going for he was like let's not leave this room until we decide on the metric together.

34:13 Or at least decide that we need a metric, right. And so, it seems that everyone agreed that we needed a metric, we didn't even come close to deciding what it was, but it was a good discussion. And I feel like there is a lot of merit to that type of thinking. But you know, treating this like a software project instead of like a hobby project basically.

34:31 Right, well yeah, let's make this a data different problem that we can play grass and science and statistics show whatever, right?

34:38 Exactly, so I think that's going to happen this year.

34:42 Yeah, that's cool, like one talk comes to mind is like track the number of Requests from PEP2 versus PEP3...

34:49 Precisely.

34:49 Not even the download, just the number of like how much are people using this version versus that version, right, these kinds of things, you should totally be on the track.

34:57 Yeah and so there is some really great blog post that have come out about this Donald Stufft just did one today, his domain is caremad.io and he just did this big blog post breaking down all the PyPy numbers so I highly recommend checking that out, because it shows you- it's weird, web people are adopting Python 3 faster than the rest of the Python community. By about 4%, which is pretty interesting.

35:21 That is really interesting. I think that is something I would have- not predicted, but would say yeah that seems like you told me something true, just hearing it out of the blue is -because you have so much more control over the server, you managed a thing, and on the other hand if you are shipping it out to the world you don't know where it is going to land.

35:39 Exactly. There is a scientific Python community. And I think that they are having smaller amounts of Python 3 adoption but I have to learn more.

35:46 Yeah ok interesting like the iPython guys and so on...

35:49 Yeah, I continue them and they support Python 3 with like anaconda, but I think that is like a new thing that people are still adjusting to. Because the scientists- if you are a data scientist you are probably not a programmer, and you are someone who can use these tools, and so you do not care if it is Python 2 or 3, you just want to get your job done basically. So I think Python 3 has a lot of political buy basically.

36:10 Cool. So you said you work at Heroku, right?

36:12 I do, I am the Python Overlord at Heroku.

36:15 It's good to be the Overlord.

36:17 I enjoy it.

36:18 So what are you doing there, what do you guys do with Python at Heroku?

36:21 So, I am basically, Heroku supports a number of languages, we have like Ruby, Java, Node, PHP support, Python, and I am just in charge of everything related to Python. So I read a lot of documentation, I help customers to improve the product and if you like get push Heroku and like you have a Python app then all the code that runs at is me. And then I go and speak to conferences, and I'm at the podcasts... This is technically work I guess...

36:52 Yeah this is work. That's awesome, yeah, it's really cool to be kind of out there it's the community front for Heroku

37:03 Yeah, I enjoy it.

37:04 Do you guys use it a lot internally like can you talk about what technology you have for yourselves?

37:08 Yeah, so there is a couple of different teams internally that use a lot of Python. There are specific spots in Heroku that are just kind of infrastructure, that are written in Python, so if all of our postgre servers continuously backed up the wall logs get continuously backed up to S3 and that's written with the tool called "wally" which we have open sourced; and if you go to our GitHub you will see it is written in Python.

37:34 And a couple of different things, if you do a git pushed to us over SSH I believe all of those are received by a Twisted which is pretty cool, but there is a couple different teams that also use a lot of Python- there is the Heroku connect team which gives you a postgress database and that's like a Django app, and there is a couple of other teams that- a lot of tools that are built with Python.

38:04 Cool, that sounds really interesting.

38:06 It's pretty good.

38:08 So before we go let me ask you a question I asked all my guests- in addition to Requests, what is your favorite package on PyPy?

38:17 Uh, do I have to pick one?

38:19 Well, give me a couple that you really like, mostly what just draws peoples' attention to stuff that's cool.

38:25 One of my favorites is called "Docopt" and what- so if you want to do a command line tool you have to parse command line arguments, and to do that is not very fun, there is argparse and optparse that are built into the standard library and they are both terrible. I wrote something that makes it a little easier but this guy killed it- so what he did was- it turns out that the help string that's printed out when you do like command-h on the command line, you see those little brackets and stuff that like tells you all the different options, that is an Ansi standard, it turns out.

39:02 And so what he does is you just you write the help script, like you write the help page and you say all the different options and you write the little arguments, which are optional, which ones are positional and stuff and from that he completely writes all the parsing code for you. So you just give it a string and he will just just say give me my arguments and it gives you a dictionary with all the stuff and the bulletins and the true/false and it is incredible, I love it.

39:29 That's really fantastic.

39:31 Yeah, and it's been imported to like every language, but I think Python is the default one so it is- I'm a big fan of that project.

39:39 Right, that's cool. I feel like I reinvent the wheel when I go there and I am just like ok start from scratch and so that's great to bring that in.

39:45 Yeah. So another one that I am excited about but haven't used yet is called "click" by Armin Ronacher, I am sure you will talk to him about that when you do your podcast with him. But, a couple of years ago I wrote a library for myself called "clint" and clint was a command line application tools, set of utilities basically, I had a bunch of cool stuff in there, like prompting for yes and no, parsing arguments, writing files to the home directory on every type of computer. My favorite module that I had in there was called "eng.join" so you would give it a list of strings and then it would join them properly with like an oxford comma and a conjunctions-

40:32 Nice.

40:32 Yeah, it was like really cool code, but it kind of I need to separate all that stuff out because it's kind of a dead project I haven't used in years. And click is Armin's approach to the same problem, it doesn't do all the same stuff, but it is like you know, Armin does amazing code, and so this is a project that just is like... I am really excited about that.

40:55 Cool. Excellent. Those are really nice. If I wanted to get started with Heroku, as a Python let's say I am writing a Django app or Flask or something, what do I do?

41:05 Well, you go to Heroku.com and you sign up for account and it should tell you what to do.

41:11 Awesome. Ok, so everyone check that out-

41:16 And if you have any questions about Heroku and Python fell free to reach out to me, tweet me or whatever.

41:20 Yeah, very cool. Any final shout outs you want to give?

41:23 Off the top of my head? I am going to shout out to you. For making this awesome podcast.

41:29 Oh thank you very much, I appreciate it.

41:30 You are welcome. The world needs more Python podcasts.

41:33 I agree.

41:34 You are doing good work.

41:36 My original goal was not to start a Python podcast but just to talk to some of them, and I found that they all had their last show like a year ago, and I was like wait a minute, that can't be right- I looked and looked and looked and it is sadly right.

41:49 Mine included. I have a one episode podcast, but it is great, it had 6000 listens.

41:55 That's awesome. Where do people find it?

41:58 It's called "Import this". It's Alex Gaynor and I and we two other episodes recorded it just a matter of editing time, I would like to get sponsor so if we got a sponsor it would be easier to...

42:11 It would boost you back into it....

42:12 Yeah it just it took me 8 hours to add a first episode because I was doing like radio lab quality so I need to settle for lower quality or ideally I would pay a company to edit it for me but... I don't know. We'll see.

42:28 Exactly, that would be nice. I think editing is actually the most work.

42:32 It is, yeah. It's ridiculous. So we'll get back to it eventually.

42:38 Definitely, you should. All right, Kenneth thank you so much for being on the show, it's been really interesting and I especially appreciate the insight into the language summit and stuff like that.

42:47 Absolutely, thank you for having me.

42:47 Yeah, you bet. And I will talk to you later.

42:51 Have a good one.

42:51 This has been another episode of "Talk Python To Me" Today's guest was Kenneth Rietz, thanks Kenneth. Remember, you can find the links from the show at talkpythontome.com/episodes/show/6. And if you are feeling generous, please check out our Patreon campaign at patreon.com/mkennedy or work with your company to encourage them to sponsor an episode. Be sure to subscribe to the show, visit the website and choose subscribe in iTunes or grab the episode RSS feed and drop it into your favorite podcatcher. You can find both of these in the footer of every page. This is your host, Michael Kennedy, thanks for listening.

43:30 [music]