#6: Requests, PyCon, and Python's future Transcript

00:00 Talk Python to Me, episode number six, with guest Kenneth Wright, recorded Friday, April 17th, 2015.

00:09 Hello and welcome to Talk Python to Me, a weekly podcast on Python, the language, the libraries, the ecosystem, and the personalities.

00:42 This is your host, Michael Kennedy.

00:44 Follow me on Twitter where I'm @mkennedy, and keep up with the show and listen to past episodes at talkpythontome.com.

00:51 This episode will be talking to Kenneth Wright about requests, the package, API design, PyCon, and more.

00:59 Hello, Michael here.

01:02 I've got a couple of news items for you.

01:04 First, thank you to everyone who contacted or mentioned the show on Twitter,

01:08 where we're at Talk Python.

01:10 I really appreciate the feedback and suggestions.

01:12 It's also a nice way to let others know about the show, and it makes a big difference for us.

01:17 Next, if you're listening to the show and you have not yet subscribed, please do.

01:21 Open up your favorite podcatcher and search for Python.

01:24 The show should be right near the top.

01:26 If it isn't, send me a note and I'll try to fix that.

01:29 How about a t-shirt?

01:31 I think I have a really cool design for a Talk Python themed t-shirt.

01:35 If there's enough interest, I'll actually create it.

01:38 Visit talkpythontome.com, look in the footer, and click t-shirt, or just visit talkpythontome.com slash home slash shirt and press the button.

01:49 Finally, I want to say thank you to Tobias, Macy, and Chris Patty.

01:53 In some crazy coincidence, the Python community was without a proper podcast for almost a year.

01:59 Then Tobias and Chris and I independently set about creating new podcasts.

02:04 And with all the setup and scheduling and a million little items it takes to get a podcast off the ground,

02:10 we ended up releasing in the very same week.

02:13 Amazing.

02:14 Check out their show called Podcast Under Under a Knit at podcastinit.com.

02:19 Now, let's get on to speaking with the amazing Kenneth Wrights.

02:23 Kenneth, welcome to the show.

02:26 Thank you for having me, Michael.

02:28 Yeah, this is great.

02:30 You've built some amazing open source packages, and I'm glad we got a chance to share that with the world.

02:35 Absolutely.

02:36 I'm really glad to be here.

02:38 Cool.

02:39 So before we get into sort of modern day, what are you working on and so on,

02:43 I think people find it really interesting to see how everyone has come to Python

02:48 and got started in this whole community.

02:51 So maybe you could just tell me a bit about how you got started in Python and what's your story?

02:55 I started doing Python when I was in undergrad.

02:58 I went to George Mason University, and they had a Python class.

03:02 And I had already been doing a lot of programming languages throughout my life.

03:06 You know, since I was a young kid, I was learning like basic and C and play with Pascal and stuff like that.

03:11 And Python really attracted me as a language.

03:13 And so I had dropped out of school shortly thereafter and just kind of started doing it full time.

03:19 And here I am.

03:19 That's awesome.

03:21 So you're really glad you just took Python and ran with it, huh?

03:24 It's working out well?

03:25 Yeah, absolutely.

03:26 I was doing PHP for a while, and I did a little bit of Java and all kinds of stuff.

03:31 But Python has really resonated deeply with me and what I'm doing.

03:34 Yeah, that's cool.

03:36 So the reason that I reached out to you, the sort of thing that I know you for, if you will, is your request packages, which is sort of an HTTP client library for Python.

03:45 Maybe start, I suspect a lot of people know about requests, but not everybody.

03:51 So maybe give me like the 30-second elevator pitch.

03:53 Like, what's wrong with URL lib?

03:55 Why did you create this request package and go with that?

03:58 Yeah, so request is called HTTP for humans.

04:01 And originally when I created it, it was not called that.

04:04 It was actually called HTTP for Python that doesn't suck.

04:07 It was a much more negative title.

04:10 And it kind of evolved as I was going along.

04:13 And I decided I didn't want to be putting down other projects, but I wanted to be building up the community.

04:17 So I called it HTTP for humans.

04:19 And effectively the pitch is that if you're, you know, the internet is basically the way we communicate.

04:25 You know, it's like everything that we do usually goes over the internet when we're building tools today.

04:30 You know, the entire world revolves around the internet.

04:33 And so making calls to the internet with Python should be as simple as the print statement.

04:39 And if you use the built-in tools that come with Python, it's tremendously difficult.

04:43 So I built requests to make it really great.

04:47 And I believe I succeeded.

04:48 Yeah, I think you did a really fine job.

04:51 And, you know, whenever I've needed some sort of HTTP client for Python, I definitely use requests.

04:55 I mean, you know, if I think what I have to write in order to do an authenticated request with URL lib, you know, it's ridiculous.

05:04 The default password or password manager with a default realm?

05:07 Yes, exactly.

05:08 So you have to specify the domain or whatever it is.

05:11 Take an opener and then install the opener.

05:13 And then it's a nightmare.

05:14 It's a total nightmare.

05:15 So, yeah, it's definitely a breath of fresh air.

05:18 Thank you.

05:19 So, you know, building libraries like that, you probably spend a lot of time thinking about APIs and what makes a good API.

05:26 Yeah.

05:27 Request is kind of like my – if people ask me what my favorite libraries are, I usually answer request is my favorite one.

05:33 Because it is.

05:35 I've put a lot of thought and design into the way it works.

05:38 And it is kind of like my personal, like, you know, totem of good design.

05:44 And, yeah, there's a lot of – I'm very opinionated about certain things in Python.

05:48 Actually, PyCon – Raymond Hediger just did a great talk at PyCon called Beyond Pep8.

05:54 And he kind of got into a lot of these things, which was – it's all basically about, like, taking the basic data model in Python.

06:01 And, you know, like, you can make an object behave in any number of ways and using properties and stuff.

06:07 And I think it's a really good starting place for someone that wants to know how to build good Python code.

06:13 Yeah.

06:13 Okay.

06:13 That's really cool.

06:14 And, you know, all those shows are recorded.

06:16 And we'll talk about that a little bit later.

06:17 So I recommend people go check that one out.

06:20 And just for the listeners who maybe are catching this later or got the recording after a few weeks later, you know, this is April 21st.

06:28 And PyCon was – it was like a week ago?

06:30 Yeah.

06:31 I just got back two or three days ago from PyCon.

06:34 Yeah.

06:34 It was last week, wasn't it?

06:36 That's right.

06:36 Yeah.

06:37 Well, I stayed extra long for the sprints.

06:38 So it was good.

06:40 I was there for, like, 12 days in total.

06:41 It was ridiculous.

06:43 Amazing.

06:43 Amazing.

06:43 Amazing.

06:44 So before we get to that, you know, can you kind of say a few comments about what you think makes good API design?

06:51 I mean, obviously, Request has it.

06:54 But what do you think, like, the essential ingredients are?

06:56 Well, so I have a litmus test that I pitch people.

06:58 And it's basically that if you have to look at the documentation every time you use the module, then you should either find a new module or build a new module.

07:05 That's a good plan.

07:07 Yeah.

07:08 Good recommendation.

07:09 It's, like, surprisingly difficult when you're using code.

07:11 Like, if you have to...

07:12 I'm not saying, like, you know, if you pick it up for the first time in months, then looking at the docs.

07:16 But, like, there are certain code bases where, like, you just have the docs and you're looking at them more than you're looking at the code.

07:22 And to me, that's not, you know, that's a testament to an unergonomic design.

07:28 Yeah.

07:29 Or a really hard problem space, one of the two.

07:31 Right.

07:32 It could be a super hard problem space.

07:34 But...

07:35 But even in those cases, there's, you know, you can build a layered API and, you know, you can have a nice human layer as well as a low-level one.

07:42 Absolutely.

07:43 Just because your library might do 100 things, there's probably, you know, what, two or three very common use cases.

07:49 Maybe there should be some, like, higher-level abstraction that you use for that.

07:53 Precisely.

07:54 And then you dig down.

07:55 You know, I'm thinking of things like we just spoke to Mike Bayer from SQLAlchemy.

08:01 Very nice.

08:01 And with SQL...

08:02 Yeah, which is a great conversation.

08:04 And, you know, with SQLAlchemy, you've got two, literally two layers you can think of, right?

08:09 There's, like, the SQLAlchemy core, which is purely about data access.

08:12 Then there's the ORM, which is kind of like the, hey, let's keep it a little bit easier, but you have maybe less control over the performance.

08:19 Or so on.

08:19 It's funny you mention that, because I actually want to work on a new project that's like SQLAlchemy for humans.

08:24 But we'll see how it goes.

08:28 Yeah, excellent.

08:29 Excellent.

08:29 So before we move off of requests, let me ask you just a couple more questions on it.

08:34 Are there, like, really notable users or sort of, like, famous use cases that you can talk about?

08:40 Yeah, well, it's basically, it's quite ubiquitous.

08:44 Basically, if anyone's doing Python and HTTP, they are using requests.

08:47 The one that I'm the most proud of is Amazon uses it for their internal API tools.

08:53 So everything that runs EC2 and S3 and all of AWS, every single API call goes through requests, which is astonishing, because that means, like, it's powering the entire internet.

09:03 Yeah.

09:04 And you, in some way, you are, like, a layer beneath so much of the internet.

09:08 That's really cool.

09:09 Yeah, I'm really proud of that.

09:11 Speaking of APIs, you know, one of the things that really puts external pressure on APIs is the whole async programming model.

09:19 So, you know, traditionally with things like Twisted, but, you know, more modern with AsyncIO, does requests have any support for that?

09:29 Or what do you think about, you know, APIs in general adapting to these async stories?

09:34 So the old way, sorry, like, kind of the current best practice way of doing it is using gevent.

09:39 And what that does is it monkey patches the entire standard library.

09:44 And it adds basically callbacks throughout every piece of blocking code in the standard library, in theory.

09:53 I mean, that's what it's designed to do.

09:55 It doesn't do a perfect job, but it gets the job done most of the time.

09:58 And then what you do is you have an event loop, and you say, like, instead of, like, you know, you have a function that's, like, go get this HTTP request with requests, and that blocks.

10:07 You know, it's, like, a very, like, blocky process.

10:10 But if you do it with gevent, you can just say, like, run 100 of these, and it does them all concurrently, and it comes back.

10:15 And it works really well.

10:16 Then in Python 3, I think in 3.2 they added it.

10:20 They added this new library called AsyncIO.

10:23 And it's Guido's, it used to be called, known as Tulip.

10:26 And it is the new asynchronous way of interacting with, just, you know, doing asynchronous work, especially over the network.

10:33 And it's still very new.

10:35 It's what is considered to be a provisional package within a standard library, which means that it is not officially done.

10:43 So there's, like, a small core that's still being iterated on, and then there's going to be, like, an official HTTP client after that.

10:50 And I don't necessarily think that will replace the usage of requests or these blocking-style services, but it's too early to tell, I would say.

10:58 But right now you can use gevent, and, like, you get the best of both worlds, and it works really well.

11:03 Yeah, that's, okay, that's really cool.

11:05 I don't think every API or every use case needs to use AsyncIO.

11:09 I mean, it's one of these special case scenarios.

11:13 I'm running a web server, and I want to support, you know, 20,000 requests on this machine, right?

11:19 Maybe you need to think about async programming.

11:21 But if you're just writing, like, a basic web scraper, then maybe you do, maybe you don't.

11:25 It depends what you're up to.

11:26 Yeah, I mean, you can always scale as long as your performance is predictable.

11:30 And unfortunately, when you're using something like gevent, it makes things a lot less predictable.

11:34 But, you know, it depends on what you're doing, really.

11:37 It's one of those things where you just have to try it and see if it works for your project.

11:42 A lot of times, your code isn't compatible because of different C dependencies, because

11:45 that always blocks.

11:45 And, you know, it's kind of like this big hairball you're getting yourself into.

11:49 But, you know, if you have the right type of code and you use gevent, you can, you know,

11:55 come close to, like, node performance benchmarks and go.

11:58 It's really fantastic.

11:59 Yeah, that's really great.

12:00 Michael here.

12:08 Thank you so much for listening to and spreading the word about Talk Python TV.

12:13 I'm still looking to line up stable corporate sponsorships, but I wanted to tell you about

12:17 a community-based campaign I'm launching to allow listeners to directly support the show.

12:21 We are running a Patreon campaign.

12:23 Consider supporting us today at patreon.com slash mkennedy.

12:26 And thanks for listening.

12:27 Okay, so like we were saying before, you just came back from PyCon.

12:40 And you, it's amazing.

12:41 You spent 12 days there.

12:42 You've been, like, completely immersed.

12:44 What are your thoughts?

12:44 On the conference itself?

12:46 Yeah, just the whole experience and what was the vibe of Python this year and so on.

12:51 Well, so PyCon is kind of like my way of measuring the year every year.

12:56 Like, you know, every year, like, most people, like, think of the new year as the start of

13:00 their new year.

13:00 But for me, like, PyCon is kind of like where I, like, you know, sit down and think about

13:05 my life and all this stuff.

13:06 And it's great because, like, I go to all these tech conferences all the time for work and speak

13:11 at these conferences and events and I hang out with all these people everywhere.

13:14 And I get to see almost all of them at PyCon.

13:16 So for me, it's a very social event.

13:18 It's a very, it's very work-oriented, but it's also very personal.

13:22 And it's just kind of like, you know, this most intense week of my life.

13:25 And I had a really great PyCon.

13:28 I'd say this is my best PyCon so far.

13:30 The energy of the whole thing was really good.

13:33 There were some things that were a little different this year, even from last year, which was also

13:37 in Montreal.

13:37 I don't know.

13:40 I liked it, though.

13:40 It was good.

13:42 To me, I personally spent a lot more time interacting one-on-one with people instead

13:47 of, like, doing these big group talks, like kind of normally happens where you get, like,

13:50 20 people in a circle and everyone's talking.

13:52 I used to do that constantly.

13:54 And then this year, I spent a lot more time, like, you know, spending time with the people

13:59 I wanted to see.

13:59 And that seemed to be really beneficial to me.

14:01 Yeah, that's really excellent.

14:03 Some of my favorite conferences, I've hardly gone to any of the talks.

14:06 It's just you kind of go there and you spend your time with the people and in the environment.

14:11 And, you know, maybe you see the talks, maybe you don't.

14:13 Exactly.

14:13 At PyCon, that's known as the hallway track.

14:15 Yeah, awesome.

14:17 How many years have you been going?

14:19 Excuse me.

14:21 Let's see.

14:22 This was, my first one was 2011.

14:24 So, I guess this is my fourth PyCon.

14:26 Excellent, excellent.

14:27 Yeah, I couldn't go this year.

14:28 My wife was also traveling and we have small kids and we'd be in trouble if we both left

14:32 town.

14:32 Well, there's always next year.

14:33 Yeah, and it's coming to my hometown, so it's looking good.

14:36 Yeah, that'll be great.

14:37 Brandon Rhodes is running this year, so I'm really excited to see what he,

14:41 what he does to it.

14:42 Yeah, that'll be fantastic.

14:43 So, can you tell me what some of your, you know, speaking of not going to the talks and

14:48 then asking you the opposite question, can you tell me what some of your favorite sessions

14:52 or topics that were covered at PyCon?

14:54 Well, I, I didn't go to any talks.

14:58 Awesome.

14:59 So, you really enjoyed the hallway track.

15:00 That's fantastic.

15:01 Yeah, I have almost a no-talk policy because I go to, I go to a lot of conferences and they

15:05 all kind of blur together eventually.

15:06 But, and especially at PyCon, they're all already up online.

15:09 Like, you can watch them now.

15:11 And so, I just kind of, you know, I do that.

15:13 And I, I'm there to see the people.

15:15 But I have watched a few already.

15:17 And I, I really enjoyed, Jacob Kaplan Moss, who is one of the runners of the Django

15:22 project.

15:22 And a coworker of mine did a really great keynote that I enjoyed, about just kind

15:28 of like the diversity in tech, problem in that space.

15:31 He, he went up on stage and was like, everybody, I'm a mediocre programmer.

15:35 And I was like, that's pretty cool.

15:37 And I was thinking like, if I was to do that talk, I think I would do the opposite.

15:40 I'm like, I'm an amazing programmer and so are all of you.

15:43 But, but, yeah, it was a good talk.

15:47 And so, that's the only one I think I've really sat down and watched.

15:49 Although I watched Raymond's talk as well on beyond pepe.

15:52 And that was, I think, fantastic for, cause people, I've never seen it so well solidified

15:57 before how to write good Pythonic code.

15:59 So, I'd recommend checking that out as well.

16:01 Okay.

16:02 Yeah.

16:02 Those, those both sound really awesome.

16:03 So, because I couldn't go, I kind of, you know, YouTube was my connection to PyCon this

16:09 year.

16:09 So, I'm like, all right, well, let me go through and sort of make a playlist for myself and what

16:14 I think, you know, at least just reading the descriptions seems like, you know,

16:19 kind of essential or the real valuable stuff.

16:22 Now, you know, not to disparage anybody else's stuff that's not on the list or whatever, but

16:26 these are just the ones that jumped out at me.

16:27 Right.

16:28 Yeah.

16:28 So, on YouTube, I made a playlist.

16:30 You can find it at bit.ly slash PyCon 2015 MK, all lowercase.

16:38 And I think, how many videos do I have there?

16:40 I've got 29 videos, apparently.

16:42 Nice.

16:43 Yeah.

16:44 And so, you know, some of them that I thought were pretty cool was there's one called Bytes

16:48 Inside the Machine, Inside the CPython Interpreter.

16:51 And that's by Allison Captur.

16:53 And I thought that was really cool because a lot of time, you know, you spend time so high

16:58 in sort of the Python code, like looking down at actually what happens when you run it.

17:03 I thought that was pretty cool.

17:04 Andrew Baker did one on demystifying Docker.

17:08 Oh, yeah.

17:09 And I think Docker is going to be massive.

17:11 You know, it's already kind of massive, but I think it's just at the beginning, you know?

17:15 I agree.

17:16 Yeah.

17:17 And so, I think understanding Docker is going to be really important if you're doing anything

17:21 involving like a server.

17:22 Which?

17:23 It's pretty much everyone.

17:25 Not everyone.

17:26 There's plenty of people that just run, you know, scripts on their machines.

17:30 But for the most part, I think people need to understand that.

17:33 So, that's pretty cool.

17:33 And then Gary Ward did one on how to write reusable code, which sounds a little bit like

17:37 beyond PEPT-8.

17:38 Yeah.

17:39 Yeah.

17:40 And let's see.

17:41 A few more.

17:41 Ryan Kelly did one on PyPy.js.

17:43 Oh, I didn't hear about that.

17:45 That sounds amazing.

17:46 Yeah.

17:46 Doesn't that sound cool?

17:47 Like, you know, running Python.

17:49 Is this a JavaScript interpreter?

17:54 I don't remember exact.

17:56 I think it.

17:57 I suspect it is.

17:58 Because PyPy is, in addition to being a Python interpreter, it's also a platform for building

18:03 other interpreted languages.

18:04 Right.

18:05 It may well be.

18:06 So, maybe it's running.

18:07 Yeah.

18:07 I actually should stop talking because I now know.

18:10 I've now told you as much as I know about it.

18:11 Other than watching it because that one I haven't seen.

18:14 You know, I'd like to get your thoughts on one that Guido did.

18:17 Just the topic, not because I know neither of us have seen it yet, is type hints.

18:21 Oh, yes.

18:22 I've talked to Guido about type hints.

18:24 What do you think about type hints?

18:25 I don't care about them.

18:27 But I know he's really excited about them.

18:29 And it's interesting.

18:31 So, it seems like those are being built to serve this particular market that I am very

18:35 disconnected from.

18:35 And I think that the people that are involved in the core development of Python are connected

18:39 to.

18:40 And it seems to be like these people that are like building compilers and like, you know,

18:44 they're like taking Python code and turning it into something else for like optimization

18:48 and stuff.

18:49 And I think that that's the idea behind type hints.

18:53 I think that's like a use case.

18:55 It's definitely a use case.

18:57 Another one that jumps to mind is if you're building editors like the PyCharm guys.

19:01 Yeah.

19:02 And that seems just really odd to me to build, you know, to build language features around

19:08 that.

19:09 But because, but I'm not like, I'm not in that space.

19:12 So, I don't know what the real motivation is.

19:15 But it just seems a little odd to me.

19:16 And everyone I talk to also doesn't care about type hints.

19:20 But it's also one of those things that I don't think you'll need to care about if you're not

19:23 going to use them probably.

19:25 Yeah, I suspect that's right.

19:26 You know, if you're coming from Java or C# or some other typeful language, that might

19:31 convince you to make the leap.

19:33 Yeah.

19:33 Whereas if you've been doing Python for 10 years, you're like, we don't need this, you

19:37 know?

19:38 Yeah, I'm definitely in a different camp than Guido is.

19:40 I'm very much, I love Python 2.7.

19:42 And I feel like I could use it for the rest of my life.

19:45 I'm planning on like, you know, fully embracing 3 when everyone else does.

19:51 And so, there's a lot of, you know, like all this new development on these type hints and

19:55 stuff doesn't sound as exciting to me as like, making sure that we have a way to like ship

19:59 like standalone binaries.

20:01 I feel like that would be a really great use of our time.

20:03 I'm not saying that Guido shouldn't be working on this.

20:06 Like, he should work on whatever he wants to work on.

20:08 And I'm really excited that he's putting the work into this.

20:10 But I hope other people, you know, don't stop working on other things because of it.

20:15 And I don't think they will.

20:16 That's not happening.

20:16 I was at the Language Summit this year.

20:18 And we had a really great conversation about the next year of Python.

20:21 And there was a lot of talk.

20:23 Guido did one on type hints.

20:25 There's a lot of really great stuff that kind of came up.

20:28 Okay, that's cool.

20:29 Can you talk about what else was at the Language Summit?

20:31 Let me think.

20:33 I was only there for the second half.

20:34 I did a talk on requests.

20:35 And because everyone is really interested in adding it to the standard library.

20:40 So I did a talk about the pros and cons of that.

20:43 And basically stated that requests is a critical piece of infrastructure for the Python community.

20:49 Because pip relies on it.

20:51 And like pip ships with Python.

20:52 And, you know, it's like the most popular Python package there is.

20:57 So, and I have security updates.

21:00 You know, when there's like a CVE that's released for HTTP and OpenSSL and stuff, we have to make changes.

21:06 And, you know, Python has a 16-month release cycle.

21:08 And we've released a version by maintainers in like 12 hours.

21:14 So there's a big difference there.

21:16 So if requests was to go into the standard library, it would be, I think, irresponsible.

21:20 So we decided together as a group that requests is going to be officially recommended in the Python documentation.

21:27 Instead of going into the standard library.

21:31 Which seems like a good compromise.

21:33 Right.

21:34 That makes a lot of sense.

21:35 And, you know, HTTP being such a critical but vulnerable layer as well.

21:41 And morphic layer as well.

21:42 What, sorry?

21:44 Amorphic layer as well.

21:45 It changes over time.

21:46 Yeah, absolutely.

21:47 It's definitely changing.

21:48 And if that needs to be fixed, it needs to be fixed now, not in a year.

21:54 So that's really interesting.

21:55 Yeah, I think that was a good takeaway.

21:57 It seems like a good takeaway.

21:58 Cool.

21:59 Well, I guess one more that I thought was worth calling out.

22:02 And maybe we'll do a show on this one.

22:03 It's a guy named David Beasley.

22:05 Yes.

22:06 Did one on modules and packages.

22:09 Oh, very nice.

22:10 Did a three-hour workshop on it.

22:13 And you think, what is there to know about modules and packages for three hours?

22:16 But actually, there's a lot of really interesting stuff about, like, maintainable sub-imports when you're building large packages.

22:24 And all sorts of cool stuff that I think is actually a lot of people would get a lot out of.

22:28 So that's also on that list of videos.

22:31 Yeah, David Beasley always does amazing work.

22:32 He's kind of crazy.

22:34 Yeah, it was really good.

22:36 So that was fantastic.

22:39 So what else would you like to get out into the community about Python and just let people know while you got a chance?

22:46 Let me think.

22:47 I've been very concerned for a long time about the adoption of Python 3.

22:51 And I still am.

22:52 And we had some discussion about that at PyCon as well.

22:55 And it's interesting.

22:56 It seems as though on certain platforms, like where I work at Heroku, we have about a 10% in the last month or so deploy rate for Python 3, which is really surprising.

23:06 And it seems to me as though this, and it's also, if you look at the cheese shop numbers, Python 3 is starting to get a little bit more usage.

23:15 You know, it's still extremely minimal and far behind what we want.

23:17 But the needle is moving.

23:19 And it's really fascinating because the only thing that I can tell has changed is that Django, when they made their last release, they officially, they just used Python 3 by default.

23:28 And that seems to have a tremendous impact on people, on users of the language in general.

23:34 Just having literally the quick start for Django in all the documentation is just Python 3 by default.

23:39 So I think that's really interesting.

23:41 And I think that there's a lot to learn about that and just kind of think about that and just be like, you know, maybe if I share code samples with people, they should be Python 3, not Python 2.

23:51 You know, maybe that'll help make a big difference.

23:54 You know, if we just like shift our default mode, maybe it'll get us over this hump.

23:59 Because I am concerned about the future of the language.

24:01 I think that we have a big marketing problem.

24:03 I think that there's this product that's being built that no one wants.

24:06 And so no one's using it.

24:07 And I think that it's just all about getting on the same page.

24:11 Yeah, I think that's a really good idea.

24:14 I mean, even if it's just going to blog something and put up a code snippet, if you started making that Python 3 more frequently, that would probably send some kind of signal to the world, you know?

24:22 Yeah, it's just so interesting to see.

24:24 I mean, like there's a, the needle is like actually moving.

24:26 And it is because Django changed their docs to use Python 3.

24:30 And that's just, that blows my mind.

24:32 That's not, it can't be that easy.

24:36 Well, nobody would have, you know, thought that would make the difference.

24:40 But, you know, sometimes things are unpredictable, right?

24:43 Exactly.

24:43 Yeah.

24:44 So I'm all in favor of us.

24:45 You know, I've been for a long time.

24:47 If you talk to me in private, I'd always say that I'm a Python 2 life, Python 2 user for life.

24:53 And I do kind of feel that way.

24:54 But the reality is that the Python community is valuable enough to me that I don't care what I'm using as long as we're all using the same thing.

25:01 So I think that we should all just kind of embrace the change and just see what happens.

25:07 You know, I think when you look at the actual code, sort of like modern Python 2 versus Python 3, there's not that much of a difference.

25:14 It's kind of the libraries or the packages that you want to take along.

25:18 Like, oh, this doesn't support Python 3.

25:20 Well, this project isn't using Python 3.

25:22 Yeah, it definitely depends on the style of project you're doing, too.

25:25 So if you're doing something that deals with bytes as text, which I do a lot because I work with HTTP, then Python 3 is the worst language in human history, basically.

25:36 Yeah, it's true.

25:38 Because the string stuff is really pretty different, right?

25:41 Yeah.

25:41 So if you have a byte string and you can't do anything with it anymore, it's just like a byte array of bytes, you know, of integers.

25:51 And so if you can't do, like, formatting and stuff, I think they added formatting back, actually, I think in 3.4.

25:57 But they didn't add the .format method.

25:59 They did the percentage, the modulus override, which is ridiculous.

26:04 But so you can format strings again.

26:06 But still, you can't parse them very easily.

26:08 You can't work with them very easily as text.

26:10 And to me, Python 3 makes a lot of sense in a world where everything is Unicode, but we don't live in that world.

26:16 You know, because, like, when you're sending stuff over the internet, when you're writing it to disk, when you're sending an email, it's all actually bytes.

26:24 And so I really enjoyed the flexibility that Python 2 had in that.

26:29 And that was, like, my favorite part of the language.

26:31 And they kind of took that away from me.

26:32 So I am de-incentivized to use 3 personally.

26:37 And if you look at, like, Armin Ronecker, the guy who wrote Flask, he writes really detailed blog posts about the same problem.

26:43 And it sucks.

26:45 But at the same time, like, if you – usually when you're interacting with that stuff, you're dealing with libraries.

26:50 You know, we deal with it because we're library authors.

26:52 So if we solve the problem, then it's okay for everybody else.

26:55 Right.

26:56 You guys are basically the firewall to that problem, right?

26:59 You live literally at the network layer, both of you.

27:02 And by the way, we're going to have him on and do a show on Flask.

27:04 Oh, fantastic.

27:05 And other topics as well pretty soon.

27:07 I'm looking forward to listening to that.

27:08 Yeah.

27:09 But I think, you know, you probably feel it worse than a lot of people because you live at the network boundary.

27:14 Yeah.

27:14 To me, adding Python 3 to requests was, like, one of the most painful things I've ever done as a programmer.

27:20 Definitely one of the most frustrating, if not the most frustrating.

27:23 But I learned a lot from it, too.

27:25 It got better in some ways.

27:27 Like, in Python 2, it was admittedly unclear if you were dealing with bytes or Unicode.

27:34 So before in requests, if you got back an HTTP response, and if I could decode it to Unicode, it would just be Unicode.

27:43 The content of it would be, like, response.content.

27:48 And if it was Unicode, it would be Unicode.

27:50 If it was bytes, it would be bytes.

27:51 And it would, like, change.

27:52 And then when I went to 3, that, like, that API doesn't work anymore.

27:56 It's just not possible.

27:57 So you had to split them up.

27:58 So now there's .content, which is always bytes, and there's .text, which is always Unicode.

28:02 And that is a superior API design and something I should have done before.

28:06 And using Python 3 helped me design it better.

28:09 So I do think it's a good, you know, I think that there's a lot of benefits to using 3.

28:14 I think that, you know, there'll be a lot less data loss errors that people have.

28:18 But at the same time, it's, like, really fucking frustrating.

28:23 Yeah, it makes a lot of sense.

28:24 So how do you think Python 3 would look different if it was 90% use case?

28:30 Like, do you think there would be different pressure applied to it?

28:35 Or would it just be slightly more polished?

28:37 Do you mean if I...

28:40 So, like, with requests, I say that...

28:42 Well, you'll imagine, like, we could snap our fingers and tomorrow, like, 90% of Python projects that are active today are using Python 3.

28:49 Gotcha.

28:50 What would that change?

28:51 What would the world look like?

28:52 Do you think Python 3 would be changed by that?

28:56 Or do you think people would just adapt?

28:57 Like, are there things in Python 3 that just haven't had enough, you know, we need it this way pressure put upon it?

29:03 Oh, I see what you're saying.

29:05 I feel like it is possible for us to have a 90% adoption rate now.

29:11 It just takes a lot of time.

29:13 And really the problem is that Python 3 right now isn't attractive enough to incentivize people to rewrite all their code to work with 3.

29:22 If you have existing code, you know, you're not going to...

29:24 You have to change it in order to get it to work.

29:26 And so Python 3, I do think to get to that point, we either need a lot of time or we need to incentivize users by making something really attractive that looks really nice.

29:38 You know, there's been some talks of, like, what if we remove the gill?

29:40 Would that be attractive enough?

29:42 You know, and stuff like that.

29:44 There's a lot of different options.

29:45 And none of them are very clear at this point.

29:48 Right.

29:48 That's a really interesting idea.

29:50 And I do think you need, like, the killer reason.

29:52 Yeah.

29:52 It forced a lot of people to go to the trouble of actually changing code.

29:56 Like, it's one thing to say, we'll do new projects at Python 3.

29:59 But, you know, how many new projects do people do versus, like, we've had this whole infrastructure for 10 years and now.

30:05 It's so funny.

30:06 I went to go do that recently and it was just so frustrating.

30:09 I think I was using Flask.

30:12 I don't think it has.

30:12 They support 3, but I don't know if it's, like, proper 3 support.

30:16 There was something I was running into.

30:17 And it was just, like, I wanted to throw my computer out the window.

30:20 You know, and it's, like, everything works great with 2.

30:23 Why should I have to deal with this?

30:24 So I don't.

30:24 And that's what everybody does because they don't.

30:26 So it'll be interesting to see what happens.

30:29 I have faith that the community will, you know, whatever happens is what is intended to happen.

30:34 You know, it'll be for the best.

30:35 Yeah.

30:37 I agree.

30:38 I think it's just a little unhealthy that there's such, like, the forward progress is kind of being put on the language on 3.

30:45 But a lot of the real work is still being done in 2.

30:48 And it just seems like that can't be a real positive thing.

30:51 Yeah.

30:52 And to me, the biggest problem is that we stand up and preach, you know, PEP20, the Xenopython, all the time.

30:57 And one of the best ones in there is there should be one and only one obvious way to do it.

31:01 And you go to python.org and you want to go download Python.

31:05 And you have to pick Python 2 or Python 3.

31:08 And it's like, what?

31:09 No.

31:11 You know, like, it's confusing for new users.

31:15 And so you have a lot of new users that are using 3.

31:18 And they're actually really adamant about 3.

31:20 So if you go onto, like, Reddit or Hacker News and you see anything about Python 2, you'll have all these people that are, like, bashing them and being like, why aren't you using 3, man?

31:29 That's so old.

31:31 And it's like, who are these people?

31:32 Because I go to conferences and I meet all the Python developers that go to conferences.

31:35 And those people aren't at the conferences.

31:37 And it took me a while to figure it out.

31:39 It's the new people.

31:40 Because when you're new and you're learning to code and it's like your hobby, you're, like, online all day, like, typing on Hacker News and on Reddit, you know?

31:47 So there's, like...

31:49 If you're not busy, you can go complain.

31:50 Yeah, exactly.

31:51 So, you know, the people who work for companies, they're using 2, you know?

31:54 But, you know, the new kids are starting to use 3.

31:57 And I would like for that to be the same community and not two different communities.

32:02 And it appears as though at this time there are two different communities.

32:04 So I'm just...

32:05 I just want us to be unified instead of separated.

32:08 Yeah.

32:09 Well, that makes sense.

32:09 I mean, if you're starting a new project, you have the flexibility and whatever.

32:13 But if you're working on older technology that, you know, was written before Python 3 even came out, there's maybe no choice at all, right?

32:20 It's...

32:21 That's what you're going to be using.

32:22 So a lot of the companies I visit, they're still Python 2 as well, of course.

32:27 Yeah.

32:27 I don't know any major companies that are using 3 at all, but I'm sure there are some.

32:31 Yeah.

32:33 There's a few I've talked to, but I can't...

32:36 I don't want to say it because I might get it wrong and conflate them with other companies I spoke to.

32:40 But yeah, there's a few, but you're right.

32:42 But it's not a large portion.

32:44 Yeah.

32:44 But I'm hopeful that everything will unfold as it's supposed to.

32:49 Did you guys talk about that at the Language Summit?

32:51 It was not brought up as much as you'd think.

32:54 It was more...

32:54 So there was one great talk.

32:56 It was by Jacob, again.

32:59 And he went up on stage and he was kind of just showing some numbers that he has access to on different adoption rates.

33:08 And it was interesting.

33:10 So basically what he said was that, you know, if we want to treat this like a...

33:15 Oh no, it was Glyph.

33:16 Glyph did a talk.

33:17 Sorry.

33:18 Jacob did a great talk on this same subject, but Glyph did a fantastic talk.

33:21 The author of Twisted.

33:23 And he got up there and he was saying, you know, look at this problem space that we have.

33:26 We all agree that it's a problem, right?

33:29 And everyone agrees.

33:30 And he moves forward.

33:31 And he's like, okay.

33:32 So the problem is that we don't know what...

33:35 We don't have anything to measure to see if we're improving on this problem.

33:38 So the first thing we need to do is decide what does...

33:42 You know, when this problem is solved, what does that look like, right?

33:44 So to do that, you have to decide to measure something.

33:47 So, you know, ideally it'd be like 20%.

33:50 You know, so an example metric would be like 50% of Python downloads, of package downloads from the GShop or Python 3.

33:57 Or like a majority of the downloads from Python.org for Python versions would be Python 3.

34:02 Like, you know, let's pick something and try to optimize it, basically.

34:06 And that was very much like what he was going for.

34:09 He was like, let's not leave this room until we decide on a metric together.

34:13 Or at least decide that we need a metric, right?

34:15 And so it seemed as though everyone agreed that we needed a metric.

34:19 We didn't even come close to deciding what it was.

34:21 But it was a good discussion.

34:24 And I feel like that there's a lot of merit to that type of thinking.

34:27 You know, treating us like a software project instead of like a hobby project, basically.

34:32 Right.

34:32 Well, yeah, let's make this a data-driven problem that we can apply graphs and science and statistics to or whatever, right?

34:37 Exactly.

34:38 Awesome.

34:38 Exactly.

34:39 So I think that's going to happen this year.

34:41 It got everyone thinking, so.

34:42 Yeah, that's cool.

34:43 Like, one thought comes to mind is like track the number of requests from PIP2 versus PIP3.

34:48 Precisely.

34:49 Just, you know, just not necessarily even the downloads.

34:51 Just the number of like how much are people using this version versus that version, right?

34:56 Those kinds of things.

34:56 You should totally be able to track that.

34:58 Yeah, and so there's some really great blog posts that have come out about this.

35:00 Donald Stuff just did one today.

35:03 His domain is caremad.io, I believe.

35:05 And he just did his big blog post breaking down all the PIPI numbers.

35:11 So I highly recommend checking that out because it shows you it's weird.

35:13 Web people are adopting Python 3 faster than the rest of the Python community by about 4%, which is pretty interesting.

35:20 That is really interesting.

35:22 I think that is something I would have not predicted but would say, yeah, that seems like you're telling me something true.

35:28 You know, just hearing out of the blue is because you have so much more control over the server, right?

35:32 You manage a thing.

35:33 On the other hand, if you're shipping it out to the world, you don't know where it's going to land, right?

35:38 Exactly.

35:39 Yeah, there's a scientific Python community.

35:41 And I think that they're having smaller amounts of 3 adoption, but I have to learn more.

35:46 Yeah, okay, interesting.

35:47 Like the IPython guys and so on.

35:48 Yeah, and Continuum.

35:49 And like they support 3 with like Anaconda.

35:52 I think that is like a new thing that people are still adjusting to because these scientists, you know, if you're a data scientist, you're probably not a programmer.

35:59 And, you know, you're someone who can use these tools.

36:03 And so you don't care if it's 2 or 3.

36:05 You just want to get your job done, basically.

36:06 So I think 3 has a lot of political buy-in, basically.

36:10 Cool.

36:10 So you said you work at Heroku, right?

36:12 I do.

36:12 I'm the Python overlord at Heroku.

36:14 It's good to be an overlord.

36:16 I enjoy it.

36:18 So what do you do in there?

36:19 What do you guys do with Python at Heroku?

36:21 So I am basically, you know, Heroku supports a number of languages.

36:24 We have like Ruby, Java, Node, PHP support, Python.

36:28 And I am just in charge of everything related to Python.

36:31 So I write a lot of documentation.

36:34 I help customers and I just improve the product.

36:37 And if you do like get push Heroku and like you have a Python app, then all the code that runs that is me.

36:43 And then I go to speak to conferences and, you know, I'm on a podcast.

36:49 This is technically work, I guess.

36:50 It's a good gig.

36:53 Yeah, this is work.

36:54 Yeah.

36:54 That's awesome.

36:55 Yeah, it's really cool to be kind of out there as the community front for a cool company like Heroku.

37:02 Yeah, I enjoy it.

37:03 Do you guys use it a lot internally?

37:05 Like what can you talk about what technology you have for yourselves?

37:08 Yeah.

37:08 So there's a couple different teams internally that use a lot of Python.

37:11 There's specific spots in Heroku that are just kind of infrastructure that are written in Python.

37:17 So if all of our Postgres servers get continuously backed up, the wall logs get continuously backed up to S3.

37:26 And it's written with a tool called Wally, which we have open sourced.

37:29 And if you go onto our GitHub, you'll see it's written in Python.

37:32 And a couple different things.

37:35 If you do a git push to us, I believe over SSH, I believe all of those are received by a twisted conch shell, which is pretty cool in Python.

37:44 But there's a couple different teams internally that also use a lot of Python.

37:48 There's the Heroku Connect team, which lets you interact with your Salesforce's data over Postgres.

37:53 It gives you a Postgres database to read and write all your Salesforce data to.

37:57 And that's like a Django app.

37:59 And there's a couple other teams that, you know, a lot of tools that are built with Python.

38:04 Cool.

38:04 That sounds really interesting.

38:06 Yeah.

38:06 It's pretty great.

38:07 So before we go, let me ask you a question that asks, you know, all my guests.

38:12 In addition to requests, what is your favorite package on PyPy?

38:17 I have to pick one?

38:18 Well, give me a couple that you really like.

38:21 Just kind of mostly when I just draw people's attention to stuff that's cool.

38:24 Okay.

38:25 So one of my favorites is called DocOpt.

38:27 D-O-C-O-P-T.

38:28 And what that is.

38:30 So if you want to do a command line tool, you have to parse command line arguments.

38:35 And to do that, it's not very fun.

38:37 There's arg parse and opt parse that are built into the standard library.

38:41 And they're both terrible.

38:43 I wrote something that makes it a little easier.

38:45 But this guy killed it.

38:46 So what he did was, it turns out that the help string that's printed out when you do like command dash H, right, on the command line.

38:55 You know how you see those like little brackets and stuff that like tells you all the different options?

39:00 That is an ANSI standard, it turns out.

39:03 And so what he does is you just you write the help script.

39:07 Like you write the help page.

39:09 And you know, and you like say all the different options.

39:11 And you write, you know, the little arguments and which ones are optional and which ones are positional and all this stuff.

39:15 And from that, he completely writes all the parsing code for you.

39:20 So you just give it a string and you just say, give me my arguments.

39:23 And it gives you a dictionary with all the stuff and the booleans and the true falses.

39:27 And it's incredible.

39:28 I love it.

39:28 And that is really fantastic.

39:30 Yeah.

39:31 And it's been it's been ported to like every language.

39:34 So but I think Python is the default one.

39:36 So it's I'm a big fan of that project.

39:38 Right.

39:39 That's cool.

39:40 I always feel like I'm reinventing the wheel when I go over there.

39:42 It's like, OK, start from scratch.

39:43 So that's great to bring that in.

39:45 Yeah.

39:45 Let's see.

39:47 Another one that I am excited about but haven't used yet.

39:50 It's called Click by Armin Roeneker.

39:51 And I'm sure you'll talk to him about that when you do your podcast with him.

39:54 But a couple of years ago, I wrote a library for myself called Clint.

39:59 And Clint was a command line application tools.

40:03 It's a set of utilities, basically.

40:05 I had a bunch of cool stuff in there like, you know, prompting for yes and no, parsing

40:10 arguments, writing files to the home directory on every type of computer, you know.

40:16 And I had my favorite module that I had in there was called eng.join.

40:21 So you would give it, you know, a list of strings.

40:24 And then it would it would join them properly with like an Oxford comma and conjunctions, you

40:30 know.

40:30 Oh, nice.

40:32 Yeah.

40:32 It's like really cool code.

40:33 But it kind of I need to separate all that stuff out because it's kind of a dead project.

40:37 I haven't used it in years.

40:38 And click is Armin's approach to the same problem.

40:43 It doesn't do all the same stuff.

40:45 But it's like, you know, Armin does amazing code.

40:47 And so this is a project that just is like, if you're going to write command line applications,

40:52 you need to use click.

40:53 So I'm really excited about that.

40:54 Cool.

40:55 Oh, excellent.

40:56 Those are really nice.

40:58 If I wanted to get started with Heroku as a Python, let's say I'm writing a Django app or Flask or something.

41:04 What do I do?

41:05 Well, you go to Heroku.com and you sign up for an account and it should tell you what to do.

41:09 Awesome.

41:12 Okay.

41:12 So everyone check that out.

41:14 Yeah.

41:15 And if you have any questions about Heroku and Python, feel free to reach out to me or tweet at me or whatever.

41:19 Yeah.

41:20 Very cool.

41:20 Any final shout outs you want to give?

41:22 Off the top of my head?

41:24 What do you think?

41:25 I'm going to shout out to you for making this awesome podcast.

41:28 Oh, thank you very much.

41:29 I appreciate it.

41:30 You're welcome.

41:31 The world needs more Python podcasts.

41:33 I agree.

41:33 You're doing good work.

41:34 Yeah.

41:35 My original goal was not to start a Python podcast, but just to talk to some of them.

41:39 And I found that they all had their last show like a year ago.

41:43 I'm like, wait a minute.

41:44 That can't be right.

41:45 Nice.

41:45 Looked and looked and looked and looked.

41:46 Yeah.

41:47 It's sadly right.

41:48 Mine included.

41:49 I have a one episode podcast, but it was great.

41:53 It had 6,000 listens.

41:54 That's awesome.

41:55 Where do people find it?

41:57 It's called Import This.

41:59 Import This.

42:00 It's called Skainer and I.

42:01 And we have two other episodes recorded.

42:04 It's just a matter of editing time.

42:05 I would like to get a sponsor.

42:06 So if we got a sponsor, it would be easier to edit.

42:09 It would boost you back into it.

42:12 Yeah.

42:12 It's just like it took me eight hours to edit the first episode because I was doing like

42:16 Radiolab quality.

42:17 So I either need to just settle for lower quality or ideally I'd be able to pay a company to edit

42:25 it for me.

42:26 But I don't know.

42:27 We'll see.

42:27 Exactly.

42:29 That would be nice.

42:29 I think editing is actually the most work.

42:31 It is.

42:32 Yeah.

42:33 It's ridiculous.

42:34 So we'll get back to it eventually.

42:36 Definitely.

42:38 Well, you should.

42:39 All right.

42:39 Kenneth, thank you so much for being on the show.

42:41 It's been really interesting.

42:42 And I especially appreciate like the insight into the language summit and stuff like that.

42:46 Absolutely.

42:47 Thanks for having me.

42:48 Yeah.

42:48 You bet.

42:49 And I will talk to you later.

42:50 Have a good one.

42:52 This has been another episode of talk Python to me.

42:55 Today's guest was Kenneth Wright.

42:57 Thanks, Kenneth.

42:58 Remember, you can find the links from the show at talkpythontimmy.com slash episodes

43:04 slash show slash six.

43:06 And if you're feeling generous, please check out our Patreon campaign at patreon.com slash

43:11 mkennedy or work with your company to encourage them to sponsor an episode.

43:15 Be sure to subscribe to the show.

43:17 Visit the website and choose subscribe in iTunes or grab the episode RSS feed and drop it into

43:23 your favorite podcatcher.

43:24 You can find both these in the footer of every page.

43:26 This is your host, Michael Kennedy.

43:28 Thanks for listening.

43:29 Stay tuned.

43:50 .