#185: Creating a Python 3 Culture at Facebook Transcript

00:00 Michael Kennedy: Do you or your team maintain a large Python 2 codebase? Would you like to move to Python 3, but there's just too much in place keeping you on Legacy Python? Then you will definitely enjoy this story from Jason Fried. He created a grassroots campaign to move Facebook's massive Python 2 codebase over to Python 3, and he made Python 3 part of Facebook's culture today. There are lessons here for every listener. This is Talk Python To Me, Episode 185, recorded November 10th, 2018. Welcome to Talk Python To Me, a weekly podcast on Python, the language, the libraries, the ecosystem, and the personalities. This is your host, Michael Kennedy. Follow me on Twitter where I'm @mkennedy. Keep up with the show and listen to past episodes at talkpython.fm, and follow the show on Twitter via @talkpython. This episode is sponsored by Lenode and Rollbar. Please check out what they're offering during their segments, it really helps support the show. Jason, welcome to Talk Python.

01:06 Jason Fried: Oh, thank you for having me.

01:09 Michael Kennedy: It's great to have you here. We've been working on this for a while, haven't we?

01:11 Jason Fried: It's been a long time.

01:13 Michael Kennedy: It's one of the more drawn-out sort of, getting a guest on the show, but I think people are going to be really happy about it. I think it's a great story, and the timing is perfect around the move to Python 3. Yeah, it's going to be super fun, so we're going to talk about how you've done this at Facebook and you didn't just start, it's a long, long journey, but before we get into all of that, let's start with a little bit of your background. How did you get into programming and Python?

01:36 Jason Fried: I didn't have anything to do with Python until I moved to Facebook, so before that, I was a Perl programmer, kind of ashamed now, but I used to love Perl back in the day, and then that was not socially acceptable when I moved to Facebook in 2011, so I had to learn Python, and surprisingly enough, I picked it up very quickly, and because I'm a little obsessive, so I dug into it enough that I started helping people, and that kind of like started, I guess I thought of myself as an expert, but, like, even today, you always get better and better as you get and you look back at some of your old stuff and you're like, yeah, I used to write code like that.

02:15 Michael Kennedy: It doesn't matter how much of an expert you are, there's still plenty you don't know, and there's still stuff that you look back and you're like, whoa, I did that, that was bad.

02:23 Jason Fried: Yeah, you're like, don't look at that old code please. It's like my skeletons in my closet.

02:28 Michael Kennedy: Exactly. Well, it shows you're making progress, right? I do think it's highly interesting that you were one of the driving forces for this entire sort of transformation at Facebook and yet, you walked in the door going, Python, is that the one with the space, or is that a different language?

02:48 Jason Fried: Yeah, I'd played with it before, but I just didn't really know how to write any code.

02:52 Michael Kennedy: You didn't come in as like, it's not like, Guido Von Rossum came in and said, all right, we're going to change things, we're going to fix it, like, basically more or less a beginner, a newcomer, was the driving force, and I think that's an interesting part of the story.

03:04 Jason Fried: Yeah, I don't think it's like, so I guess I came in like the end, it was October of 2011, on Halloween, so I really didn't get involved in the whole community of Python at Facebook until 2012, and we had like, we have internal Facebook, and there's like groups and stuff, so, there was a Python at Facebook group, and there was a lot of people learning Python because I picked it up, I guess quicker than they did, I was able to help them out, and because I was helping them out, people just assumed I knew what I was talking about.

03:36 Michael Kennedy: Interesting how that happens, right? They don't know you that well, right, they're just like, oh, this guy's telling me stuff, so he must know it.

03:43 Jason Fried: And because I'm like, you know, I think I know what I'm talking about, I can go and fix things, and I start fixing enough stuff that I start touching stuff all over the codebase and different projects that I'm not involved in cause I have to touch, like, to do the one thing and I don't like the way it looks, so I'm going to go fix it, and you somehow turn into, before you know it, people think of you as the subject matter expert.

04:07 Michael Kennedy: Yeah, that's really interesting, and I guess in some sense, you sort of fulfilled that prophecy by doing it enough until you know, but there is that weird middle ground where people see you that way and you don't really see yourself that way, but you kind of are. Yeah, interesting, so, what do you do day-to-day these days at Facebook?

04:25 Jason Fried: So I'm on a team we lovingly call the Ministry of the Silly Walks. Officially, it's called the PE Python Foundation, but it's an internal team and we are taking care of Python as a runtime at Facebook, so that means ensuring that we update it in a timely manner that we're going to have 3.7 something released in the next couple weeks, that we're going to look at those shared libraries that have existed for 10 years and nobody, like, to make sure that we're doing best practices instead of, oh, yeah, well, Ted wrote this in 2008 so, it's still good, right? You're like, no, maybe we shouldn't be rewriting our own thread pools everywhere. Maybe we should just use concurrent futures, or stuff like that.

05:11 Michael Kennedy: Yeah, exactly, like, that's built in, let's not reinvent the world. That's quite interesting, I'm sure there's some old stuff like that, so I guess one question to the extent that you're allowed to answer it, I don't want you to talk about anything you're not allowed to, but when I think of Facebook, I think of PHP. What's the story with Python at Facebook versus, I know Instagram joined, but there was Python at Facebook before Instagram was part of Facebook.

05:37 Jason Fried: The software engineers that run the front end, it's all PHP, but in the back end, people who were like SREs and these systems administrators and ops people, well, they love Python, or they did back then, so they wrote all these scripts in Python, they wrote all these things to like push the front end in Python, so you end up having this infrastructure to keep the site running, all written in Python. We've talked about this all the time, so like, if you have, if your service throws an alarm of some kind, there's another service that reads that alarm, and starts running these remediations people have written in code. Well, that code is Python, and it's all written, the service to do it is Python, and the service that distributes that is Python. It's like basically, it's we filled in the glue was with Python in the back end.

06:23 Michael Kennedy: Where all the cracks were, it's been filled with Python.

06:25 Jason Fried: Exactly.

06:25 Michael Kennedy: Interesting. Okay, how interesting, so I guess that's probably a good seque to your main topic, and because one of the things you started creating in Python 3 there, was this infrastructure stuff, so maybe let's get started by, I just wanted to ask you, I have my own strong opinions about this, but do you believe that there's any debate about whether you should use Python 2 or Python 3 these days or is that, I mean, several years ago, that was a solid debate that was raging. Do you think it still is?

06:57 Jason Fried: I don't think it's a debate anymore. Like, there's a site, I don't have a note, but you could probably find it, that it's the scientific Python packages, and it's like them signing off to drop support for Python 2.

07:10 Michael Kennedy: Yes, that was, it's like called the Python 3 statement or something to that effect.

07:14 Jason Fried: Yeah, it's crazy!

07:16 Michael Kennedy: It's done by Matteus Ociay or something like that, yeah.

07:20 Jason Fried: And there's tons of these packages.

07:21 Michael Kennedy: I also don't know the name of the URL, sorry.

07:22 Jason Fried: Yeah, but there's tons of these projects that say oh that, people just assume they will always will be Python 2, and they are like no, we're going to get rid of our Python 2 support. I think it really changed when, I think when the Python death clock went up.

07:34 Michael Kennedy: Yes.

07:38 Jason Fried: But and Guido was like look, we'll just make it January, January first of 2020.

07:44 Michael Kennedy: Yeah, yeah, let's not, yeah, I think it was on the mailing list, he said let's don't miss words about when this might be. We're going to make it January first, 2020, not you know? Yeah, I think the only really good option. Yes, exactly. The other possible and could've been at PyCon, so it could've been like a party like right then, but I feel like the sooner the better, honestly.

08:02 Jason Fried: Yeah, it's like I know we don't have this debate at Facebook anymore, we've won that debate. I tell, I do a new hire training class, and it's like well if you're already on the fence of whether you're going to use 2 or 3 you're a little late to the debate here at Facebook, we won that debate in 2014.

08:20 Michael Kennedy: Exactly.

08:21 Jason Fried: You need a time machine to beat that one at this time.

08:24 Michael Kennedy: Yeah, and there had been a lot of conversations around well, this library doesn't quite support Python 3 so you can't quite move yet. And all I've seen in the last year and a half is here's one more amazing library that's Python 3.5 or 3.6 because it's using certain types of type hint annotations and async and await, so I, it really feels like the argument and the, the evidence has shifted that you're being left behind with all these cool libraries, not stuck because of these cool libraries. So the URL that you're looking for is python3statment.org, that's a good one about the scientific ones. Okay, I agree with you, I think that's great. I remember, I think this was you, I saw this around on Twitter to the effect of when Python 3.6 shipped you said within two days Facebook was running Python 3.6 in production. Was that you, am I remembering that right?

09:16 Jason Fried: It may have been Lukash, but...

09:19 Michael Kennedy: It may have been Lukash, yeah.

09:20 Jason Fried: I may have said that, we were, we were very aggressive on 3.6, we've slowed down a bit because it's got more complicated, but like we're going to be, we're going to be 3.7 and I'm trying to push internally that we get these things released quicker than the external release schedule so that we don't build up that layer of oh people have snuck in internal patches or something.

09:44 Michael Kennedy: One of the things you talked about that I thought was interesting is basically people get, they get stuck on the idea this is going to be an impossible problem to solve and we just can't do the switch, but once you've made the switch from 2 to 3 and then 3.4 to 3.5, like this starts to become a habit, you're like oh yeah, we just do that when a new one comes out. Is it sort of feeling that way now?

10:06 Jason Fried: Yeah, we've gotten to the point, 'cause we don't support multiple versions of 3 at Facebook and we never supported multiple version of 2, you just kind of, you got moved along at whatever the slow pace that the compilers team would push things out, but for 3 it just, if we break you go to 3.7 it means you need to fix your code.

10:27 Michael Kennedy: Like it's your problem, right?

10:29 Jason Fried: Yeah, it's your problem, we're going to let you, we're not going to do it on purpose, like on purpose we're not going to like, we'll let you know, we'll try to fix it, but if we can't we're going to involve you because we don't support multiple versions of 3, we're all going to be pushing forward, you know, we don't want to be, we don't ever want to be stuck in the situation looking at where we're at 2.

10:45 Michael Kennedy: Right, I think it's a really interesting lesson, you were like well we were badly stuck, and so now we're going to be very aggressive and never getting stuck again. So your original talk, so you've given two presentations at various PyCons about this, one at PyOhio 2016 and one at the latest PyCon US in 2018, your original one was called, entitled Rules for Radicals. Why the radicals identifier? What was radical about all this?

11:11 Jason Fried: You know, when you're trying to think of a topic, rather the name of your talk you try to want to get something that will make the conference team get it and then you have to kind of get it approved internally, and I think I had three different topics, I think I had one of them was we used to call it's like lies and this, just basically about lying to people like for their own good, but like PyCon was like you can't say that, you know? I was like okay, well how about Rules for Radicals 'cause it's like that, I've never even read the book, I've read excerpts from the book, but it's like it's all about making change. So I was like what, which is kind of perfect for my talk, my talk's kind of about like driving change from a position of no power.

11:52 Michael Kennedy: Right, I think the no power aspect is one of the super interesting aspects. So many companies and people tell me that like at our company we can't, we're still on Python 2 because no one has decided, and one of the really interesting things is the Facebook engineering culture and this aspect of sort of earned authority I guess. Like you don't really have authority, but you start answering questions and helping people and like oh that person knows so I'll listen to them. So maybe, maybe talk a little bit about the culture and how that worked there so that.

12:25 Jason Fried: Yeah, it's, it's a very, it's a great place to work, so if you're on the fence about working for Facebook just work for Facebook, it's great. But the thing is it's like it's engineer led, so a manager or someone in, an engineer comes and says oh we're going to use this technology, we're going to use Twisted or we're going to use Django or no, it's usually that's engineers say you know what, if we use this module and they convince their engineers next to them across the hall, down the street, in Dublin, Ireland, and they all start using this thing, we start using it. And it's like you, you have to build up a consensus of by, by having a good idea, 'cause nobody has like the dictatorial thing like we're going to do it this way.

13:07 Michael Kennedy: Right, and it's not on high.

13:09 Jason Fried: Yeah, it's...

13:10 Michael Kennedy: Statement that we're going to be using technology or that technology, it's just like, all right, you all are smart engineers, these are the problems, these are our goals, make it happen.

13:19 Jason Fried: Exactly, it's like this is, and then our company's engineers are like look, I can't argue with this, this is what I think we need to do. And if, and they're like we should, you know, we should go to Python 3 'cause it is, it is looking this way, it's going to be the future, and at the time it was very sketchy, like it may not have been the future, but now we know we've, we clearly made the right decision. But it's like once we like look, do we want to be stuck in the old age or do we want to be able to keep moving forward? Because it's like if we don't make a decision now we need to like prepare for the other case.

13:54 Michael Kennedy: Right, and by the time that argument gets settled it could be really painful to turn a big ship of technology like Facebook, right?

14:03 Jason Fried: Yes, so we needed to get started early.

14:07 Michael Kennedy: This portion of Talk Python To Me is brought to you by Linode. Are you looking for bulletproof hosting that's fast, simple, and incredibly affordable? Look past that book store and check out Linode at talkpython.fm/linode. That's L-I-N-O-D-E. Plans start at just five dollars a month for a dedicated server with a gig of RAM. They have 10 data centers across the globe so no matter where you are there's a data center near you. Whether you want to run your Python web app, host a private Git server or file server, you'll get native SSDs on all the machines, a newly upgraded 200 Gigabit network, 24/7 friendly support even on holidays, and a seven day money back guarantee. Do you need a little help with your infrastructure? They even offer professional services to help you get started with architecture, migrations, and more. Get a dedicated server for free for the next four months. Just visit talkpython.fm/linode. What are some of the things that you felt like came out in Python 3 that subsequently came out and you're like oh, this is one of the things that really is important to us that we didn't know, but we knew things like this would be available if we did this.

15:17 Jason Fried: Well I, you know, it's like I think early on, 'cause like when we were wanting to go to Python 3 it was still Python 3.3, it was still, most people don't consider Python 3 worth, worth anything at all until like 3.4 or 3.5 came out, and some would probably say 3.6, but I think I was even on the fence at a certain point, I was very much in the 2 only school, but I went, I was really in, 'cause you know, we all know Python, we have the GIL, we unless we're doing IO bound work we're really handicapped in the SQL thread or multi-thread for CPU bound tasks, and I was big and new person in Python, I wanted to solve this solution, so it's like async, async's the solution, and at the time I was a very big gevent person. You know, I looked a little bit about Twisted, but Twisted, I was so new to the language and it was just too confusing for me.

16:07 Michael Kennedy: Right, it is a little different, yeah.

16:09 Jason Fried: But I went to a talk by Guido when he talked about Tulip which is what we call asyncio, or what was called asyncio now, and I just like had this epiphany, like all this stuff, 'cause like I'm writing, I'm writing libraries for people to use internally, and if they have not, if they decided to use Twisted they couldn't use my library if I were to give it. Well for my library in Twisted they couldn't use it in gevent or they had to like kind of make a religious decision about which framework they were going to use 'cause they were very opinionated.

16:38 Michael Kennedy: Right, so all of Python's async capabilities was like these random grassroots projects and none of them had anything to do with each other.

16:45 Jason Fried: None of them talked to each other.

16:46 Michael Kennedy: Right, exactly.

16:48 Jason Fried: And I had this, it's like wait, if this is in the Standard Lib and people can use it and they will 'cause it's easy enough to use, that means that we could all standardize around it, and I think like the day after like Tornado which has got an async framework around it for including these web, websites, it posted abilities to tie into the event loop, and then it was like well if Twisted did it now we have Twisted, you can run your Twisted application or run asyncio on a Twisted event loop and vice versa. So you can like this, we're in this magic world where I as the as the library writer don't have to make the religious decision for the application writer, that they can decide to be like natively Twisted.

17:28 Michael Kennedy: Right, because that's the wrong direction, right?

17:30 Jason Fried: Yeah, and it's like I was like we need to go to Python 3, we have to have asyncio. You know, 'cause I knew async was the future, but there wasn't a clear path in Python 2. But in Python 3 there was a path.

17:44 Michael Kennedy: Yeah, and I feel like the, the async story in Python is getting a lot better. I mean you talked about asyncio and then we have like a cleaner API with the async and await, we have a bunch of these cool web frameworks coming along based on things like uvloop which you know, replaced the asyncio event loop and all sorts of cool stuff happening there.

18:02 Jason Fried: Yeah there's, there's like even reimagining of asyncio using async await, there's two competing frameworks for asyncio, and someone has already written compatibility drivers so you can basically pass them to each other. So if you have a library written...

18:16 Michael Kennedy: That's cool.

18:17 Jason Fried: You can use 'em. I was like that is the future that I saw back in 2013, and I think it's...

18:22 Michael Kennedy: Right, that's awesome.

18:23 Jason Fried: It's going to pop, to play. And that was what I really, I was like oh, we have to do this 'cause you know, I'm tired of making these janky gevent libraries nobody can use. I want to make something people can actually start working with.

18:37 Michael Kennedy: Yeah, and I can definitely see that at Facebook, like you're building these libraries and you want people to adopt them, well, you know, you don't want to have that be a little small group, you're having rush in. So yeah, I think some of those, those additional reimaginings are probably Trio, Curio by David Beezly.

18:51 Jason Fried: Trio and Curio. Somebody has already written compatibility to make them talk to asyncio applications and I don't know if they can talk to each other, but I've seen more that talks to either or.

19:00 Michael Kennedy: Right, you can probably chain it through like...

19:02 Jason Fried: Yeah, go and train asyncio...

19:03 Michael Kennedy: Trio to asyncio to Curio or whatever, right? All right, so I think one of the really interesting parts of this, this journey is sort of go through the years, because kind of like we started off, like the view of Python 3 now is really different than the view of it five years ago.

19:20 Jason Fried: Exactly.

19:21 Michael Kennedy: There was absolutely advocates and people pulling for it, but it, it was a different world. So want to take us back to 2013 and sort of start there?

19:29 Jason Fried: Yeah, I think like it was, like back in 2013 there was a, there was a big movement of the let's just stay in Python 2 forever. You know, like, like I guess that the, I think there was this undercurrent that the core team of Python, of CPython had lost their minds. And that Python 3 was ridiculous. And it was a very, I think I may have, I was new to the language, so I just like, I just picked what everybody else was using at Facebook which was 2, 2.6.

19:57 Michael Kennedy: Well and you look around like on Stack Overflow there were people saying never move to 3, like that's just a waste. And there was, there was a lot of that sentiment happening.

20:06 Jason Fried: People were like oh don't bother learning Python 3. And it's like, and then when I learned it it was just like I, well why didn't we, you know, it's not that bad, you know?

20:15 Michael Kennedy: Honestly I feel like the differences are so small sort of modern 2.7 and Python 3 is like well you put parentheses in the print statement, like a few other handful of things, and none of them are like massive conceptually differences, right?

20:27 Jason Fried: There was some...

20:29 Michael Kennedy: It was the libraries.

20:30 Jason Fried: Yeah, the libraries, 'cause like especially 'cause in, in 2.6 like we had to change the way we did exception catching. The change, I don't, I don't even remember what it was, but I remember we had to fix it, 'cause it was like, it got deprecated by the time 2.7 came out, and no longer worked, so we had to fix stuff that was like there was so many bold crusty things that have been running on like Python 2 for like 10 years. So people were just like no, I don't want to do it again, you know, they were just like.

21:00 Michael Kennedy: Yeah, well and maybe the person who wrote that part is not even at the company anymore and no one wants to touch it.

21:04 Jason Fried: Exactly.

21:05 Michael Kennedy: Because they don't want to, they don't want that thing to become their problem, right, they just want to work on their stuff.

21:11 Jason Fried: It's the same for a lot of third party app, third party's libraries, that the person maintaining it just disappeared, we don't know if he's dead or she's dead, we just, they don't respond to emails, you know, and their project just kind of withered away, but people want to use it.

21:27 Michael Kennedy: Retired, they, they don't care about it, maybe they've moved on to Node.js or whatever, right, but for whatever reason they don't care.

21:34 Jason Fried: And the crazy part some of these modules are super popular like the MySQL Driver. That one.

21:40 Michael Kennedy: That one became basically unattended. Wow. How interesting. I can certainly see like maybe some burnout as well, right, it's just like I'm, I just got to take a break. If no one else is going to help this is just going to sit here 'cause you know, you got to, got to live your life, right? Your goal's not to just provide software to the world, you're a human, right?

21:59 Jason Fried: Exactly.

22:00 Michael Kennedy: All right, so you said back in 2013 you were kind of like I would love for this to be a thing, to use Python 3, but I don't think it's going to happen, right? You actually had access to Python 3 theoretically as an option, but in practice it was all messed up, right?

22:15 Jason Fried: Yeah, like the binaries were there for Python 3. You could, you could run them and like get the interpreter, but we have kind of a complicated environment, we have a build system set, go and built like C++ tendencies for compiled modules and they kind of make pexes which is basically, it's like an archive for all your code that you can distribute to a machine which is kind of run like a binary, but for Python.

22:40 Michael Kennedy: Right, it's like a hacked zip file that's executable.

22:43 Jason Fried: Exactly.

22:44 Michael Kennedy: And it bundles up all of your dependencies, but it does not bundle up the Python run time, right, it's effectively like a virtual environment kind of.

22:50 Jason Fried: Yeah, it's a deployable virtual environment. And the system that we had that built that didn't work. So like it would call out to Python 2 to run, to basically import your code and do a dot compile using the compile module to like get the PYC, and then we would just pack up the PYCs to make this pex, and of course it was like you would give it Python 3 code and the Py 2 compiler would blow up on it, you're just like great. And then once you fix all that it's like I, you get this pex and you run it and it just seg falls, or not just seg falls, it, it just stack traces like in the early guts where we're setting up the environment, you know, you're like oh, great.

23:28 Michael Kennedy: It's deeply down broken, like this is not my problem, but it's not going to work, right?

23:33 Jason Fried: Yeah, it's just like deeply broken and it's like I couldn't just throw my hands up at that point, but like this is never going to work. And I may have for a week or so, but like once you figure out that nobody is going to fix it.

23:43 Michael Kennedy: You do talk about that sort of be the change you want to see in the world and eventually you're just like well no one else is going to do it...

23:49 Jason Fried: I'm going to do it.

23:49 Michael Kennedy: It's going to have to be me. Right?

23:51 Jason Fried: And it turns out yeah, if you want something to change and you wait around for someone to do it, it may never happen.

23:56 Michael Kennedy: It likely will never happen.

23:57 Jason Fried: Yeah, likely will never happen. So I started fixing it and got, like I would get it to compile, then I would, I put the diff and have to like go chase down people to like who knew kind of the system I was messing with to do the building, but like hey, couldn't you review this code? They're like, here's my unit test, you know? I built some Python 3 stuff and it ran. Like okay, let's land it, you know? And then, then the thing that would happen is like the next week somebody would have committed some other Python 2 only code. You're just like oh god, it's just never ending.

24:27 Michael Kennedy: Yeah, yeah, so you talked a little bit about one of the challenges is you've got to convert the libraries you depend upon that are internal Facebook libraries to be 2 and 3 compatible. And then you also talked about like this regression problem like constantly challenges you had, I guess that wasn't until the next year, but that was a problem, right? Like you would, you would make your stuff work and then somebody would share.

24:50 Jason Fried: Exactly, 'cause.

24:50 Michael Kennedy: Library that would, wouldn't work.

24:52 Jason Fried: So in 2014, like 13, like people didn't care about Python 3, so like they knew that we've kind of, like we kind of said that it's probably the future, we should move to it, like I think people all kind of had this sense that eventually we'll have to move. Like if you want to Python 3 support they would be like well we don't have the road map for that, like we don't have the manpower to work on that right now. So you just go and fix it.

25:13 Michael Kennedy: Right, we can't stop for six weeks and, and like rewrite, or six months and rewrite this, right?

25:18 Jason Fried: Well most libraries it wasn't, it was like you, it was a handful of things you'd fix and then just like oh and they're handling it, it's like bytes and string are wrong, it's you know, you'd fix it, and people would that know, people aren't running this Python 3 instead would just break you, so I figured out it's like well, you know, that's where I think I really got, I really accepted the, the concept of unit test driven development, was that when I went out and fixed these things I immediately inject that, immediately put a test to make sure they stayed fixed.

25:46 Michael Kennedy: Yeah, I'm not going to fix this again.

25:49 Jason Fried: Yeah, and that, that will stemmed the bleeding, 'cause when next, when the next person came to write code against this library and change something they would immediately break the unit test, and you'd just be like oh, well let me fix this.

25:59 Michael Kennedy: Right. Yeah, so that's one of the things you talked about is basically instead of you playing whack a mole and constantly chasing these problems around you, you sort of found the way to get the build to work better and somehow you managed to get the build system to verify Python 3 behavior without really the authority, just kind of like this is for code quality, we're going to do this as well. And people were like okay, great.

26:29 Jason Fried: Yeah, so we had like, well I might as well, for like Falke8 it launched your code and it, you could run, like we already had a pep8 ledger, we actually had our, you know, we were kind of organized, we actually made sure that all of our Python code kind of looked the same. We would hit pep8, people may not, would argue like how do I fix this? But to make pep8 happy, but some various themes would just run their own linters and I noticed myself I would commit code and maybe like I had to fix something during code review and I forgot to run the linter on it and it would make it into production and it would break a test immediately, because it was like a syntax error something silly. And you're like oh, it's so stupid, how did I miss this. And it's like well we're not running linters automatically, but there are linters for the language like PHP or C++, or so have these linters that were kind of, it's like well like we're using Fabricator, which is this open source code review tool and it had a thing to tie into to Flake8, so I, I went ahead and set it up and I was like, well if I could run Flake8 under Python 3 and under Python 2 I would know if people, if I made like an obvious break to 2 or 3, and I did that and it was probably, it was really hell for a while for a lot of people 'cause you were getting double the amount of lint warnings in your code review, you'd get them for both 2 and 3, and I spent a lot of time to tell people it's like, okay, explain to people what's going on and why they shouldn't go secretly disable my linter. And because it was like this will make our code better quality and people like they were already running these linters, they were like oh yeah, this is great. Can we actually make them to dedupe, it'll cost you that, and that was that didn't happen till like 2016, but.

28:19 Michael Kennedy: Right, and now you're almost to the point where you dedupe it by turning off the version 2.1.

28:23 Jason Fried: Well no, I think it will cause basically if the messages were the same he didn't print it, they're like he didn't print the first...

28:29 Michael Kennedy: Right, right.

28:30 Jason Fried: He said Py links, this is the message kind of thing.

28:32 Michael Kennedy: Yeah, so another thing that you talked about doing is using the dunder future module to at least stop writing only Python 2 code, right? Like you imported the print function and division and so on, but this was like one of the early, early things even when there was not actually a path, right?

28:49 Jason Fried: Yeah, we had a linter, someone, he wanted to prolong the use of his 2.8, he wanted to keep writing 2.8, and hopefully it lived longer than he thought like maybe we'd switch to 3 in like the next year, he didn't really know, but he wanted to, he wanted Python 2 to behave more like 3, and that was pretty wise actually, and that got us very far at Facebook. And he set the linter to force everybody doing, and it caused a lot of pain at the time, but people just fixed. Like if you make it like this, like a commit hook or something, people will just try to make it happy, and if they can't make it happy then they complain.

29:25 Michael Kennedy: That's pretty interesting. Yeah, there was no real message from on high that say we're using Python 3, but when the linter says, oh, this doesn't work right, because of Python 3, well all right, we'll just fix that.

29:35 Jason Fried: Yeah, it's like oh why, why does my import not work? Oh well, you're using, you can't use these relative imports without being explicitly relative, and yeah, it took a while to unicode literals thing, probably ate it a lot of time, but we solved that back in like Python 2, and people internalized it as like if you wrote good Python 2 you were using the future imports.

29:53 Michael Kennedy: Right, I know, that's really cool. Another thing you did back in 2012, sorry 2014, when this was kind of still just a little micro-project was in the build system or all the environments you made sure that six and 2to3 were sort of omnipresent. What was the story of that?

30:11 Jason Fried: Well we had these linters in place and people were always like well how do I fix this thing? That would spend a ton of, a lot of churn on the message board was like, okay, I understand this is the problem, but how do I fix it and it's still work in 2 and 3? And it was like well you could do this extra code or hey you could use this, this module called Six or this module called future that you can import and it does it. So and people were like well, okay, so after we will add this to my build dependencies, and you're like, and or they would get mad, it's like oh, it's not there automatically. So it's like to make it easier to like lower the barrier I just, I tricked the system to injecting it to everybody's build fix so that your pex that would get produced would have Six and the future module.

30:54 Michael Kennedy: That's really cool.

30:56 Jason Fried: So that was that they could just do this code and didn't have to change anything about their system, it was like it was just that was the way you could write it, it was like it was part of the Standard Lib.

31:04 Michael Kennedy: That's great. Maybe for folks who don't necessarily have battle scars from the Python 2 days, tell 'em what Six is.

31:11 Jason Fried: So Six is like the first lang, the first library for converting 2 to 3. It allows you to do things that are completely different from either language. Like if you're setting a meta-class on a class it's different, like the syntax is different, Python 2 and 3, so six provides you a decorator you can call, which you can wrap your class with and it'll, it'll set the meta class, and Six has a thing where it like 'cause there were modules that moved between 2 and 3, and Six has a thing where you can import from a certain move location. And there was also this thing called Python called the future module, from that you could, it's kind of like Six, but it's a little different. Sometimes it's, sometimes it does nicer code, and sometimes it doesn't. I use both so I had to force that decision on people. You know?

31:57 Michael Kennedy: Yeah. Give 'em all the tools.

31:58 Jason Fried: All the tools.

32:00 Michael Kennedy: You want to crescent wrench, you want a socket wrench, whatever, just get it to run.

32:03 Jason Fried: Before six you would have to write these if statements where you would say like if sys.version.info like 'cause it's they top it like I'll set zero = 3, you would do some code, like this is a lot of garbage. And six is like just say, from six import py3, you say if Py 3. And you're like oh that's better, you know, so.

32:26 Michael Kennedy: Yeah, that's much better. This portion of Talk Python To Me is brought to you by Rollbar. I got a question for you. Have you been outsourcing your bug discovery to your users? Have you been making them send you bug reports? You know there's two problems with that. You can't discover all the bugs this way and some users don't bother reporting bugs at all. They just leave, sometimes forever. The best software teams practice proactive error monitoring, they detect all the errors in their production apps and services in real time and debug important errors in minutes or hours, sometimes before users even notice. Teams from companies like Twilio and Instacart and CircleCI use Rollbar to do this. With Rollbar you get a real time feed of all the errors so you know exactly what's broken in production and Rollbar automatically collects all the relative data and metadata you need to debug the errors so you don't have to sift through logs. If you aren't using Rollbar yet they have a special offer for you and it's really awesome. Sign up and install Rollbar at talkpython.fm/rollbar and Rollbar will send you a $100 gift card to use at the Open Collective where you can donate to any of the 900 plus projects listed under the open source collective or to the Women Who Code organization. Get notified of errors in real time and make a difference in open source. Visit talkpython.fm/rollbar today. You had sort of summarized each year your thought about how, how likely this Python 3 world might be in 2013 you said not going to work, not going to be a thing probably. 2014 you're like highly unlikely. So, then one of the really interesting things that caught my attention about your talk was in 2015 you started getting involved with teaching Python at Facebook to new hires.

34:11 Jason Fried: Yes. So yeah, we had this class, 'cause I've talked, we've already talked about, we had this kind of complicated environment rebuild, we had the Pex system, the fake sys file with the binary on top of it, and we have all this C++ that's compiled then with you and we have a whole process for getting external libraries into instead of completely to C++. There's just, it's, there's a lot, and coming with just an open source knowledge of Python you'd be lost. So we had this class that all boot campers for at least for touch engineering can learn about Python, and learn hopefully get rid of all these roadblocks if they can hit the ground and just like produce some Python code that does some good work.

34:52 Michael Kennedy: Right, here's what Python at Facebook looks like, here's how you work with our interface.

34:55 Jason Fried: Exactly.

34:56 Michael Kennedy: Stuff like that, right, 'cause that's, where are they going to learn that, right?

35:00 Jason Fried: Yeah, we'd warn them about the linter. You're like oh yeah, the linter's going to hit you and this is how you fix it. Ah like, we'd talk about the four futures. And that was already in there when I was teaching the classes, like what if we start teaching people that we expect them to write Python 3 and secretly that was like my, that was my secret like, 'cause it wasn't really true, but the new hires didn't know that.

35:20 Michael Kennedy: Right. And it was feasible. Like they could write Python 3 at this point, you'd done enough work to get the infrastructure in place so it would work.

35:29 Jason Fried: Yeah, so like, it's like we say, oh, you're expected to write Python 3 and if you have any issues, feel free to fix them yourself if you can, otherwise reach out to us, and it was like there was like a couple of friends, friends of Python 3 that was like yeah we were a group, that were a secret group that we were trying to push it forward at this point. That you could reach out to us and we'll help you or post in the message board and we'll help you with your Py.

35:53 Michael Kennedy: That's awesome.

35:54 Jason Fried: We were gathering like a Wiki.

35:56 Michael Kennedy: Yeah, and to be clear when you say if you run into a problem, just fix it, that isn't with your code necessarily, that's like you try to use one of the libraries it doesn't work in Python 3, well you're expected to go kind of try to make that dependency that we have work in Python 3 and just get them to sort of own the foundation of all the Python, right?

36:15 Jason Fried: Yeah, at Facebook there's this, this saying that's like nothing is nobody else's problem. Or it's not, something along those lines. But it's if you, it's like there's this mentality that something's broken, you don't just immediately go and find out oh, well who owns this code? I'm going to go complain. Like no, see if you can fix it. 'Cause if you fix it within like five minutes you'll be ahead of your schedule if you would have to go and like track down a person and like complain to them or get them to release something on their schedule, you can just fix it, put up a little diff and keep working. And if we taught people to just do that we're like we're kind of were tricking them to do work for us.

36:51 Michael Kennedy: Yeah, yeah there's a lot of lessons of sort distributed responsibility, distributed work in here through the linters and through this education component. And it more or less worked, right, people starting writing Python 3 code.

37:04 Jason Fried: Yeah, they would, they would join their teams after boot camp and they would say hey, how come are we not using Python 3, and the people there they would, they didn't have a good answer. And the new people would start writing Python 3. And so but like all these teams started coming out with like we're putting out Python 3 for our new project because that's all the new people want to write.

37:25 Michael Kennedy: I think that's great. So 2015 you dubbed the year of maybe, maybe it's going to work, it started to look good. And then 2016, something pretty major happened with Instagram, Instagram is one of four, if not the biggest Django deployment, and they actually did a whole keynote about their move to Python 3, but that was a big deal, right?

37:48 Jason Fried: Yes, that was huge. The thing was like well Instagram was Python 2 at the time and it was like do we, does Instagram stay on Python 2, and we were, it was like me and Lukash Longa were like he really fought hard for us, like no you guys have to move to Python 3, and if you don't move to Python 3 you should just give up the ghost and go to PHP.

38:11 Michael Kennedy: They're like whoa, whoa, whoa.

38:12 Jason Fried: They're like...

38:13 Michael Kennedy: This is like...

38:14 Jason Fried: Actually you know, it's at Facebook, that's a good sound business argument to say it's like... You should all be using PHP, we've invested a lot of money in that and it runs like the biggest website in the world. Like I think that makes sense and they're like oh, let's, let's, oh let's just stay with Python, you know, we don't want to like lose all of our good developers.

38:33 Michael Kennedy: Yeah, that's really interesting. There was a keynote I think at the 2016 or 2017 PyCon about this and it was really, really just such a step by step practical way for a large organizations to move, because it wasn't just Instagram upgraded to Python 3 they also changed Django, major Django versions, and they did all of that without branching in production. Right, that, like that is just incredible.

39:01 Jason Fried: It goes to talks about the power of tests as well. Like this is all test, unit test driven where they would take a test and they would make it work on Python 3 or like change the code enough to where it would still pass on 2 and 3 and they would...

39:14 Michael Kennedy: Right.

39:16 Jason Fried: And you know, how about, I don't know how they did Django on that, that was like.

39:19 Michael Kennedy: That's another level.

39:21 Jason Fried: That's crazy, that's another level, but I think it also involves unit tests.

39:24 Michael Kennedy: It must have, right, but still, that's impressive. Without branching in production, that's crazy. That's really super impressive and every time I hear someone say oh we can't move to Python 3 or it's totally unpractical.

39:35 Jason Fried: Yeah, we're too big to move to Python 3.

39:37 Michael Kennedy: Yeah. It's like well, here's a roadmap and a pretty insane example. You know, the rest of this stuff is not it's visible what you're working on 'cause it's infrastructure level, but still you know also impressive. Another thing that you said happened around 2016 is it became Python 3 by default.

39:56 Jason Fried: Whenever you would, like I talked about our mule system and you would, you would have a little script that you could tell it oh these are the third party dependencies I need, these are the dependencies that someone at Facebook has written, these are the C++ dependencies I need, and it would, the build system you would say build and it would give you the little pex that you could take to any system on Facebook and it would run your Python code. That thing, if just like taught me what the Wikis has told you to do, it would give you a Python 2 binary, and we had been telling people in the class, the new hire class that oh, you should add this to this as py_version = 3, and you'll get a Python 3, and people are like oh why do I have to say that. And we just say oh 'cause 2, 2 probably would have to do the default. And it's like there were so many people every day were saying oh, well release this Python 3. That it was like the only people that were still writing Python 2 code they didn't have, they didn't really, they weren't really like staunch Python 2 supporters, they were just using the defaults. So I, I had to like what if we just switched the default, see if anybody cares. And so I sent a message out like hey, I'm going to switch the default to Python 3. If you think this is a bad idea say something. And nobody said anything and people were all like thumbs up, you know, like this, this is good. And so I did the switcharoo and I made every, everybody who's currently Python 2 explicitly say that they're Python 2 and everybody who was Python 3 get rid of their version line completely, and it was just the default was the Python 3 version. And nobody got broken, there was no like major side outages, it just, that was a very smart decision. It was like look, if we were probably at like, probably less than 20%, I don't know, I think I have a graph somewhere in my talk where it was like, oh yeah, we were way less than 20%, we were not even like 10. But it felt like we were really high 'cause all the new stuff was coming out for Python 3.

41:41 Michael Kennedy: Right, and then this just, just flipped it, right? All the people who were not saying anything explicitly, they didn't care, it just, it changed, changed the tide, right?

41:50 Jason Fried: People who didn't care were versionaries, they just wanted a version of Python they could, you know, as long as it wasn't C++ these guys were happy, they just you know, not having to, they didn't care and they weren't part of this fight. So we just whatever the default was they're going to use it.

42:04 Michael Kennedy: That's really excellent, and a lot of the future work, the import, the print function and stuff, it's kind of what made that work, right?

42:09 Jason Fried: Oh yeah, and like yeah, so like a lot of the existing modules that people would already fix all that future stuff, some of them just imported fine into Python 3.

42:17 Michael Kennedy: Right, I mean how often are you like doing crazy like meta-class stuff, right, like that's not that common.

42:23 Jason Fried: More than we should. Is the case, everybody is like I don't want to use meta-class here, like maybe you shouldn't. But yeah, it's like I think in most of the major like heavily used code paths have already been cleaned up by like me or somebody else, 'cause if it's, if it's major use dependency it's already been fixed for Python 3, 'cause by 2016 we already had a services in production running Python 3.

42:49 Michael Kennedy: Yeah, that's great. So one of the things I think is interesting, there's a quote that you talk about from one of the engineers that sort of touches on this conception that Python 3 is slower, and it says something to the effect of, we just did a very basic 2 to 3 on our service, which means they ran the 2to3 program on the server to just automatically upgrade it. And everything just worked. Now we're using half as much memory and it's 40% faster.

43:14 Jason Fried: Yeah, this is true, that's a true quote. I saw that, I think I may have stated that the diff is being approved, and it was like simple, like it was just like changing the way they catch exceptions, right, 'cause it's like I think before it involved, I didn't remember these effects, the memory just lost, left off my head space, but it was like it was just basic things that you were like oh, that's easy. Like you didn't even need the tools to change it, it was like it wasn't, it didn't have a lot of code that needed to be changed. But so it was like surprising, there was like oh, that's all you changed? And you're 40% faster, really?

43:51 Michael Kennedy: Yeah, which speaks to the different runtime, right? The different CPython.

43:55 Jason Fried: Yeah, and for a long time Python 3 was a lot slower. And I think still there might be some edge cases that are faster on Python 2.7.

44:04 Michael Kennedy: A lot of probably has to do with strings, right, like the different behaviors around strings and bytes and all that stuff.

44:10 Jason Fried: I think it's gotten faster in 3.6, like they, they really like pulled ahead to use less memory and stuff with all the different fixes, but I think that when that thing came out people were still comparing it to like 3.2 and 3.3, and they just hadn't...

44:23 Michael Kennedy: Right.

44:24 Jason Fried: It was their argument and they just hadn't actually updated their benchmarks to use new versions, 'cause they had already given up on Python 3. They were like oh Python 3 is just too slow. And then it's like by the time 3.5 came out I think that, or that might've been out 3.4, but it was like it's kind of snuck up, like actually the core devs been working a lot to make this really fast.

44:46 Michael Kennedy: And that work continues today, right?

44:48 Jason Fried: Yes, yeah.

44:49 Michael Kennedy: Python 3.7 bound function calls, so function methods basically are 20% faster.

44:53 Jason Fried: Oh, I can't wait.

44:56 Michael Kennedy: Yeah, in most cases, right, and that's just 3.6 to 3.7.

45:00 Jason Fried: This is great.

45:00 Michael Kennedy: So yeah, it's all good.

45:02 Jason Fried: I'm hoping for next week. If Lukasz shares this I am hoping for next week.

45:08 Michael Kennedy: Okay, so you had deemed 2016 Python 3 by default, that was good, and then 27 is sort of overtaking Python 2, and one of the things you highlighted then that happened was now Instagram is running on Python 3 it has access to this amazing new features that were blocked off like asyncio and the typing, you know, type hints and all that stuff, right?

45:31 Jason Fried: Yeah, it's like I keep saying, oh you can use typing in Python 2, by it's like the types in type 2 or 3 are semantically different, like things that you can do in 2, like you may have, they may have different types or they may have the same type, but they behave differently. So it's like your type formation may not be the same between versions, so we told them to hold off till they would to 3, 'cause there's, maybe we'd have to double the work.

45:55 Michael Kennedy: Right, and you don't have it like in the function, you've got it like in doc string type format, right? Which is not nearly as nice.

46:02 Jason Fried: Yeah, it's not as nice.

46:04 Michael Kennedy: No colon, enter, or whatever, right?

46:06 Jason Fried: Yeah, and people wanted to be in line, they wanted, 'cause if it's a comment it's going to like, it's not always going to be correct. So at least with the type annotations like you would know, like it would, like it's like oh I changed the name of the import, oh I got to go fix all the types. But if it was a comment you wouldn't know that till you decided to actually run the type checker. In the early days before we even had a type checker it was like we were just putting these types for humans to read.

46:28 Michael Kennedy: Yeah, exactly. So one of the things that you said around this time is that working in Python was fun again, right, people were super excited about all the new stuff and the new capabilities and it just kind of revitalized people's joy, right?

46:41 Jason Fried: Yeah, it's like we, like I guess 'cause of the whole 2, 3 problem like people who were like fans of Python were like they were really negative, they were like Python is dead. You know, I guess kind of like the Perl is dead.

46:56 Michael Kennedy: Right. It kind of in their, in your mind though, if you're in your mind you're not switching to Python 3.

47:00 Jason Fried: It is dead, yeah, for them.

47:02 Michael Kennedy: Well people have stopped working on this for since 2008, right? It's just nothing is happening anymore, but once you're over that hurdle, all of a sudden it's like oh look, this pool is warm, it's fun here.

47:15 Jason Fried: So like all this for a while it's like let's go to Go, let's rewrite everything in Go, and, I guess if we decided to do that we would still be working on it till the next four years. But that luckily it's like people were like, oh wait, this all is good stuff, all this cool stuff in Python 3, like Instagram found out with asyncio, it's like this is good stuff, and people would convert just to have that stuff.

47:39 Michael Kennedy: Yeah, and I think that's the main argument now is there is so much that is inaccessible to Python 2 that it's so exciting and interesting these days that it's just crazy to not just do the simple work to get on to that side where you have that as an option.

47:52 Jason Fried: Yeah, when I would tell people in my boot camp, it's like if you were going to write a library for Python users don't write Python 2 support. For one, it's going to waste your time 'cause it's just, it's extra load for you, you have to make sure it works in both languages, you all going to use the new stuff, and yeah, I'm not telling them this, but it means that those Python 2 users will have to move to three to use your new good stuff.

48:12 Michael Kennedy: Yeah, absolutely. And there's some realistic reasonable examples for that, like the Django team talked about how they, they were able to drop a ton of code and the Jupyter team, Jupyter I don't know, NumPy, I can't remember, but some stuff talked about on the Python statement that we talked about earlier, like there's significant amount of simpler code and not just less code, but really truly simpler, and they say that that helps bring on new people as contributors, right, they don't have to figure out how to write it in two different languages.

48:45 Jason Fried: Oh yeah.

48:46 Michael Kennedy: Sort of versions, like there's one thing to write, this is how you do it.

48:49 Jason Fried: Oh yeah, so the compatibility layers in some of these modules were horrific. You know, that's why a lot of people were angry, it's like 'cause the languages were in certain low level cases languages were different enough that you would have stuff that would be simple a couple of lines would turn into like 20 lines, you know, so it worked in both languages.

49:07 Michael Kennedy: Yeah, like why can't I just type import this thing. And it's like well you see, you got to do this line, and.

49:13 Jason Fried: We decided to rename it.

49:15 Michael Kennedy: Yeah, yeah, exactly, I understand. Okay, so that was 2017, and 2018 you have declared victory, is that right?

49:23 Jason Fried: Yes, yeah, at this point we know Python 3 has won at Facebook and I think it's won in the world like at this. I don't think it, even like with Guido's retirement from the core team I think it's, it's a done deal, you know? Nobody is, there's no, there's no strong figure for the Python 2 camp at this point.

49:43 Michael Kennedy: Yeah, I know there are plenty of people hanging on, but I think there's just so many reasons to move now it's much harder to make that as a reasonable argument. It's just like well I just haven't done it yet.

49:56 Jason Fried: It's kind of, it's a hard place to be in. It's like oh, it's like we're going to be at this dead thing, and you know, maybe someone will take care of it. That's not a good vision for your future if you're staking your time and project on somebody got to find in their heart to take care of it.

50:12 Michael Kennedy: Yeah, it's really rough. So let's talk about this Tweet that Lukasz Langa had sent out and you had mentioned in your presentation, and it's a graph, like a performance graph of an app that was written there on Python 2.7 and then when it switched to Python 3.6, can you just give us the quick summary of that? Like it's.

50:31 Jason Fried: Yeah, probably. I don't know what this is.

50:34 Michael Kennedy: Yeah, I know, it's not super important, not yeah, yeah.

50:36 Jason Fried: We kind of dealt with this now. You know, we've been for two weeks just helping them fix 'em like issues 'cause their code base was pretty old, like it was older than my tenure at Facebook, so like once at the end of the week we switch, we fixed all our little problems, they switched over, and immediately saw this drastic drop in their memory usage. There was some CPU, oh it's half the memory, oh yeah, use half as much memory.

51:02 Michael Kennedy: Yeah, it looks like four times improvement in CPU and half the memory. So 2x in memory.

51:08 Jason Fried: Yeah, they were just like, we were just, that's I think, Lukasz immediately copied the graph and Tweeted it, you know, it was just like this is insane.

51:15 Michael Kennedy: Yes, exactly.

51:16 Jason Fried: And this goes back to what we were saying before.

51:17 Michael Kennedy: Yeah, the world seemed to have liked it. There's like 1,800 likes or something, just like yep. This is the kind of stuff that people need to see because you can tell them like well, it's better, or whatever, but it's, it's a whole another thing to say well those, that, that fleet of servers can be one server that you don't have a hard time maintaining anymore, right, like that's a much stronger story, and you don't have to do anything as much...

51:37 Jason Fried: Yeah, so this means that they were able to get rid of machines, because they're not doing as much work anymore, and that makes everybody happy, that makes the bosses happy 'cause they're not spending as much money, especially if you're like at a cloud deployment and you're paying for each one of those nodes cost you money, it's, yeah it's like you were saying nobody was saying these things, it's like there was just this common understanding that oh, Python 3 is slower than Python 2, and we had to like break that lie that was still kind of floating out there. And we, Lukasz used to talk that a lot of us said that a lot of the like core people that like, not like core people, but like people that wrote like really hard problems in Python who have solved like all kinds, they were very negative, and it was 'cause they were solving the really low level problems, but it was like the average like Joe or Jane writing Python code, they're never going to see those issues so that when somebody comes out and they're like oh I wrote this, this database driver and it was a pain to do for Python 3, it's like they do a disservice to everybody who would never see those issues.

52:44 Michael Kennedy: Right, because your, almost no one is natively, deep down at the native network layer, except for people writing web frameworks or data access, you know, there's like a few folks and they...

52:57 Jason Fried: They are the most vocal.

52:59 Michael Kennedy: Take a bit of a beating, right? But they're also highly respected folks because they're the head of these major web frameworks or database projects, and they're like this was super hard, this was the hardest thing I've done. And it's like well, okay, that sucked and we're sorry, but like that is not what it's going to be like for other people, they still import your thing and call query and it's all good.

53:20 Jason Fried: Exactly.

53:22 Michael Kennedy: Yeah, and that's pretty interesting. So I think, I think these graphs and also the story around Instagram, right, like Instagram as a whole was able to turn off servers, was able to use less memory, like they went pretty deep into that in the keynote that they cover that as well. So one other thing that you talked about in the 2018 timeframe which I guess is now is this end of life of Python 2 internally. Like we know about the Python 2 death clock, at least it exists, I don't know if everybody knows about it, but we know that there's an official external deadline for Python 2 at January first 2020, but you've also internally announced a deadline for this. Right?

54:00 Jason Fried: Yeah, we want to make sure that we don't want to push past January first, so next year is going to be the year that I go around and help kind of like go through and like find out those people that are still stuck on 2 and maybe they've been in a cave all these years and they haven't known that 3's a thing, but or maybe the team just doesn't exist anymore, that the thing is just running on automation, because we're, we're.

54:24 Michael Kennedy: How is that website still up?

54:27 Jason Fried: It's, well yeah, sometimes some things are just written well enough and they have enough automation around it to keep it up, that you just you know? You figure out that oh wait, there is no team supporting this, you know, like oh wow.

54:38 Michael Kennedy: Like that's a success story, right, but yeah, it's easily challenged.

54:41 Jason Fried: It auto builds and auto deploys every month and nobody touches it. Yeah, it's like so I think we're already internally, we're well over 80% of the libraries are all Python 3 and I think we're well over 60 something for like, like actually oh there's pexes, like the amount of pexes we shipped for Python 3, so it was figured like well how long do we want to support six, 'cause we don't want to get into the situation where we're having to support Python 2 past the death clock, the deadline of the end of life.

55:10 Michael Kennedy: Right.

55:10 Jason Fried: So.

55:11 Michael Kennedy: You don't want to be in 2020 doing anything with...

55:13 Jason Fried: Yeah, so we're telling people that...

55:15 Michael Kennedy: To trying to keep it going.

55:16 Jason Fried: By the end of 2019 at Facebook we're going to want to make it impossible to ship new Python 2 code. Just so it's like, it's got, there's got to be some cut off at some point, we're still going to support things that were already written I think until like, I think we said we're giving people maybe till 2022, but it's like at that point you won't be able to run it anymore, it's like we're going to get Python 2 actual run times off the servers, we're going to get them off the, out of the compilers, we're going to make sure.

55:47 Michael Kennedy: At that point, yeah, all the pex they won't work anymore, right, they're missing their, their CPython.

55:53 Jason Fried: If it's just something that hasn't been built in like five years we will still use it and somehow miraculously works, it will stop working.

55:58 Michael Kennedy: Right, and then there probably will be a little bit of a scramble to patch that stuff up, make it work.

56:02 Jason Fried: We're hoping we'll catch all that in 2019 and I think we will.

56:06 Michael Kennedy: Yeah, yeah, yeah, you guys have a lot of automated builds and whatnot.

56:09 Jason Fried: Yeah, we have a lot of automation that can tell us immediately if I say that oh, people that said they want 2 are now getting 3, I'll immediately serving test failures if it's not working.

56:18 Michael Kennedy: Exactly, just a little reporting I'll tell you all that. Okay, so Jason, that's a super interesting story and I think it's a real success, and one of the things I don't think we necessarily covered as well as you did in your talk here is just at no point along the way did some manager or organization come along to and say to you like, okay, make this happen. It's just like you wanted to see this happen and you just started working cooperatively with these different teams until, until it became a thing, right?

56:49 Jason Fried: Yeah, at some point like, this was in 2016, when Instagram was moving it was kind of like hey maybe we need a team to help people move or help like Instagram. We've probably done, we kind of like escaped the bullet of being the team that helped to move them from 2 to 3, but 'cause we did that for Instagram, we did it for a couple of other people, but we were like no, let's be the team that stewards Python for the future, you know, it's like we also are more lofty than just being the converter, the converting janitors or something.

57:19 Michael Kennedy: Right, that's fair, very much more lofty and more long term, right?

57:23 Jason Fried: Yeah, that's, yeah.

57:23 Michael Kennedy: Yeah, yeah very nice. So I guess transitioning, like this is what you guys did, this is what you were able to sort germinate within Facebook, what would you say to other companies that are stuck on Python 2?

57:38 Jason Fried: Well I would say if they're engineer led like Facebook and there's a lot of in the we're in this internet economy and there's a lot of startups that are out there they are very much engineer led, this is the same model works, you just basically you know, you can go and attack through code quality. I always tell somebody I didn't always work in the Bay Area and the Silicon Valley and these startups, I used to work what I used to call real, real corporations and consulting, and it's like there you got managers, and instead of convincing other engineers you just convince the manager, and it was sometimes easier.

58:11 Michael Kennedy: Right, there's one person to convince and if you win that battle it's done, right, it's coming from on high and it's just said.

58:17 Jason Fried: Yeah, if you can attack it with like some of the things you talked about it being faster, maybe it's going to use less money, maybe we'll use less servers and you can give them the example of Instagram, they were able to do it without slowing down, they didn't, they didn't stop releasing features to convert to Python 3, they did it all in line.

58:33 Michael Kennedy: Yeah, it's, I really think that Instagram example, I'm so glad they did it as keynote with all the clear steps because it's not just an inspiring example, but it's, it's a step by step roadmap to how it happens.

58:45 Jason Fried: Exactly yes, it's very inspirational.

58:48 Michael Kennedy: So I guess you know, this is a five year journey more or less, what was easier than you expected and what was harder?

58:54 Jason Fried: Oh I did, there was several times where I made, I made prognostications that were completely wrong. Like I thought that we would always have Python 2, it's like the default, and we would still be a fringe thing, but it's like we quickly overtook Python 2 faster than we thought, like I remember we, we actually had a message in our internal like IRC, like hey when Python 3 gets to 29% let's have a party. And like I just notice that nobody like seen it for a while I noticed like oh wait a minute, we're at like 80, wow. So outdated message, let's have a party, we're already at 90%.

59:28 Michael Kennedy: There you go. Yeah, and that's really something that's so rare, like you're pretty much there at that point.

59:34 Jason Fried: Yeah, so I definitely think we're going to have an internal funeral for Python 2 and I already oh it's like oh okay, it's like we'll get something, we'll make it look like a casket and have people talk about the good things that Python 2 did and we'll get rid of it.

59:50 Michael Kennedy: Python 2 got us to where we are today, right, like it was a major part of that journey so.

59:55 Jason Fried: Yeah, we can acknowledge it, we just don't have to be stuck there.

59:57 Michael Kennedy: Exactly. What was harder? You didn't like say I actually expected to work that well, it sounds like in the early days you were kind of like ah, this is a great try but I'm not so sure it's going to work so maybe, maybe you just, you saw it as hard and it actually wasn't, but.

01:00:13 Jason Fried: Yeah, it wasn't as hard as I thought it was going to be. Like there's still the long tail of stuff, like you know, we're getting smaller and smaller values and I think it's 'cause we have like in every company as the company grows you have more people writing code and that code is Python 3 your percentages just get pushed down automatically. So I think that that is the hardest thing to convince teams that like give them the, like there are teams that say they want to move, like everybody has like oh we want to move, we just don't have the manpower, we don't have a normal roadmap to move, and they tend to now 'cause we said they hey it's 2019 is the year, you have to do it now or never.

01:00:53 Michael Kennedy: Got to start running.

01:00:54 Jason Fried: So that's probably that's like they are, and just there's some teams that feel hopeless.

01:01:00 Michael Kennedy: Yeah, I could imagine.

01:01:01 Jason Fried: It's like kind of like their iPod.

01:01:03 Michael Kennedy: Yeah, do you have companies that you acquire that bring like new Python 2 as another challenge every now and then or does that not happen too much?

01:01:12 Jason Fried: Sure, I actually talked about that, but then like sure, there's just got to be some things, but I think we kind of like there's enough people internally that like I guess when you're a good leader like and people don't know that you're a leader is that when they start repeating the same arguments that you were giving years ago and it's like coming out of their mouth instead of your own mouth I think you know you've won, so it's like these people show up hey I got these Python 2 app, it's not working the way I want it to, and what's the first from some random person I've never even met? I'll get some things like, why don't you switch to 3?

01:01:43 Michael Kennedy: It has a defect, let's fix it.

01:01:45 Jason Fried: Yeah it's like, switch to 3 and then we can help you. And nobody knows how to do 2 anymore.

01:01:49 Michael Kennedy: Yeah, yeah, that's great, it's great, I like it. I mean more organizations should be here. All right, so I want to get your thought on one final thing, give a quick shout out to something that could help people as well, so the same guy that put together that was involved in putting together in sun setting in Python 2 support, the Python 3 statement, started calling Python 2 Legacy Python.

01:02:10 Jason Fried: Oh I do like that.

01:02:12 Michael Kennedy: And Python 3 modern or just straight Python, there's Python and then there's Legacy Python. And so the question is no.

01:02:18 Jason Fried: We do that internally.

01:02:19 Michael Kennedy: Do you like this idea?

01:02:20 Jason Fried: Yes, we call it Legacy Python, I love it.

01:02:22 Michael Kennedy: I do too.

01:02:23 Jason Fried: I actually use that to identify other things I'm trying to sunset.

01:02:26 Michael Kennedy: That is beautiful, so and I think when you spoke about talking to managers, like if you go to a manager and they say like look, our entire infrastructure, our entire system is built on this legacy thing that's going to be unsupported in a year, like they're like whoa, whoa, whoa, that's scary, that sounds horrible, we got to fix this, right? Like if you say Python 2 versus Python 3, like ah, techno mumbo jumbo leave me alone, right? But Legacy they know that's bad, they don't want that.

01:02:49 Jason Fried: Yeah, we, we hit it's like, as soon as we we took jabs at Python 2, some supporters, we would just to like make like we would call them ludites 'cause they didn't like the feature, they didn't like new technology, so we're like hey are you like horse drawn carriage? Is that how you got to work today?

01:03:09 Michael Kennedy: Exactly, like how do you power that computer, like do you plug it in or you got a pedal the thing?

01:03:14 Jason Fried: Singer sewing machine days.

01:03:17 Michael Kennedy: Yeah, exactly, interesting. Cool, I'm glad that, that's definitely something I'm trying to push on the community as well and I think it's resonates with people honestly. All right, then a quick shout out, a friend of this show, Anthony Shaw wrote a cool course on converting from Python 2 to Python 3 over at Plural Sites people should check out his course as well if they want to go on. But certainly they should check out the keynote at PyCon 2016 or 2017 from Instagram 'cause that really is, you know, this is the road to how you do it.

01:03:43 Jason Fried: It's 2017.

01:03:45 Michael Kennedy: 2017, thanks. Yeah, so that's a great one. All right, well we could talk for a long time, but I want to be respectful of your time so we'll just leave it here I guess. So I'll ask you two quick questions before we get out of here. First of all, if you're going to write some Python code what editor do you use?

01:04:00 Jason Fried: Well I'm nuclei, so Atom. The editor. And it's split with them because I stake a lot of them, and but if I, if I, I used to be a big fan of PyCharm, it's just setting it up all the time was kind of, that takes a lot of time so I always use Atom.

01:04:15 Michael Kennedy: Atom's quicker, yep, all right cool. And then notable PyPI package, maybe not the most popular one, but something you're like oh, people should know about this.

01:04:25 Jason Fried: Wow, let's see, Click.

01:04:25 Michael Kennedy: Click. Yeah, quickly about, what's the elevator pitch on Click?

01:04:29 Jason Fried: Well like internally people write these, they want to write these nice like Git style applications that do sub commands, and the--

01:04:38 Michael Kennedy: git add .git...

01:04:38 Jason Fried: Yes, exactly.

01:04:40 Michael Kennedy: Like that kind of stuff, right?

01:04:41 Jason Fried: The argparse or parse to make that happen is horrendous and it's disgusting and people have written modules around these things to make it and all of them are awful, and then it's like, oh it's like well there's this thing, I wrote, I used Click for one of my projects. I think the project that was the first Python 3 project, used it, I was so happy with it, I just kept telling people you have to use this thing, because it means that somebody I get off the street who's been programming Python in open source they, it will feel natural for them to write an application with us without having to use some internal like CLI parser.

01:05:18 Michael Kennedy: Yeah, that's sort of like great. Yeah, it's definitely one of the most popular ways to build CLI apps, which is you know, quite a few of the Python ones, so very cool. All right, Jason, so people are excited, they want to take their organization on the same journey, what's the final call to action, like what do they need to do?

01:05:35 Jason Fried: You just got to set, set some goals. Set a plan like to, like you know, demand the future. Like we're going to be there, we're going to go to Python 3. We're not just going to stay there, where like that's, this is a stop we're going to go to, it's a stop over until Python 3.7, it's a stop over to Python 3.8, we want to be modern so that when the new things happen we can move with them.

01:05:54 Michael Kennedy: Yeah, and they're happening fast.

01:05:56 Jason Fried: Exactly.

01:05:57 Michael Kennedy: And they're happening in large numbers so you definitely want to be there. All right, thank you so much for sharing this story, I think it's going to inspire a lot of folks.

01:06:05 Jason Fried: I hope so.

01:06:06 Michael Kennedy: To take some action, yeah, I think so.

01:06:07 Jason Fried: It's been fun.

01:06:07 Michael Kennedy: Yeah, it sure has. Nice chatting with you and goodbye.

01:06:11 Jason Fried: Bye.

01:06:12 Michael Kennedy: This has been another episode of Talk Python To Me. On this episode our guest was Jason Fried and it's been brought to you by Linode and Rollbar. Linode is bulletproof hosting for whatever you're building with Python. Get four months free at talkpython.fm/linode. That's L-I-N-O-D-E. Rollbar takes the pain out of errors, they give you the context and insight you need to quickly locate and fix errors that might have gone unnoticed until your users complain of course. Track a ridiculous number of errors for free as Talk Python To Me listeners at talkpython.fm/rollbar. Want to level up your Python? If you're just getting started try my Python Jumpstart by Building 10 Apps or our brand new 100 Days of Code in Python. And if you're interested in more than one course be sure to check out the Everything Bundle, it's like a subscription that never expires. Be sure to subscribe to the show, open your favorite pod catcher and search for Python, we should be right at the top. You could also find the iTunes feed at /itunes, Google Play feed at /play, and direct RSS feed at /rss on talkpython.fm This is your host, Michael Kennedy, thanks so much for listening, I really appreciate it. Now get out there and write some Python code.