WEBVTT

00:00:00.001 --> 00:00:06.760
Since 2008, there's been this tension in Python where much of the effort to improve it has been focused on Python 3,

00:00:06.760 --> 00:00:13.300
and many of the developers were left stuck in Python 2, primarily because important packages were not yet Python 3 capable.

00:00:13.300 --> 00:00:21.980
We've moved into an era where most of the packages anyone is using are fully Python 3 enabled, and many are Python 3 only.

00:00:21.980 --> 00:00:23.920
The latest Django framework, for example.

00:00:23.920 --> 00:00:28.800
There are many carrots and a number of heavy sticks encouraging us all to move to Python 3.

00:00:29.000 --> 00:00:32.240
But what if you have a large code base that needs to be migrated?

00:00:32.240 --> 00:00:35.580
What are the concrete steps and the gotchas in this whole process?

00:00:35.580 --> 00:00:38.580
This week, we welcome back Anthony Shaw to the show.

00:00:38.580 --> 00:00:44.260
He just published a new course on migrating Python 2 code, and he's here to share his tips with us.

00:00:44.260 --> 00:00:49.260
This is Talk Python To Me, episode 155, recorded February 13, 2018.

00:00:49.260 --> 00:01:09.300
Welcome to Talk Python To Me, a weekly podcast on Python, the language, the libraries, the ecosystem, and the personalities.

00:01:09.880 --> 00:01:11.440
This is your host, Michael Kennedy.

00:01:11.440 --> 00:01:13.440
Follow me on Twitter, where I'm @mkennedy.

00:01:13.440 --> 00:01:19.900
Keep up with the show and listen to past episodes at talkpython.fm, and follow the show on Twitter via at Talk Python.

00:01:20.600 --> 00:01:23.560
This episode is sponsored by Linode and Rollbar.

00:01:23.560 --> 00:01:25.780
Please check out what they're offering during their segments.

00:01:25.780 --> 00:01:27.300
It really helps support the show.

00:01:27.300 --> 00:01:39.080
Before we get to the interview with Anthony, I just want to let you all know, I recently published a brand new course at Talk Python Training called Eve, Building RESTful APIs with MongoDB and Flask.

00:01:39.220 --> 00:01:45.480
This course is written by Nicola Hiroshi, who was actually the very first guest on Talk Python.

00:01:45.480 --> 00:01:47.260
He's episode number one.

00:01:47.260 --> 00:01:51.460
He also happens to be the creator and maintainer of the Eve framework.

00:01:51.460 --> 00:02:04.440
Eve is this framework that lets you take Flask and MongoDB and turn your collections into a RESTful API with validation, schema verification, permissions, all sorts of cool stuff.

00:02:04.440 --> 00:02:06.240
It's a powerful, powerful framework.

00:02:06.240 --> 00:02:08.680
The course is really well done by Nicola.

00:02:08.960 --> 00:02:11.360
Check it out at training.talkpython.fm.

00:02:11.360 --> 00:02:15.820
Now, let's chat with Anthony about getting that Python 2 code out of here.

00:02:15.820 --> 00:02:18.060
Anthony, welcome to Talk Python.

00:02:18.060 --> 00:02:18.980
Hi, Michael. How are you?

00:02:18.980 --> 00:02:21.360
Hey, it's great to catch up with you. It's been a while.

00:02:21.360 --> 00:02:23.220
Yeah, definitely. It's great to be on the show.

00:02:23.220 --> 00:02:26.080
Doing well and looking forward to this conversation.

00:02:26.080 --> 00:02:33.280
I think the time really has come for this whole conversation about like, why are we not on Python 3?

00:02:33.280 --> 00:02:38.500
And then, you know, maybe what are the actual concrete steps that'll help you get over the blockers?

00:02:38.700 --> 00:02:46.220
I think we keep talking about how it's been, you know, 10 years since Python 3 has been out and trying to lay guilt on people to move across the Python 3.

00:02:46.220 --> 00:02:49.080
But there's really, it's really a bit more complicated than that.

00:02:49.080 --> 00:02:51.220
So I'm looking forward to kind of diving into some detail.

00:02:51.380 --> 00:02:52.000
Yeah, absolutely.

00:02:52.000 --> 00:02:57.000
So we had you on the panel show a while ago on Contribute to Open Source.

00:02:57.000 --> 00:02:58.420
And that was really, really great.

00:02:58.420 --> 00:03:02.600
But I don't know that we got a chance to get your whole story and how you got into programming Python.

00:03:02.600 --> 00:03:04.180
So why don't you let us know?

00:03:04.180 --> 00:03:04.680
Yeah, sure.

00:03:04.780 --> 00:03:07.860
So I got into programming at a really early age.

00:03:07.860 --> 00:03:09.320
I think I was 12.

00:03:10.340 --> 00:03:13.440
And I guess that probably would have been mid-90s.

00:03:13.440 --> 00:03:19.840
And in those days, there was not really a free access to compilers and online knowledge and things like that.

00:03:19.940 --> 00:03:26.440
So my programming experience was based off of boxes of software and floppy disks that I could get my hands on.

00:03:27.280 --> 00:03:30.240
So I learned some pretty odd programming languages to start off with.

00:03:30.240 --> 00:03:34.180
One of those was Dbase 4, which is sort of an accounting programming language.

00:03:34.180 --> 00:03:39.820
And I tried to make a Robot Wars simulator in ASCII, which is pretty challenging.

00:03:39.820 --> 00:03:42.960
So yeah, that's kind of as I evolved from there.

00:03:42.960 --> 00:03:51.480
Over the last seven years, I've been working in software teams in C#.net and got into Python a few years ago.

00:03:51.480 --> 00:03:53.080
I was in Seattle.

00:03:53.080 --> 00:03:55.840
Over the weekend, I became pretty sick.

00:03:56.040 --> 00:03:59.760
I was stuck in my hotel and I decided to pick up a new programming language.

00:03:59.760 --> 00:04:07.500
So I was contributing to the Apache Lib Cloud project, which I've contributed a lot to actually over the last three years.

00:04:07.500 --> 00:04:12.220
And last year, joined the Apache Software Foundation as a member, which is great.

00:04:12.220 --> 00:04:19.720
So yeah, just kind of fell in love with the Python language, the simplicity and the speed at which you can get something working.

00:04:19.720 --> 00:04:21.480
And it still scales pretty well.

00:04:21.480 --> 00:04:22.400
So that's great.

00:04:22.400 --> 00:04:23.500
Yeah, that's really interesting.

00:04:23.500 --> 00:04:25.400
So you kind of healed yourself with Python.

00:04:25.680 --> 00:04:26.580
Yeah, definitely.

00:04:26.580 --> 00:04:30.400
You're really making me think back to when I was younger as well.

00:04:30.400 --> 00:04:33.600
When you used to get software tools literally in boxes.

00:04:33.600 --> 00:04:44.020
Like I think I remember going to the store and buying Visual C++ 1.52 and getting all the DVDs, like the six or seven DVDs and just flipping them in one after another.

00:04:44.020 --> 00:04:45.220
Those are the times, right?

00:04:45.360 --> 00:04:47.960
Yeah, but the best thing about the box was it always came with a manual.

00:04:47.960 --> 00:04:54.040
And, you know, as a kid, I used to sit there and read the manual to understand everything the language could do and how it worked.

00:04:54.280 --> 00:04:59.000
And, you know, see who didn't want a copy of this particular language or Visual Studio.

00:04:59.000 --> 00:05:01.340
Yeah, and leveraging the books, which is great.

00:05:01.340 --> 00:05:02.680
But nowadays, we have the internet.

00:05:02.680 --> 00:05:05.260
So we don't need that anymore.

00:05:05.740 --> 00:05:06.160
It's interesting.

00:05:06.160 --> 00:05:08.840
It's much more JIT style, right?

00:05:08.840 --> 00:05:10.500
Like it used to be, here's the book.

00:05:10.500 --> 00:05:11.640
This thing contains everything.

00:05:11.640 --> 00:05:13.380
So you can just go through it, right?

00:05:13.380 --> 00:05:16.060
You would never try to do that now because it's just too comprehensive.

00:05:16.060 --> 00:05:16.840
Pretty funny.

00:05:16.840 --> 00:05:17.460
Okay.

00:05:17.460 --> 00:05:18.880
Well, that's really cool.

00:05:18.880 --> 00:05:21.580
And maybe tell people really quickly about Apache Lib Cloud.

00:05:21.740 --> 00:05:23.980
Like that's a pretty big project and you've done a lot with it.

00:05:23.980 --> 00:05:24.520
Yeah, sure.

00:05:24.520 --> 00:05:28.440
So it's a cloud abstraction library in Python.

00:05:28.440 --> 00:05:34.780
So you've got some major cloud platforms like AWS, Azure, Google Cloud.

00:05:34.780 --> 00:05:49.300
And really what it does is it provides a common class and a common set of models in Python that you can connect to all these different cloud platforms and get lists of servers or your DNS records or your storage objects.

00:05:49.520 --> 00:05:53.760
And the responses you get back from the API will be consistent across all the different clouds.

00:05:53.760 --> 00:05:59.060
It's a great way of writing compatibility code essentially across different cloud platforms.

00:05:59.060 --> 00:06:00.640
We don't just support the big four.

00:06:00.640 --> 00:06:03.720
Yeah, it's about 50 different cloud platforms.

00:06:03.720 --> 00:06:11.120
There's some pretty niche regional ones, especially, you know, there's like a great cloud platform in Switzerland, for example, that's really popular over there.

00:06:11.120 --> 00:06:11.960
Yeah, that's really cool.

00:06:11.960 --> 00:06:25.420
Also, if you're going to do some sort of DevOps type thing, you could say right against Lib Cloud, like spin up the servers and you could just point it at EC2, point it at Azure or whatever, right?

00:06:25.420 --> 00:06:26.720
It just does it.

00:06:26.720 --> 00:06:27.420
It just does it.

00:06:27.420 --> 00:06:29.040
There's a lot of spoon bending under the covers.

00:06:29.760 --> 00:06:33.340
Yeah, the APIs were never really consistent.

00:06:33.340 --> 00:06:34.560
And that's been on purpose, really.

00:06:34.560 --> 00:06:45.100
You know, they're trying to offer the full functionality of what their cloud can through the API, but also portability is not really in their best interest because they want to kind of keep you locked into the platform.

00:06:45.420 --> 00:06:52.140
Yeah, do you feel like the cloud is like the next great lock-in, maybe even larger than, say, Windows and Mac?

00:06:52.140 --> 00:06:52.560
Absolutely.

00:06:52.560 --> 00:06:57.700
And that's kind of really why I've been dedicated to the Lib Cloud project for the last few years.

00:06:57.700 --> 00:07:02.200
I mean, even in my current job, I guess I'm, you know, working pretty late hours most days.

00:07:02.200 --> 00:07:10.360
And I'm just trying to find whatever time I can to keep contributing to it because I think it's really important, especially when I think back to how I learned programming.

00:07:10.460 --> 00:07:21.840
It was, you know, spending whatever pocket money I could on books and trying to find whatever software I could and assembling PCs out of old bits of hardware that nobody wanted so I could keep learning.

00:07:21.840 --> 00:07:40.240
And my concern is that if cloud becomes the way of running computing, and all of it costs money, then it's going to push out all these people who don't have access to cash, like students, young people, people in areas of geographies where they don't necessarily have access to.

00:07:40.240 --> 00:07:43.460
Typically, US dollars to pay for all these cloud resources.

00:07:43.460 --> 00:07:47.000
And then also countries where network connectivity is not that great.

00:07:47.000 --> 00:07:49.220
I do a lot of work in Africa, for example.

00:07:49.220 --> 00:07:56.100
And there's some countries in Africa where their network connectivity can be pretty sporadic, as well as the sort of power as well.

00:07:56.100 --> 00:07:58.100
So that can cause all sorts of challenges.

00:07:58.100 --> 00:08:03.340
And yeah, I really care about people having access to learn and run computing.

00:08:03.340 --> 00:08:04.620
So I don't want them to be pushed out of that.

00:08:04.620 --> 00:08:05.560
That's a really cool project.

00:08:05.560 --> 00:08:09.260
So you're like a cloud freedom fighter, keeping the cloud free for everyone.

00:08:09.260 --> 00:08:09.560
Awesome.

00:08:10.120 --> 00:08:10.440
Yeah.

00:08:10.440 --> 00:08:13.580
So maybe talk about what you do day to day.

00:08:13.580 --> 00:08:19.660
You have this kind of mission of teaching people, getting people into programming in Python on a grand scale, right?

00:08:19.660 --> 00:08:20.480
Yeah, no, definitely.

00:08:20.480 --> 00:08:25.220
So I'm basically head of talent development at a company called Dimension Data.

00:08:25.220 --> 00:08:32.000
So it's a 30,000 people organization, operates in 50 countries, so pretty much all over the world.

00:08:32.680 --> 00:08:40.460
And my role is to look after the skills of the people within the organizations that can include lots of things.

00:08:40.460 --> 00:08:47.240
On the technology side, one of the big pushes we've been doing is to try and convince as many people as possible to learn Python.

00:08:47.480 --> 00:08:50.180
So we're up to 4,000 people now.

00:08:50.180 --> 00:08:50.820
Yeah.

00:08:50.820 --> 00:08:52.640
And we partnered with Talk Python Training.

00:08:52.640 --> 00:08:55.480
You're far too modest about the quality of those courses.

00:08:55.480 --> 00:08:57.280
They've been so popular.

00:08:57.280 --> 00:09:04.920
The Python Jumpstart by Building 10 Apps is the single most popular course we have within the whole company now on our learning management system.

00:09:05.640 --> 00:09:12.020
Yeah, and it's really cool to visit offices all over the globe and see the little Python logo popping up in all these different places.

00:09:12.020 --> 00:09:13.060
So it's been great.

00:09:13.060 --> 00:09:13.540
That's awesome.

00:09:13.540 --> 00:09:18.000
Why don't you talk just really quickly about how you got people to do this?

00:09:18.000 --> 00:09:23.060
Because so many organizations, whether it's Python, my classes, some other random classes,

00:09:23.060 --> 00:09:29.000
they have problems of getting people to actually want to learn, to be excited about it, and so on.

00:09:29.000 --> 00:09:33.000
You have a lot of cool gamification and sort of social aspects to it, right?

00:09:33.000 --> 00:09:35.220
The first bit is really about confidence.

00:09:35.220 --> 00:09:36.500
And curiosity.

00:09:36.500 --> 00:09:39.480
So if you ask people, well, how did you get into programming?

00:09:39.480 --> 00:09:45.820
There's always typically a story about how they were curious and how they explored things in an environment where it's quite safe.

00:09:45.820 --> 00:09:49.060
And I think people think of programming, I blame them.

00:09:49.060 --> 00:09:49.780
I blame Hollywood.

00:09:49.780 --> 00:09:56.680
You know, you see these whizzing zeros and ones going by on a green screen, and they make it look complicated.

00:09:56.680 --> 00:10:00.900
And it kind of puts people off because they think that's maybe not something that's for me,

00:10:00.900 --> 00:10:03.580
or that could be something I'm going to struggle with.

00:10:03.660 --> 00:10:06.660
So the first thing you're going to do is build up people's confidence to just give it a try,

00:10:06.660 --> 00:10:12.240
and then put them in an environment where they can just learn the very basics and build them up from there.

00:10:12.240 --> 00:10:14.660
So that's really what we've been trying to do.

00:10:14.660 --> 00:10:16.840
First of all, the course is super important.

00:10:16.840 --> 00:10:28.180
But then also encouraging people to sit with each other in offices and different places and just help each other and collaborate with each other and work through some of the hurdles they might have.

00:10:28.180 --> 00:10:32.420
So in different countries, you've got different programs depending on what food is popular.

00:10:32.940 --> 00:10:41.180
So South Africa, for example, we've got there's a donut morning that runs every week, where people to get together and talk about their Python challenges and work through the course and things like that.

00:10:41.180 --> 00:10:47.500
Thailand, as an example, they have a challenge to do one of the apps in the jumpstart course every week.

00:10:47.940 --> 00:10:51.100
And they have a little celebration every time someone finishes one of the apps.

00:10:51.100 --> 00:10:53.800
So that's kind of why we've seen so much popularity in Thailand.

00:10:53.800 --> 00:10:57.060
It's all about people, really, and getting them together and helping them learn.

00:10:57.060 --> 00:10:58.220
Yeah, that's awesome.

00:10:58.220 --> 00:11:03.140
Hopefully a lot of organizations can adopt some of that because that's a really big difference.

00:11:03.400 --> 00:11:10.260
I mean, just think of a company of 30,000 people where they all have some capability to automate parts of their job.

00:11:10.260 --> 00:11:11.500
It could really make a big difference.

00:11:11.500 --> 00:11:12.160
Yeah, definitely.

00:11:12.160 --> 00:11:17.180
It's not that we're trying to make everyone developers because that's not what we need to do.

00:11:17.180 --> 00:11:22.520
It's that basic coding skills is going to be almost like a requirement going into the future.

00:11:22.520 --> 00:11:30.660
So just as I can use Microsoft Outlook or Excel, that's kind of assumed when you go into a professional job nowadays.

00:11:31.000 --> 00:11:38.380
And I think programming, now that it's being taught in schools to pretty much all students in some countries, it's going to be really useful.

00:11:38.380 --> 00:11:44.040
And especially in our space in technology, everything is now software-defined and automated and has APIs.

00:11:44.040 --> 00:11:47.500
So coding is now the new way of interacting with it.

00:11:47.500 --> 00:11:48.640
So people need to learn that.

00:11:48.640 --> 00:11:49.720
It's definitely a superpower.

00:11:49.720 --> 00:11:50.840
It's awesome that you're doing that.

00:11:50.840 --> 00:11:53.960
So speaking of courses, we were talking a while ago.

00:11:53.960 --> 00:11:59.320
You said, hey, I'm working on this really cool course on moving from Python 2 to Python 3 and helping people with that.

00:11:59.320 --> 00:12:01.240
So I'm really excited that you built this.

00:12:01.240 --> 00:12:03.680
This is definitely something that should exist out there.

00:12:03.680 --> 00:12:06.740
Why don't you tell people quickly about your course and how they can get to it?

00:12:06.740 --> 00:12:09.120
Then we'll get into the details of why and how.

00:12:09.120 --> 00:12:13.280
This is a course really talking about how to move from Python 2 to Python 3.

00:12:13.280 --> 00:12:17.540
And it goes through a whole range of things, some of which we'll talk about today.

00:12:17.680 --> 00:12:19.480
So that's, you know, what changed?

00:12:19.480 --> 00:12:20.620
How should you move?

00:12:20.620 --> 00:12:22.180
How should you plan for the change?

00:12:22.180 --> 00:12:23.460
How to test it?

00:12:23.460 --> 00:12:29.140
Really kind of all the things that you'd need to know to move a library or an application from 2 to 3.

00:12:29.140 --> 00:12:31.100
So it's a pretty big challenge.

00:12:31.460 --> 00:12:33.820
But I've tried to really break it down into small pieces.

00:12:33.820 --> 00:12:40.400
So I think, you know, if you can break something down to small enough pieces and just start with the first one, then it really makes any problem a lot easier.

00:12:40.400 --> 00:12:43.440
So this will be a course that will be available on Pluralsight.

00:12:43.760 --> 00:12:51.020
So if you have a Pluralsight subscription, if you're a professional developer, your company should have already provided you a Pluralsight account if you've asked.

00:12:51.020 --> 00:12:56.460
And if they haven't, you should push them a bit harder, especially access to some of the talk Python courses as well.

00:12:56.460 --> 00:13:00.180
I'd say, you know, really kind of lean on your employer to give you access to this stuff.

00:13:00.180 --> 00:13:02.060
So, yeah, that will be out on Pluralsight.

00:13:02.060 --> 00:13:03.800
It's about two hours long.

00:13:03.800 --> 00:13:05.080
So it's not too bad.

00:13:05.080 --> 00:13:09.880
You can watch it over, you know, maybe like a short afternoon or something and play with some of the examples.

00:13:09.880 --> 00:13:24.680
Yeah, maybe you could even have people do something really awesome, kind of like what you're doing with some of your teams, where you could literally get the entire team together and sit and experience the course like in a projector in a conference room in an afternoon.

00:13:24.680 --> 00:13:29.180
And then or maybe in the morning, in the afternoon, you go and actually start the conversion and plan that out.

00:13:29.180 --> 00:13:29.840
Right. That'd be awesome.

00:13:29.840 --> 00:13:30.440
Yeah, definitely.

00:13:30.440 --> 00:13:36.820
A lot of the stuff in the course is actually about having conversations with your team about how to move the application.

00:13:37.520 --> 00:13:44.000
I think I've kind of assumed that what you're trying to move to Python 3 is typically a medium to large application.

00:13:44.000 --> 00:13:47.180
If it was a simple script, it would be pretty straightforward.

00:13:47.180 --> 00:13:49.840
So, yeah, it's kind of assuming it's a large one.

00:13:49.840 --> 00:13:51.740
It's not just you working on the project.

00:13:51.740 --> 00:13:54.620
So, yeah, I mean, let it out.

00:13:54.620 --> 00:13:57.560
If it's like 500 lines, like you could just rewrite it.

00:13:57.560 --> 00:13:57.980
Right.

00:13:57.980 --> 00:13:58.940
It's not a big deal.

00:13:59.400 --> 00:14:09.860
But if it's Instagram with a million lines of code, if it's Dropbox, if it's Bank of America, like it is the essence of your product and hundreds of people work on it.

00:14:09.860 --> 00:14:11.780
That is an entirely different undertaking.

00:14:11.780 --> 00:14:12.040
Right.

00:14:12.040 --> 00:14:12.580
Yeah, definitely.

00:14:12.580 --> 00:14:13.220
For sure.

00:14:13.980 --> 00:14:27.640
So, before we get into the details of sort of how, like three years ago, it really felt to me like there was this debate of whether Python 2 or Python 3 was the right thing to do for your projects.

00:14:27.640 --> 00:14:32.540
It seems to me as if that's more or less settled in Python 3's favor.

00:14:32.540 --> 00:14:33.500
What do you think?

00:14:33.500 --> 00:14:36.420
Is that debate still worth having or is it kind of done?

00:14:36.420 --> 00:14:42.160
I'm still having the debate with people on different projects and things like that.

00:14:42.160 --> 00:14:47.560
I say, you know, like I said earlier, it's been out for a decade, but that's not really a fair statement.

00:14:47.560 --> 00:14:50.940
I mean, 3.4 has been out for four years now.

00:14:50.940 --> 00:14:55.400
And I'd say that's probably the first kind of really stable release of the 3 series.

00:14:55.400 --> 00:15:00.760
So, we've really had the last four years to kind of work with Python 3 and some of the changes.

00:15:01.200 --> 00:15:12.280
And Python 2.7 specifically is such a stable release of Python 2 that it's been a real challenge for people, I think, to have a reason to move across.

00:15:12.280 --> 00:15:16.580
So, Python 3.4 is as good as Python 2.7.

00:15:16.580 --> 00:15:18.140
There's a few more extra features.

00:15:18.140 --> 00:15:19.780
Futures came out there.

00:15:19.780 --> 00:15:22.480
I think the AsyncIO library came in 3.4.

00:15:22.480 --> 00:15:24.880
So, there's kind of like a couple of reasons to move.

00:15:24.880 --> 00:15:30.080
But definitely with 3.5, 3.6, and now 3.7, there are lots of reasons to move across.

00:15:30.080 --> 00:15:32.580
So, I think that's settled that part of the debate.

00:15:32.580 --> 00:15:41.040
And then also, we've got the upcoming, you know, end of support for Python 2.7, which people are going to say, you know, well, it works fine.

00:15:41.040 --> 00:15:45.620
Maybe if security holes and things like that come out in the future, we can resolve them.

00:15:45.620 --> 00:15:49.980
But lots of people don't actually realize there are existing holes in Python 2.7.

00:15:49.980 --> 00:15:54.320
In the XML modules, for example, there are some pretty well-documented flaws.

00:15:54.320 --> 00:15:58.320
In some of the temp file standard library modules, there are some security holes.

00:15:58.320 --> 00:16:03.860
So, this is going to happen that there are reasons to move across to Python 3 from a security point of view.

00:16:03.860 --> 00:16:12.580
But then also, I don't think we should keep trying to divide the community because we've seen time and time again in other languages like Perl 5 to 6, for example.

00:16:13.660 --> 00:16:22.320
And then, you know, .NET's going through it at the moment, moving to .NET Core, where they're kind of, there's been a huge overhaul trying to fix all the big issues that they had in the old version.

00:16:22.620 --> 00:16:28.160
And it's kind of separating the community until they can all agree to move forward and get on with the new.

00:16:28.160 --> 00:16:29.520
Well, I think that's a really good point.

00:16:29.520 --> 00:16:34.760
And one of the major frustrations and challenges of that, I think, is you mentioned the .NET Core stuff.

00:16:34.760 --> 00:16:35.780
I think you see it there.

00:16:35.840 --> 00:16:47.800
You certainly see it with the core developers in Python, is that the people who create and maintain and improve Python are putting all of their energy and effort into Python 3, right?

00:16:47.800 --> 00:16:49.240
That's where the work is being done.

00:16:50.240 --> 00:17:03.640
And you're only going to, you know, you're only going to get the benefit and realize all those things that they've added in terms of performance and memory and language features and everything if you make the jump, right?

00:17:03.640 --> 00:17:04.320
Yeah, definitely.

00:17:04.320 --> 00:17:04.680
Yeah.

00:17:04.680 --> 00:17:10.460
So the sooner that gap is sort of over with and everybody's back focused on the same thing, I think it's really interesting.

00:17:10.460 --> 00:17:19.740
I was at PyCascades recently, a conference in Vancouver, and Guido was there, and he gave the keynote, which was a retrospective on Python 3.

00:17:20.000 --> 00:17:26.900
And he talked about why he felt there were challenges moving to Python 3, why this whole thing began, right?

00:17:26.900 --> 00:17:33.060
Obviously, there's the benefits, but like why, what could they have done to avoid this gap, basically?

00:17:33.060 --> 00:17:34.580
It's a really interesting talk.

00:17:34.580 --> 00:17:35.980
I'll link to it in the show notes.

00:17:35.980 --> 00:17:45.180
One of them I think that was interesting was basically underestimating the importance and significance of third-party libraries, right?

00:17:45.180 --> 00:17:46.820
So how do you think we're doing there, right?

00:17:46.860 --> 00:17:52.520
There's places you can go check and say, does my library support Python 3 for the popular ones on PyPI?

00:17:52.520 --> 00:17:54.540
Like what ones are converted and so on, right?

00:17:54.540 --> 00:17:56.560
Yeah, I think we're definitely up there now.

00:17:56.560 --> 00:18:02.280
There's a few tools you can use that check for Python 3 compatibility for your modules.

00:18:02.520 --> 00:18:13.940
So if you're pulling stuff from PyPI, but from what I checked last time, of the 200 most popular modules on PyPI, I think 95% of them are now compatible with Python 3.

00:18:13.940 --> 00:18:18.120
And the ones which are left over have Python 3 equivalents.

00:18:18.540 --> 00:18:21.520
So I think, you know, for the big packages, we're pretty much done.

00:18:21.520 --> 00:18:28.780
It's kind of a lot of the really niche applications and more obscure packages that have been a bit of a challenge.

00:18:28.780 --> 00:18:36.900
And yeah, there's definitely ways to, you know, upgrade third-party packages if you can't find the original developer.

00:18:37.340 --> 00:18:43.640
Some of that has really been because people publish them and then, you know, say, oh, I'm done with this and move on to something else and it never gets updated.

00:18:43.640 --> 00:18:44.760
Yeah, for sure.

00:18:44.760 --> 00:18:49.060
And you actually wrote a nice article on like, I depend on a package that's Python 2.

00:18:49.060 --> 00:18:49.920
Now what?

00:18:49.920 --> 00:18:50.900
How do I fix this, right?

00:18:50.900 --> 00:18:57.180
Yeah, I've had that issue many times where I've pulled it in a package that only works in Python 2 and I've had to update it to 3.

00:18:57.540 --> 00:19:11.960
So that's kind of where some of my experience in writing this course has come from has been, you know, updating other people's applications, but also maintaining some pretty big packages, you know, like Apache Lib Cloud, which supports both 2 and 3, which is the sort of portability approach.

00:19:11.960 --> 00:19:12.520
Pretty interesting.

00:19:12.520 --> 00:19:22.040
So before we move on to the goodness in Python 3, I want to try to spread this concept a little farther in the community.

00:19:22.040 --> 00:19:24.660
I talk about it all the time with Brian Okken on Python Byte.

00:19:25.380 --> 00:19:29.700
And if you say there's Python 2 and there's Python 3, they kind of feel like equivalent.

00:19:29.700 --> 00:19:31.420
Oh, there's these two things and they're similar.

00:19:31.420 --> 00:19:42.300
But if we all start calling Python 3 just Python and Python 2 legacy Python, when we have these conversations with managers and stuff, they're like, wait, our system is written in legacy Python.

00:19:42.300 --> 00:19:43.140
That doesn't sound good.

00:19:43.140 --> 00:19:43.660
What is that?

00:19:43.660 --> 00:19:47.440
So, you know, hopefully that term resonates with people.

00:19:47.440 --> 00:19:48.380
Yeah, I think so.

00:19:48.380 --> 00:19:49.320
Yeah, and definitely.

00:19:49.320 --> 00:19:58.080
I mean, some of the tools I work with a lot is Ansible, SaltStack, and another tool called StackStorm, which are about infrastructure automation.

00:19:58.080 --> 00:20:03.500
All of those were written in Python 2 originally, and they still primarily support Python 2.7.

00:20:03.500 --> 00:20:08.420
And these are huge code bases, and they're all trying to go through the migration at the moment to Python 3.

00:20:08.420 --> 00:20:14.040
So, yeah, it's definitely, they're seeing, I think, in the last 18 months, I think there's been a big shift.

00:20:14.040 --> 00:20:18.920
You know, people really starting to realize that this Python 3 thing isn't going to go anywhere.

00:20:18.920 --> 00:20:20.900
It's not going to go away.

00:20:20.900 --> 00:20:22.440
They're not going to go and say, you know what?

00:20:22.440 --> 00:20:23.500
We made a mistake.

00:20:23.500 --> 00:20:27.140
We're going to move back to byte strings instead of unicode strings.

00:20:27.140 --> 00:20:28.100
Yeah, yeah, exactly.

00:20:28.280 --> 00:20:33.720
All right, so let's maybe a good place to talk about what you get by moving to Python 3.

00:20:33.720 --> 00:20:34.420
What's better?

00:20:34.420 --> 00:20:35.520
Actually, what are the tradeoffs?

00:20:35.520 --> 00:20:36.160
What's worse?

00:20:36.160 --> 00:20:37.360
Things like that.

00:20:37.360 --> 00:20:42.660
The most wide-reaching shift is that Python 3 is faster.

00:20:42.660 --> 00:20:45.320
And I put that in air quotes, not that you can see them.

00:20:45.320 --> 00:20:53.660
If you go on speed.python.org, they run a continuous speed test against all the different versions of Python.

00:20:53.860 --> 00:20:58.060
And you can go and compare specific things between Python 2 and Python 3.

00:20:58.060 --> 00:21:02.460
And you can even compare different versions, say, like 3.5 versus 3.6.

00:21:02.460 --> 00:21:11.480
So in pretty much all of the cases that I've used, Python 3.5 and above is faster than Python 2.7.

00:21:11.480 --> 00:21:16.460
So you're probably going to see a speed improvement in moving to Python 3.

00:21:16.460 --> 00:21:20.880
Then you've got the async and await support in AsyncIO.

00:21:21.360 --> 00:21:25.620
And if you start to implement some of those features, you're definitely going to see a big speed improvement.

00:21:25.620 --> 00:21:27.240
So that's a big shift.

00:21:27.240 --> 00:21:29.360
That could just completely unlock it.

00:21:29.360 --> 00:21:30.620
You know, just one story on that.

00:21:30.620 --> 00:21:40.500
There was a guy who we talked about AIo HTTP client, which is a little like request, but lets you await while you're waiting on the server and do other requests.

00:21:40.660 --> 00:21:46.940
This guy was downloading a whole bunch of stuff off a server and it was taking like a day or something really, really long.

00:21:46.940 --> 00:21:50.220
And he switched from request to the async and await version.

00:21:50.220 --> 00:21:53.520
And then his machine crashed because he ran out of memory going so fast.

00:21:53.520 --> 00:21:55.620
Then he had to like throw out a little bit.

00:21:55.620 --> 00:21:58.640
But it was like it was like a minute versus a day.

00:21:58.640 --> 00:22:01.100
And it was something really super dramatic like that.

00:22:01.100 --> 00:22:06.900
So when you think about performance, it's never so simple as just it's always faster in this way or that.

00:22:06.900 --> 00:22:15.240
But it could be, well, there's this new algorithm or technique that's unlocked that is so much better that, you know, you shouldn't compare apples to apples.

00:22:15.240 --> 00:22:20.900
You should like pick the new shiny apple and compare that to the old, you know, the old unpolished apple.

00:22:20.900 --> 00:22:22.640
I don't know what the metaphor there is.

00:22:22.700 --> 00:22:24.860
But there's a bunch of great stuff in there.

00:22:24.860 --> 00:22:27.680
And then, you know, you talked about performance in terms of like speed.

00:22:27.680 --> 00:22:29.160
There's also improved memory, right?

00:22:29.160 --> 00:22:29.840
Yeah, definitely.

00:22:29.840 --> 00:22:34.020
There's been some significant improvements under the covers in Python 3.

00:22:34.020 --> 00:22:39.340
So, I mean, if you're just writing with Python 3, you'd never really see most of those changes.

00:22:39.340 --> 00:22:50.980
But in the actual execution, they did a lot of work to clean up a lot of the C code and definitely a lot of the optimizations around how it handles memory consumption and garbage collection and things like that.

00:22:50.980 --> 00:22:53.300
So, I've definitely seen some big improvements.

00:22:53.300 --> 00:23:00.400
There's some more coming in 3.7 in terms of how you call methods, which are part of a class instance.

00:23:00.400 --> 00:23:03.060
So, that's going to get a bit of a speed improvement, about 20%.

00:23:03.060 --> 00:23:06.740
So, yeah, there's lots of things coming and there's lots of reasons to move across.

00:23:06.740 --> 00:23:09.240
Yeah, there's actually a lot of stuff still coming.

00:23:09.240 --> 00:23:23.960
And I think it's interesting to, while we're on this performance bit, is I think this is around the 3.5 timeframe where the core developers were like, one of the actual things we need to focus on, one of the features is actually performance.

00:23:23.960 --> 00:23:28.700
Because it's great that we have async and await or type annotations or something like that.

00:23:28.700 --> 00:23:35.480
But if we can say your code uses half the memory or runs twice as fast, like that will solve the Python 2.3 divide right there, right?

00:23:35.480 --> 00:23:42.220
So, they've actually been putting a lot of effort into 3.5, 3.6 and 3.7 around that goal.

00:23:42.620 --> 00:23:43.120
Yeah, definitely.

00:23:43.120 --> 00:23:48.120
And also, Python 3 has kind of embraced iterators more in some of the built-ins.

00:23:48.120 --> 00:23:56.360
So, when you run filter, for example, or zip in Python 3, it will return an iterator instead of a list, which it would in Python 2.

00:23:56.360 --> 00:24:06.400
And now, when you're working with dictionaries, if you're calling keys, values, and items on a dictionary, it's going to return a view, which has an iterator built into it.

00:24:06.400 --> 00:24:08.020
Whereas in Python 2, that would be a list.

00:24:08.020 --> 00:24:12.560
So, yeah, they definitely kind of embraced iterators a lot more in the built-in types.

00:24:12.560 --> 00:24:16.980
And you'll get definitely memory, better memory consumption by using those.

00:24:16.980 --> 00:24:20.140
What's cool about that is a lot of those is you don't do anything.

00:24:20.140 --> 00:24:26.640
You could just, you already were like four in over in the keys or something, but now it just does it better.

00:24:26.640 --> 00:24:27.700
Really nice.

00:24:27.700 --> 00:24:28.020
Yeah.

00:24:28.020 --> 00:24:29.560
Just don't modify the dictionary.

00:24:29.560 --> 00:24:32.800
Oh, yeah, there's that.

00:24:35.260 --> 00:24:38.060
This portion of Talk Python To Me is brought to you by Linode.

00:24:38.060 --> 00:24:41.980
Are you looking for bulletproof hosting that's fast, simple, and incredibly affordable?

00:24:41.980 --> 00:24:46.920
Look past that bookstore and check out Linode at talkpython.fm/Linode.

00:24:46.920 --> 00:24:48.840
That's L-I-N-O-D-E.

00:24:48.840 --> 00:24:53.320
Plans start at just $5 a month for a dedicated server with a gig of RAM.

00:24:53.320 --> 00:24:58.340
They have 10 data centers across the globe, so no matter where you are, there's a data center near you.

00:24:59.020 --> 00:25:13.680
Whether you want to run your Python web app, host a private Git server, or file server, you'll get native SSDs on all the machines, a newly upgraded 200 gigabit network, 24-7 friendly support, even on holidays, and a seven-day money-back guarantee.

00:25:13.680 --> 00:25:16.020
Do you need a little help with your infrastructure?

00:25:16.020 --> 00:25:22.260
They even offer professional services to help you get started with architecture, migrations, and more.

00:25:22.260 --> 00:25:25.360
Get a dedicated server for free for the next four months.

00:25:25.360 --> 00:25:28.200
Just visit talkpython.fm/Linode.

00:25:28.200 --> 00:25:31.960
Let's talk a little bit about the carrots.

00:25:31.960 --> 00:25:33.120
You talked about the stick.

00:25:33.120 --> 00:25:37.860
Python 2 is going to go end of life in 2020, just to put a date on it.

00:25:37.860 --> 00:25:38.740
It's unclear.

00:25:38.740 --> 00:25:40.860
Have you heard anything about the actual day?

00:25:40.860 --> 00:25:41.860
I only know the year.

00:25:41.860 --> 00:25:45.840
I've heard it's PyCon, so it'll be PyCon 2020.

00:25:45.840 --> 00:25:47.620
It should be PyCon 2020.

00:25:47.620 --> 00:25:48.820
I hope they do that.

00:25:48.820 --> 00:25:51.580
I think there's going to be a burning of the source code or something.

00:25:51.580 --> 00:25:53.080
We'll see what happens on the day.

00:25:53.080 --> 00:25:57.440
That'd be a great party to celebrate what it's done for us, and like a goodbye party, basically.

00:25:57.440 --> 00:25:58.000
That'd be awesome.

00:25:58.460 --> 00:25:58.700
All right.

00:25:58.700 --> 00:25:59.620
So that's the stick.

00:25:59.620 --> 00:26:04.360
If you don't move and there's a security vulnerability or something bad, you're in a bad place.

00:26:04.360 --> 00:26:06.480
And it's not just the runtime, right?

00:26:06.480 --> 00:26:07.680
It's not just Python.

00:26:07.680 --> 00:26:11.200
It's the stuff that you might depend upon on top of it that hasn't moved.

00:26:11.200 --> 00:26:11.960
We'll get into that.

00:26:11.960 --> 00:26:13.440
But let's talk about the carrots.

00:26:13.440 --> 00:26:18.480
Like, what are some of the good things you get from moving to Python 3, other than the performance, which we already talked about?

00:26:18.480 --> 00:26:18.740
Yeah.

00:26:18.740 --> 00:26:23.800
So there's a lot of work with async and await being introduced as new keywords.

00:26:23.800 --> 00:26:28.940
Anyone who's worked with C# before will definitely be familiar with that syntax.

00:26:28.940 --> 00:26:33.280
So I think that's actually where it was inspired by was from the C# language.

00:26:33.520 --> 00:26:44.340
So really, this is a way of making it a bit easier to, I guess, work with asynchronous programming and to specify a bit more upfront that your functions can be awaited on.

00:26:44.340 --> 00:26:48.140
Especially when you're waiting on something, like you're calling a web service or a database.

00:26:48.140 --> 00:26:52.440
Your code could be doing so much more while you're just waiting for that network response, right?

00:26:52.440 --> 00:26:53.940
And this makes it drop dead simple.

00:26:53.940 --> 00:26:54.720
Yeah, definitely.

00:26:54.720 --> 00:26:57.760
I think there's still a bit of confusion with the sync and await.

00:26:57.760 --> 00:27:03.580
You know, we've not seen it that widely adopted because it really takes a bit of time to get around the concept.

00:27:03.580 --> 00:27:07.740
You can't just stick it in front of every method and expect your code to just run faster.

00:27:07.740 --> 00:27:10.100
It doesn't quite work like that.

00:27:10.100 --> 00:27:14.900
It's more about scalability than it is about pure performance increase for a single request, right?

00:27:14.900 --> 00:27:15.740
Yeah, definitely.

00:27:15.740 --> 00:27:20.600
And if you're working with REST APIs a lot, which in the modern world is pretty typical,

00:27:20.600 --> 00:27:26.900
then waiting for a response back from an API can keep your code, you know, hanging, essentially.

00:27:26.900 --> 00:27:29.560
So this is a great way to kind of speed that up.

00:27:29.560 --> 00:27:31.320
Dictionaries have got a bit of an overhaul.

00:27:31.320 --> 00:27:33.100
Especially around microservices, right?

00:27:33.100 --> 00:27:38.600
If you're calling more than it's not just one thing, you're calling a bunch of little services that you could really like clog that up.

00:27:38.600 --> 00:27:39.420
So this will unlock it.

00:27:39.420 --> 00:27:39.940
Yeah, definitely.

00:27:39.940 --> 00:27:40.920
Yeah, dictionaries.

00:27:40.920 --> 00:27:41.320
Go ahead.

00:27:41.320 --> 00:27:48.640
There's been a separation before between the order in which the dictionary keys have been set up based on how they were hashed.

00:27:49.120 --> 00:27:53.780
And in the newer versions of Python 3.6, they will be ordered by default.

00:27:53.780 --> 00:27:56.540
However, you should not assume that that will always be the case.

00:27:57.480 --> 00:28:04.780
So whilst it's a great feature, you know, you should probably still import from collections and get ordered dicks that way.

00:28:04.780 --> 00:28:07.300
So the performance improvements are pretty great.

00:28:07.300 --> 00:28:09.500
Also, there's the yield from keyword.

00:28:09.800 --> 00:28:19.020
So if you're working with generators, iterators a bit more, then, you know, you can leverage that to make sure you don't have to exhaust the generator before you yield it back.

00:28:19.020 --> 00:28:25.940
It really makes like that sort of consuming generators and passing them on or the really good and recursive sorts of situations.

00:28:25.940 --> 00:28:29.280
You can sort of looping over and yielding an individual item.

00:28:29.280 --> 00:28:30.440
You just say, here's a generator.

00:28:30.440 --> 00:28:32.900
Give me all this, you know, return all the stuff out of there.

00:28:32.900 --> 00:28:33.440
It's nice.

00:28:33.640 --> 00:28:34.100
Yeah, definitely.

00:28:34.100 --> 00:28:36.220
Type annotations is a great feature.

00:28:36.220 --> 00:28:40.860
And there's some really interesting ways to work with type annotations.

00:28:40.860 --> 00:28:46.400
Data classes is coming out in 3.7, which heavily leverages type annotations.

00:28:46.400 --> 00:28:54.220
So this is really a way of creating a class which represents a model of data, like the response you would get back from an API.

00:28:54.220 --> 00:28:57.140
And you can actually say, here are the types.

00:28:57.560 --> 00:29:04.560
And if you're working with a good IDE, it's going to give you a lot of hinting as to what methods are going to be on that instance and how to leverage it.

00:29:04.560 --> 00:29:06.460
So that's another great, great feature.

00:29:06.460 --> 00:29:08.760
Yeah, type annotations are really, really great.

00:29:08.760 --> 00:29:11.640
I was a little unsure what they would do to the language.

00:29:11.640 --> 00:29:17.480
Would they make it like C# or Java or something like that by, you know, making you say the types everywhere?

00:29:17.480 --> 00:29:22.880
And it's, I think a little sprinkling of them around is really working out quite nice from my experience.

00:29:22.880 --> 00:29:27.100
There's some really cool stuff that Facebook and Instagram folks are doing type annotations.

00:29:27.100 --> 00:29:40.040
So when we were talking earlier about the third-party packages that were not converted to Python 3, there was maybe a couple that you said, well, these aren't converted, but there's Python 3 equivalents.

00:29:40.040 --> 00:29:47.020
But what I'm noticing a lot more recently over the last year or so is things that are only available in Python 3.

00:29:47.020 --> 00:29:51.740
So really awesome third-party libraries you only get, right?

00:29:51.740 --> 00:29:53.540
So it's kind of almost the reverse, right?

00:29:53.540 --> 00:29:56.880
Like, well, if you don't move, you know, the world is moving on without you.

00:29:56.880 --> 00:29:58.700
So some of those come to mind.

00:29:58.700 --> 00:30:00.900
You already mentioned the REST full services.

00:30:00.900 --> 00:30:05.140
So there's API Star, which is the newfangled thing from Tom Christie.

00:30:05.140 --> 00:30:06.600
He does Django REST framework.

00:30:06.600 --> 00:30:14.480
But this is like it uses type annotations to inject parameters from the API and all sorts of really interesting stuff.

00:30:14.480 --> 00:30:21.100
There's Quart, which is a async and await enabled Flask API compatible derivative.

00:30:21.480 --> 00:30:23.840
And maybe the biggest one of all is Django 2.

00:30:23.840 --> 00:30:28.700
Yeah, Django 2 is a great step forward, I think, in terms of getting people to move to Python 3.

00:30:28.700 --> 00:30:36.940
And, you know, even using Python 3 as the default on their tutorials and stuff like that, you know, contributed a lot to the uptake.

00:30:37.440 --> 00:30:46.980
Looking at PyPI download stats for some of the packages that I maintain or work with, you still see like a large number of people using Python 2.

00:30:47.260 --> 00:30:52.140
And I guess the question is, do they know which version of Python they're consuming?

00:30:52.140 --> 00:30:58.420
Or are they just, you know, running some commands in a shell and it's pulling in packages from somewhere else and they don't really question it?

00:30:58.420 --> 00:31:00.300
So I think that's a really interesting point.

00:31:00.300 --> 00:31:06.840
And how much of that is continuous integration systems that happen to be set up to test on all the versions of Python?

00:31:06.840 --> 00:31:09.460
So they also are pulling Python 2.7.

00:31:09.860 --> 00:31:10.440
Yeah, definitely.

00:31:10.440 --> 00:31:12.780
And there's also still a challenge.

00:31:12.780 --> 00:31:18.300
I mean, Django, I think, have stepped forward and said, you know, from Django 2, we're just not going to support Python 2.

00:31:18.300 --> 00:31:23.860
And if you do pip install Django from a Python 2 distribution, it will just give you an error.

00:31:23.860 --> 00:31:27.260
Unfortunately, the error is not this doesn't support Python 2.

00:31:27.260 --> 00:31:28.560
It's something really obscure.

00:31:28.560 --> 00:31:32.680
It seems like that'd be pretty easy to put in the setup.py for that package.

00:31:32.680 --> 00:31:38.040
Just go, if version, you know, major is two or less, just raise this error.

00:31:38.040 --> 00:31:39.800
Sorry, wrong version or something, right?

00:31:39.800 --> 00:31:41.420
But it doesn't matter.

00:31:41.420 --> 00:31:47.580
The fact that Django, all Django work going forward is focused on Python 3 only is going to make such a huge difference.

00:31:47.580 --> 00:31:52.560
It let them forget trying to write two different versions of code for everything they're doing.

00:31:52.560 --> 00:31:55.240
They can just focus on new features, which is really good.

00:31:55.760 --> 00:32:00.360
You touched on data classes, but I don't know that everyone necessarily knows exactly what they are.

00:32:00.360 --> 00:32:03.320
Could you give us just like the quick summary of what those things are?

00:32:03.320 --> 00:32:04.460
There are three, seven.

00:32:04.460 --> 00:32:05.100
They're coming soon.

00:32:05.100 --> 00:32:07.480
It's very similar to the Attrs project.

00:32:07.480 --> 00:32:23.640
And this is if you have a class in Python, which only has really one purpose, and that is to store some data to reflect on some data that you might have in a database or in Mongo, or it gets responded from a REST API.

00:32:24.040 --> 00:32:28.420
So this would have a number of fields like ID and name and things like that.

00:32:28.420 --> 00:32:34.620
And you want to store them in an object so that you can write the fields and you can instantiate them quickly.

00:32:34.620 --> 00:32:38.980
So there's a lot of boilerplate code you typically have to write to do that sort of thing.

00:32:38.980 --> 00:32:41.500
If you use Atters, it definitely becomes a lot easier.

00:32:41.700 --> 00:32:50.120
But I can't think of how many times I've had to write a class with just a long list of, you know, init variables and then just basically storing each one of those in the class instance.

00:32:50.120 --> 00:32:54.220
So this basically gets rid of all that boilerplate for you and just automates it.

00:32:54.280 --> 00:33:03.720
So you declare a data class, you specify all the fields you want to have, and then it implements __init__() for you, dunderrepper, dunderstr, dundereq.

00:33:03.720 --> 00:33:06.200
And it's got some great other features as well.

00:33:06.360 --> 00:33:13.080
You can make it immutable by saying it's frozen, in which case it also implements dunder hash, which actually can be kind of challenging to do.

00:33:13.080 --> 00:33:13.720
Pretty cool.

00:33:13.880 --> 00:33:19.580
So yeah, these are all sorts of things that are in Python 3 only, right?

00:33:19.580 --> 00:33:24.040
We kind of have to wait for a little while because 3.7 is not yet out.

00:33:24.040 --> 00:33:25.380
We'll talk more about that in a bit.

00:33:25.380 --> 00:33:29.520
But maybe let's talk about some of the strategies of actually doing it.

00:33:29.520 --> 00:33:31.840
Okay, so people think they sound great.

00:33:31.840 --> 00:33:32.520
They're like, awesome.

00:33:32.520 --> 00:33:34.180
Data classes, I definitely want that.

00:33:34.180 --> 00:33:35.240
Ace to get away, I want that.

00:33:35.240 --> 00:33:38.060
But I have all this code that's in Python 2.

00:33:38.060 --> 00:33:39.660
How do I move it?

00:33:39.660 --> 00:33:40.820
What are my strategies?

00:33:40.820 --> 00:33:47.400
In the course, the way I try to explain this problem is, so I live near the beach on the east coast of Australia.

00:33:47.400 --> 00:33:55.240
And I went down to the beach and basically shot a series of photos of me jumping from one rock to another one with some water in the middle.

00:33:55.240 --> 00:34:02.000
And what I was trying to illustrate is that you're on Python 2 and you're on solid ground and you want to move to Python 3.

00:34:02.000 --> 00:34:08.600
And it's extremely tempting to just say, okay, let's just go and rewrite the whole thing and jump from one side to the other.

00:34:08.600 --> 00:34:24.400
But the real challenge with that, I think, is if it's a large application, then the amount of time you're going to spend suspended in midair where your application neither works in Python 2 or Python 3 could be quite a long period of time.

00:34:24.400 --> 00:34:31.420
And if any customers come to you or you need to make any changes to the code base or whatever during that time, it's going to be really challenging.

00:34:31.420 --> 00:34:37.360
So, you know, you could go and create a separate branch of your application and just rewrite everything in Python 3.

00:34:37.360 --> 00:34:49.060
But definitely you're going to get also the temptation to go, oh, well, since we're rewriting in Python 3, you know, let's also go and rewrite everything in microservices and use the latest design patterns.

00:34:49.060 --> 00:34:58.220
So, you know, let's just move to the newest version of Python 2, you know, let's re-engineer the whole application from scratch.

00:34:58.220 --> 00:35:03.020
So, you know, if you've got a big enough team and a big enough budget, then great, you know, knock yourself out.

00:35:03.020 --> 00:35:12.880
I think a more realistic approach is to start to introduce Python 3 support in specific modules or parts of your application.

00:35:13.440 --> 00:35:15.840
So this is kind of like the portability approach.

00:35:15.840 --> 00:35:26.560
Now, you probably don't want to support Python 2 for very long after you've done the move, but it's a lot easier to introduce Python 3 support to an already working application.

00:35:26.560 --> 00:35:31.340
I think that's the most important thing is that your application today, or at least I hope it does, works.

00:35:31.340 --> 00:35:40.560
So, you know, really introducing Python 3 support piece by piece means that you can maintain a working application whilst moving to Python 3.

00:35:40.560 --> 00:35:43.220
That's going to be a lot easier to, you know, land with the managers.

00:35:43.220 --> 00:35:49.160
Yeah, it definitely is to say our code, we're making it better and eventually we'll be off of legacy Python.

00:35:49.160 --> 00:35:51.700
But at any moment we could ship it.

00:35:51.700 --> 00:35:57.360
And it's just as we work with it, it gets closer and closer to that goal, which is really awesome.

00:35:57.360 --> 00:36:05.080
I think one of the most insane stories and insane in an awesome way stories of this is what Instagram was doing.

00:36:05.080 --> 00:36:10.980
And they really documented this at PyCon 2017 with one of the keynotes there.

00:36:10.980 --> 00:36:12.840
Yeah, the Instagram story is amazing.

00:36:12.840 --> 00:36:19.320
And I think it's a great case, you know, to talk about the power of Python and the scalability of Python 3 as well.

00:36:19.320 --> 00:36:24.360
You know, whenever I hear someone say Python's not scalable, I just say, oh, how many users do you have?

00:36:24.360 --> 00:36:25.900
Is it more than, you know, half a billion?

00:36:25.900 --> 00:36:28.280
Well, you probably don't have an issue.

00:36:28.280 --> 00:36:29.780
Exactly.

00:36:29.780 --> 00:36:31.220
What response time do you need?

00:36:31.220 --> 00:36:33.680
Six milliseconds is not fast enough, right?

00:36:33.740 --> 00:36:37.000
I mean, maybe, maybe very rarely, but yeah, it's amazing.

00:36:37.000 --> 00:36:47.100
So at PyCon 2017, Instagram, a couple of the engineers from there got up and they gave this really compelling presentation of they were on an old version of Django.

00:36:47.100 --> 00:36:49.660
And I don't even think it was necessarily Python 2.7.

00:36:49.660 --> 00:36:52.340
It was like a pretty old version of Python.

00:36:52.340 --> 00:36:54.340
It definitely was some variant of two.

00:36:55.080 --> 00:37:09.020
And they talked about how they moved all of Instagram over to Python 3 without taking down the site, without creating a separate branch and continuously shipping and running their program.

00:37:09.580 --> 00:37:17.780
And so some of the techniques were things like, we're going to make our code run on both Python 2 and 3 instead of just rewriting it.

00:37:17.780 --> 00:37:19.420
So we're just going to make it compatible with both.

00:37:19.420 --> 00:37:24.280
And then they started running unit tests on both versions, Python 2 and Python 3.

00:37:24.280 --> 00:37:27.320
And they'd only deployed a production if it would pass on both.

00:37:27.320 --> 00:37:29.180
But for a long time, production was Python 2.

00:37:29.180 --> 00:37:38.640
And then slowly they started like running production Python 3 versions, but only giving those to the internal people and, you know, just slowly growing up.

00:37:38.700 --> 00:37:39.520
But it was really great.

00:37:39.520 --> 00:37:40.800
They did this major conversion.

00:37:40.800 --> 00:37:42.140
They didn't branch.

00:37:42.140 --> 00:37:44.460
They didn't, like, kill the momentum for a while.

00:37:44.460 --> 00:37:45.420
It's pretty cool.

00:37:45.420 --> 00:37:48.920
Yeah, I think it kind of goes back to some other examples.

00:37:48.920 --> 00:37:57.880
I think of Windows Vista as well as a great example of where to be cautious of reintroducing bugs that you solved, you know, decades ago.

00:37:57.880 --> 00:38:04.140
I remember the first time they released the beta, they'd say, oh, we've rewritten the TCP IP stack from scratch.

00:38:04.820 --> 00:38:13.180
And they'd actually re-implemented something called ping of death, which is like a late 1980s security hole in the stack.

00:38:13.180 --> 00:38:20.760
So you should definitely not dismiss a lot of the robustness of your old application, which is why testing is so important in the migration story.

00:38:20.760 --> 00:38:25.400
So, you know, as I talk about in the course, Python is a very, very dynamic language.

00:38:25.400 --> 00:38:30.320
You know, you can pretty much override anything and all sorts of crazy things can happen at runtime.

00:38:30.640 --> 00:38:37.760
So the better testing coverage and, you know, the more robust test suite you have, the easier it's going to be to move.

00:38:37.760 --> 00:38:48.200
And then also making sure that you don't reintroduce bugs that you've already solved, which is why I'm more of a fan of the sort of gradual migration process than the completely rewrite.

00:38:48.200 --> 00:38:53.820
Because I'd be concerned that you'd introduce bugs, which you either didn't have before or you'd already solved.

00:38:54.060 --> 00:38:59.600
Yeah, there's always so the temptation is right there to just say, you know, we could rewrite this.

00:38:59.600 --> 00:39:01.200
We could do it better.

00:39:01.200 --> 00:39:02.920
And that may be a great thing.

00:39:02.920 --> 00:39:07.820
But I would say, like you pointed out before, separate that from the concept of moving to Python 3.

00:39:07.820 --> 00:39:12.580
Do the move and then think about a restructuring of some part, if that makes sense.

00:39:12.580 --> 00:39:15.500
But there's no reason to conflate them and make them both riskier.

00:39:15.500 --> 00:39:16.200
Yeah, definitely.

00:39:16.500 --> 00:39:16.640
Yeah.

00:39:16.640 --> 00:39:19.260
So how about some of the tools that people might use?

00:39:19.260 --> 00:39:19.820
Right.

00:39:19.820 --> 00:39:22.960
There's some linters and other things that might work, right?

00:39:22.960 --> 00:39:23.180
Yeah.

00:39:23.180 --> 00:39:28.880
So probably the easiest one to use is when you're running Python, you can use the dash three flag.

00:39:28.880 --> 00:39:37.260
So if it's Python 2, you can use the dash three flag and it will actually give you warnings on the shell where you've used something which is not compatible in Python 3.

00:39:37.260 --> 00:39:39.940
So that's kind of like the easiest way of doing things.

00:39:39.940 --> 00:39:41.680
You've also got some linters.

00:39:42.080 --> 00:39:51.140
So if you're using pilot as an example, if you're using pilot from Python 2, it will only give you linting warnings that are relevant to Python 2.

00:39:51.140 --> 00:39:54.480
And if you're calling it from Python 3, it will give you newer warnings.

00:39:54.480 --> 00:40:02.840
So linters can be really useful in terms of understanding or at least trying to guess what types certain variables are.

00:40:02.840 --> 00:40:05.840
Tox is a tool which I highly recommend.

00:40:06.660 --> 00:40:13.880
So let's say you already, at least I hope, you've got some sort of tests written in unit test or pytest or something like that.

00:40:13.880 --> 00:40:18.760
Basically just introducing Tox, which isn't actually that complicated.

00:40:18.760 --> 00:40:23.220
It's just pip installs Tox and then you can run Tox hyphen quick start.

00:40:23.220 --> 00:40:27.380
And it will just ask you a couple of questions about which Python versions you want to test.

00:40:27.540 --> 00:40:36.600
So Tox, basically, each time you run it, it will create a virtual environment for each version of Python and it will run your test suite for those different versions.

00:40:36.600 --> 00:40:38.380
So if you just wanted to target.

00:40:38.380 --> 00:40:39.320
That's really cool.

00:40:39.320 --> 00:40:40.580
Yeah, it's fantastic.

00:40:40.580 --> 00:40:47.140
I think people have been a little bit put off because it takes it takes longer to run the test suite on multiple versions.

00:40:47.540 --> 00:40:53.960
If you install a tool called Detox, it will actually run the multiple versions in different threads.

00:40:53.960 --> 00:40:57.440
So it takes the same amount of time as it would just to run your normal test suite.

00:40:57.440 --> 00:40:58.600
Yeah, that's pretty cool, actually.

00:40:58.600 --> 00:40:59.140
Detox.

00:40:59.140 --> 00:40:59.540
I love it.

00:40:59.540 --> 00:41:00.080
Yeah.

00:41:00.080 --> 00:41:10.640
So the idea is you can basically have your continuous integration or just your unit test execution basically automatically run on all the versions that you check.

00:41:10.640 --> 00:41:13.140
And it could even be things like PyPy, right?

00:41:13.140 --> 00:41:15.980
It doesn't have to be just Python 2 versus Python 3 and CPython.

00:41:15.980 --> 00:41:16.900
Yeah, definitely.

00:41:17.180 --> 00:41:21.440
PyPy runs on a two equivalent syntax.

00:41:21.440 --> 00:41:25.280
And then PyPy 3 has still been a bit of a challenge, actually.

00:41:25.280 --> 00:41:28.420
We've had challenges in pytest support.

00:41:28.420 --> 00:41:30.800
I'm not sure if they've fixed that yet with PyPy 3.

00:41:30.800 --> 00:41:32.780
It's been a bit of a thing to get over.

00:41:32.780 --> 00:41:40.520
So yeah, I do caution people who are using PyPy and moving to PyPy 3 and the supportability of third-party packages.

00:41:40.520 --> 00:41:42.200
Yeah, that's definitely a little bit tricky.

00:41:42.200 --> 00:41:45.160
So it sounds like you definitely recommend continuous integration.

00:41:45.160 --> 00:41:46.580
Yeah, continuous integration.

00:41:46.820 --> 00:41:51.780
So start off simple, which is just run your tests in Python 3.

00:41:51.780 --> 00:42:05.680
And another thing you can do is Python actually has a compile-all method, which is in the standard library, which will go through a directory that you have, look at all the Python source files, and try and just compile them all up front.

00:42:06.340 --> 00:42:14.380
That's another handy trick you can use just to check, just to do a quick sense check of the syntax that you've got and make sure you've not done anything blaringly obvious.

00:42:14.380 --> 00:42:22.340
Right, because normally, because normally, you've got to do a quick sense of the Python files, and you've got to do a quick sense of Python 3.

00:42:22.340 --> 00:42:27.040
But you can sort of force Python to parse all the files and turn them into PYCs, right?

00:42:27.040 --> 00:42:27.720
Yeah, definitely.

00:42:27.720 --> 00:42:40.860
And then as you start to introduce unit testing, think about, in particular, the string change in Python 3, which we'll probably talk about in a minute, and having more testing on your application for Unicode strings.

00:42:41.340 --> 00:42:43.920
So that's definitely going to bring up some challenges in your existing code.

00:42:43.920 --> 00:42:49.520
This portion of Talk Python To Me has been brought to you by Rollbar.

00:42:49.520 --> 00:42:53.200
One of the frustrating things about being a developer is dealing with errors.

00:42:53.200 --> 00:43:02.200
Relying on users to report errors, digging through log files, trying to debug issues, or getting millions of alerts just flooding your inbox and ruining your day.

00:43:02.200 --> 00:43:08.940
With Rollbar's full-stack error monitoring, you get the context, insight, and control you need to find and fix bugs faster.

00:43:09.400 --> 00:43:12.940
Adding Rollbar to your Python app is as easy as pip install Rollbar.

00:43:12.940 --> 00:43:17.180
You can start tracking production errors and deployments in eight minutes or less.

00:43:17.180 --> 00:43:21.360
Are you considering self-hosting tools for security or compliance reasons?

00:43:21.360 --> 00:43:24.940
Then you should really check out Rollbar's Compliance SaaS option.

00:43:24.940 --> 00:43:34.320
Get advanced security features and meet compliance without the hassle of self-hosting, including HIPAA, ISO 27001, Privacy Shield, and more.

00:43:34.680 --> 00:43:35.700
They'd love to give you a demo.

00:43:35.700 --> 00:43:37.400
Give Rollbar a try today.

00:43:37.400 --> 00:43:41.080
Go to talkpython.fm/Rollbar and check them out.

00:43:43.220 --> 00:43:54.020
Yeah, when I hear people working with challenges in their code or problems they have converting, number one is there's some package I depend upon and it's not working.

00:43:54.020 --> 00:43:55.860
But number two is strings.

00:43:57.160 --> 00:44:05.780
It seems like that catches a lot of people, especially if they're working at the network layer where they're getting bytes off the network and then they want to do string likes.

00:44:05.780 --> 00:44:08.440
It's really ultimately supposed to represent a string or something?

00:44:08.700 --> 00:44:13.500
Yeah, I hugely support the decision in Python 3 to change the core types.

00:44:13.500 --> 00:44:15.780
I think it was the right choice.

00:44:15.780 --> 00:44:19.120
So kind of going back to, I guess, what changed.

00:44:19.120 --> 00:44:23.700
In Python 2, there's a type called string or STR.

00:44:24.440 --> 00:44:28.520
And I think people have used it for two different purposes.

00:44:28.520 --> 00:44:30.580
And this is kind of coming back to some of the issue.

00:44:30.580 --> 00:44:33.120
It's an 8-bit string type.

00:44:33.120 --> 00:44:45.660
And it just happens that the ASCII characters, which are the English alphabet plus things like, you know, basically everything you would find on an American or an English keyboard fits in the ASCII character set.

00:44:45.900 --> 00:44:51.400
Now, that set, because it doesn't have that many characters, fits inside 7-bits.

00:44:51.400 --> 00:45:00.520
So you can use it as a way of transferring, sending strings, which only include American English characters.

00:45:00.520 --> 00:45:07.340
So people would use the STR type for text and also use it for binary data.

00:45:07.340 --> 00:45:11.500
So if they were reading images, as an example, you'd still use the STR type.

00:45:11.500 --> 00:45:16.280
So I think part of the issue has been that there was also a Unicode type.

00:45:16.280 --> 00:45:23.380
So if you're a really good citizen, you know, you would try and use the Unicode type, but it would introduce all these weird issues in Python 2.

00:45:23.380 --> 00:45:32.540
So one of those issues is that Python 2 will try its hardest if you concatenate a string type and a Unicode type.

00:45:32.540 --> 00:45:34.920
So this is the byte string and the Unicode string.

00:45:34.920 --> 00:45:40.980
It would basically try and convert the byte string to Unicode and then bring the two things together.

00:45:40.980 --> 00:45:48.220
Now, what that leads to is runtime errors specific to the values that you have in the strings.

00:45:48.220 --> 00:45:58.560
So this is kind of the Achilles heel of Python 2's Unicode support is that it just tries to basically run either an encode or a decode at runtime.

00:45:58.560 --> 00:46:04.420
And then if you've got specific values in the strings, then your whole application can just suddenly crash.

00:46:04.420 --> 00:46:06.580
So, yeah, that's been a big challenge.

00:46:06.580 --> 00:46:11.280
And it may well be another culture that you don't typically interact with.

00:46:11.280 --> 00:46:13.840
So it's a really hard edge case to catch, right?

00:46:13.840 --> 00:46:19.200
Like Mandarin or something like that, that I would never type that into my program.

00:46:19.200 --> 00:46:20.420
But other people would.

00:46:20.420 --> 00:46:21.700
It totally makes sense to them, right?

00:46:21.700 --> 00:46:30.080
The case I talk about in the course is, you know, the argument has always been, oh, we don't need Unicode because, you know, all of our customers use English.

00:46:30.080 --> 00:46:35.680
But there's been this new culture called emojis, which is introducing Unicode characters and everything.

00:46:35.680 --> 00:46:44.100
People want to put smiley faces and snakes in their display names or their profile names or, you know, put all sorts of weird things in their addresses.

00:46:44.540 --> 00:46:52.380
So actually, I think the culture of emoji has pushed forward and people trying to think about Unicode support.

00:46:52.380 --> 00:46:54.980
And that's uncovered a lot of the challenges in Python 2.

00:46:54.980 --> 00:47:04.180
So really what they did in Python 3 is to make it a lot more explicit about binary data and text data.

00:47:04.440 --> 00:47:10.520
So the, what was the Unicode type in Python 2 is now the string type in Python 3.

00:47:10.520 --> 00:47:19.000
And if you want to use binary data from reading and writing to say images or network sockets, there's a new type called bytes.

00:47:19.000 --> 00:47:21.300
And those things are very, very different.

00:47:21.300 --> 00:47:31.340
And if you have a string literal, so if you just say hello world in double quotes or single quotes in Python 3, that will be a Unicode string by default.

00:47:31.340 --> 00:47:33.020
And I think that's caught a lot of people out.

00:47:33.500 --> 00:47:37.860
If you're just passing strings around your application, you probably won't see the difference.

00:47:37.860 --> 00:47:48.700
But if you're reading and writing from sockets or files or interacting with libraries which return binary data, then that's when you start to see these Unicode encode errors and things like that.

00:47:48.700 --> 00:47:52.240
Yeah, definitely right at the boundaries of your app where stuff comes in and out.

00:47:52.240 --> 00:47:53.360
That's the danger zone.

00:47:53.360 --> 00:47:59.280
So one other tool I wanted to just throw out there while you're on this topic is mypy.

00:47:59.660 --> 00:48:05.660
So mypy is an optional static typing, linter typing for Python 2 and for Python 3.

00:48:06.580 --> 00:48:18.560
And if you're going to move your code over, one thing you can do is you can add typing to it in the Python 2 style to actually verify that it's working the way you think and everything.

00:48:18.560 --> 00:48:24.800
And so then when you do the switch, you can make sure that, for example, you haven't switched to string for bytes or something weird like that, right?

00:48:24.800 --> 00:48:27.800
It's still all the same types that you were thinking about.

00:48:28.120 --> 00:48:33.220
This is a major project that Guido von Rossum is working on, so it's pretty significant.

00:48:33.220 --> 00:48:34.800
Yeah, that sounds really useful.

00:48:34.800 --> 00:48:42.720
Another thing you can do is just to think about when you've got strings in your code, whether they're text or whether they're binary.

00:48:42.720 --> 00:48:50.660
And I think if you just, instead of thinking about the actual types, if you think about the purpose of the string, then it's a lot easier to pull across to Python 3.

00:48:50.780 --> 00:48:51.560
Yeah, for sure.

00:48:51.560 --> 00:48:55.840
Do you want to talk about just some of the things that change, like, say, the language level?

00:48:55.840 --> 00:48:57.320
So we talked about strings.

00:48:57.320 --> 00:48:59.620
There used to be longs.

00:48:59.620 --> 00:49:00.540
Are there still longs?

00:49:00.540 --> 00:49:04.480
Long in Python 2 is a very, very, very long number.

00:49:04.480 --> 00:49:09.900
You know, almost infinitely long, depending on how much memory you have on the computer.

00:49:09.900 --> 00:49:15.500
And then int was a fixed length numeric type in Python 2.

00:49:15.500 --> 00:49:20.500
So what they've actually done is they renamed the old long type to the int type.

00:49:21.020 --> 00:49:22.520
And then the old int type is no more.

00:49:22.520 --> 00:49:29.060
So if you ever refer to long in your code, then it will give you an error in Python 3.

00:49:29.060 --> 00:49:29.940
There's ways around that.

00:49:29.940 --> 00:49:35.800
You can use the future package, and you can actually import the long type from there, which just redirects to the new int type.

00:49:35.800 --> 00:49:37.200
That's a way of safeguarding.

00:49:37.200 --> 00:49:47.540
Also, if you're working with numbers and you're using the division operator, so, you know, the forward slash, if you divided two integers in Python 2, it would return an integer.

00:49:47.760 --> 00:49:51.800
So if you said one slash two in Python 2, it would give you zero.

00:49:51.800 --> 00:49:59.460
In Python 3, the division operator will give you a floating point number back if that's the actual result.

00:49:59.460 --> 00:50:03.880
So if you do one slash two in Python 3, it will give you 0.5.

00:50:04.340 --> 00:50:07.000
So that's another thing that catches a lot of people out.

00:50:07.000 --> 00:50:09.200
That's got to be super subtle as well.

00:50:09.200 --> 00:50:13.120
If you're saying, do this division, then if it equal equals zero, we do this.

00:50:13.120 --> 00:50:13.980
Otherwise, we do that.

00:50:13.980 --> 00:50:15.880
And of course, no, no, not so much.

00:50:15.880 --> 00:50:20.560
We have a couple of tools that we'll go through and adjust some of this stuff, right?

00:50:20.740 --> 00:50:26.440
Yeah, they can at least make it easier to expect the new behavior when you're running Python 2.

00:50:26.440 --> 00:50:29.560
So I mentioned the future module that definitely does that.

00:50:29.560 --> 00:50:37.240
So you can actually import the new behaviors of certain operators and types into Python 2 using packages like future.

00:50:37.560 --> 00:50:40.180
Have you done anything with 2 to 3 or 6?

00:50:40.180 --> 00:50:46.160
So I know those are not the same purpose, but they're both sort of tools or packages to help you adjust.

00:50:46.160 --> 00:50:47.660
There's three ways of doing it.

00:50:47.660 --> 00:50:51.820
So there's the 2 to 3 application, which comes with Python 3.

00:50:51.820 --> 00:50:54.740
So it's maintained by the core team.

00:50:55.460 --> 00:51:01.460
And that will do its best to automatically make your code portable or executable in Python 3.

00:51:01.460 --> 00:51:09.640
I think when Python 3 initially came out, they said, if you pip install a package and you want it to work in Python 3,

00:51:09.640 --> 00:51:15.020
then inside setup.py, you could actually run 2 to 3 over the source code.

00:51:15.020 --> 00:51:16.260
That's pretty ambitious.

00:51:16.260 --> 00:51:17.300
Yeah, it's ambitious.

00:51:17.300 --> 00:51:21.020
And people pretty quickly realized it's not actually that straightforward.

00:51:21.340 --> 00:51:29.520
So, you know, what I recommend to people is that you can use these tools to fix really mundane things,

00:51:29.520 --> 00:51:33.120
like the use of the print statement and changing it to a print function.

00:51:33.120 --> 00:51:38.160
So, you know, that's something that's not really, you're not really adding much value by doing it by hand.

00:51:38.160 --> 00:51:42.840
So 2 to 3, and there's also two other tools called Futurize and Modernize.

00:51:42.840 --> 00:51:46.460
I'd actually recommend using Futurize instead of 2 to 3.

00:51:46.460 --> 00:51:48.720
It bases itself on the future package.

00:51:49.320 --> 00:51:54.480
And it has a lot of the logic that 2 to 3 has for fixing your code for Python 3.

00:51:54.480 --> 00:52:02.320
But it also has a number of other features, basically, for catching common things that are going to cause issues with portability.

00:52:02.320 --> 00:52:11.580
So if you install Futurize and you run it over your code base, it was going to give you a bunch of changes that it wants to make.

00:52:11.580 --> 00:52:13.660
Those would be in the form of a patch file.

00:52:13.660 --> 00:52:16.180
So it's going to say, I'm going to replace this line with that line.

00:52:16.180 --> 00:52:20.700
The biggest issue with those tools, I think, is that the code that it creates can be pretty messy.

00:52:20.700 --> 00:52:28.540
You know, we talked about dictionaries earlier and how keys, values, and items now return views instead of lists in Python 3.

00:52:28.740 --> 00:52:37.680
So the way those tools will try and get around that is that whenever you call, you know, dot items in a dictionary, it will wrap that in a list constructor.

00:52:37.680 --> 00:52:39.960
So the tool will add that for you.

00:52:39.960 --> 00:52:45.540
And then if you do something in the same line to use another iterator, it will do it again.

00:52:45.740 --> 00:52:51.920
So the code that it can create sometimes can be, you know, eye-wateringly ugly and actually not what you intended.

00:52:51.920 --> 00:52:55.020
Yeah, it could actually change the performance adversely as well.

00:52:55.020 --> 00:52:55.480
Absolutely.

00:52:55.480 --> 00:53:01.560
And there's a reason that they moved to views and there was a reason that people would typically use iter items in Python 2.

00:53:01.720 --> 00:53:03.700
So I kind of warn against that.

00:53:03.700 --> 00:53:14.220
All of these tools, 2 to 3, Futurize and Modernize, you can toggle which fixes, which is the name for the logic that changes the code you want to use.

00:53:14.220 --> 00:53:22.040
So what I recommend is that when you install them, you run in Futurize-L and it will give you a list of all the fixes you can use.

00:53:22.040 --> 00:53:31.460
And then test out different ones, see which ones you're happy with, and then explicitly turn them on or explicitly disable the ones that you definitely don't want.

00:53:31.460 --> 00:53:33.860
You know, I mentioned the dictionary one as an example.

00:53:33.860 --> 00:53:39.480
I'd be really cautious of that because it can introduce, you know, some pretty ugly code and also impact the performance.

00:53:39.820 --> 00:53:46.380
There are some others as well in terms of the way it handles strings and tuples, which can introduce a bit of mess.

00:53:46.380 --> 00:53:54.660
So the tools, I think, are good for starting off by fixing some of the really mundane replacements, such as print statements.

00:53:54.660 --> 00:53:58.000
Yeah, like if you could just have that happen, that would be nice, right?

00:53:58.000 --> 00:54:02.080
It's not going to completely move your application for you, so don't expect it to do that.

00:54:02.080 --> 00:54:04.120
Yeah, but it can do some of the low-level work.

00:54:04.120 --> 00:54:07.600
I also brought up 6, the package SIX.

00:54:08.080 --> 00:54:13.600
That's not a tool that you'd run that changes stuff, but it's like a package you can use to help create compatible code, right?

00:54:13.600 --> 00:54:16.420
Some of the big changes in Python 3 are on the core types.

00:54:16.420 --> 00:54:25.740
And let's say, for example, you've got some code which says, is this variable that I've just received in my function a string?

00:54:26.460 --> 00:54:31.100
So in Python 2, it could be an str type or it could be a unicode type.

00:54:31.100 --> 00:54:33.740
Both of those inherit from something called base string.

00:54:33.740 --> 00:54:38.440
So you can say, if is instance, you know, my variable, base string.

00:54:38.440 --> 00:54:42.920
So that doesn't work in Python 3 because base string is no more.

00:54:42.920 --> 00:54:46.540
And the string type and the bytes type are very, very distinct.

00:54:47.240 --> 00:54:51.480
So what SIX can do is it basically just introduces a redirect.

00:54:51.480 --> 00:55:01.400
So if you want to check if something is a string, you can say if this variable is instance of SIX.string types or SIX.numeric types.

00:55:01.400 --> 00:55:04.340
And then it's a really simple module.

00:55:04.340 --> 00:55:06.600
It's only about 600 lines of code, I think.

00:55:06.760 --> 00:55:09.100
And I recommend that people actually read the source code.

00:55:09.100 --> 00:55:10.500
So it's not magic.

00:55:10.500 --> 00:55:11.580
It's actually pretty straightforward.

00:55:11.580 --> 00:55:15.840
It's just a series of if statements that says, if Python 3 do that, if Python 2 do this.

00:55:15.840 --> 00:55:24.360
But it's pretty handy for the thing you really want to avoid when you're doing portability is having lots of, if it's Python 3, go and run this piece of code.

00:55:24.360 --> 00:55:26.580
If it's Python 2, go and run this piece of code.

00:55:26.580 --> 00:55:30.380
Because in the long term, you know, that's going to be really hard to maintain.

00:55:30.380 --> 00:55:31.600
It's hard to read.

00:55:31.600 --> 00:55:32.340
It's hard to follow.

00:55:32.340 --> 00:55:35.960
And it's going to introduce all these weird edge cases in your testing as well.

00:55:35.960 --> 00:55:38.500
Yeah, you definitely want to stay away from that if possible.

00:55:38.500 --> 00:55:38.960
Very cool.

00:55:38.960 --> 00:55:39.820
All right.

00:55:39.820 --> 00:55:46.280
So let's sort of wrap this up a little bit and talk about maybe what do you do after you've migrated?

00:55:46.280 --> 00:55:47.000
Now what?

00:55:47.000 --> 00:55:52.700
After you've migrated, if you're doing the portability approach, then you've got an application that works on 2 and 3.

00:55:52.700 --> 00:55:56.580
And then what I recommend you do is to basically start dropping 2 support.

00:55:56.580 --> 00:56:01.380
So the ways that you can do that is to remove some of the portability code that you have.

00:56:01.680 --> 00:56:05.440
And then you can, you know, if you're using Python 3 as your execution.

00:56:05.440 --> 00:56:09.380
So in production, basically you're calling Python 3, not Python 2.

00:56:09.380 --> 00:56:12.060
Then you can start to use some of the new features.

00:56:12.060 --> 00:56:17.740
So you can actually start to implement async and await and data classes and f-strings.

00:56:17.740 --> 00:56:18.040
Yeah.

00:56:18.040 --> 00:56:18.820
f-strings are great.

00:56:18.960 --> 00:56:20.100
Yeah, we haven't even talked about those.

00:56:20.100 --> 00:56:24.500
And those are really, really awesome, both in from a performance and a syntax perspective.

00:56:25.120 --> 00:56:29.320
So one thing that I wanted to throw in here at the end is I can see it's a challenge.

00:56:29.320 --> 00:56:39.580
It's like if you're writing some code that's going to go run on a server, really the one that comes to mind is macOS because they ship with Python 2, but they don't ship with Python 3, right?

00:56:39.580 --> 00:56:44.860
So if you're writing some code, you could say, well, I'm going to write in Python 2 and just it'll work on the Mac with nothing.

00:56:44.960 --> 00:56:47.440
But if I write in Python 3, they've got to install Python.

00:56:47.440 --> 00:56:48.240
There's all these challenges.

00:56:48.240 --> 00:56:54.040
So you can do things like use PyInstaller and package up an independent version of your app, right?

00:56:54.040 --> 00:56:59.160
Sort of make it easier to deploy anyway, but also get around some of those challenges, right?

00:56:59.160 --> 00:57:07.980
Even in the Python 3 support, which version of Python 3 you get with different Linux distributions, as an example, is, you know, it's kind of all over the place.

00:57:07.980 --> 00:57:12.920
So expecting that people have already installed Python 3 is then, okay, which version?

00:57:12.920 --> 00:57:14.680
Is it 3.4, 3.5, or 3.6?

00:57:14.740 --> 00:57:15.220
Yeah, exactly.

00:57:15.220 --> 00:57:19.880
So yeah, that's definitely been a challenge, I think, because 2.7 has been out for so long.

00:57:19.880 --> 00:57:23.860
You know, you can pretty much assume that people have already got 2.7 installed.

00:57:23.860 --> 00:57:27.820
But yeah, if your package runs on 3, then you've got to figure that out as well.

00:57:27.820 --> 00:57:35.700
Yeah, I've accidentally taken down, I think the training website, maybe the podcast website, by using f-strings in a little side utility.

00:57:35.700 --> 00:57:41.840
It's not actually run by the website, but the web framework scans all the files to look for like views and stuff.

00:57:42.180 --> 00:57:45.240
And it hit a F-string section, and it couldn't start the web app.

00:57:45.240 --> 00:57:46.460
And I'm like, oh, really?

00:57:46.460 --> 00:57:48.000
This is what takes it down?

00:57:48.000 --> 00:57:51.160
Because it was 3.5, not 3.6 on Ubuntu or something silly like this.

00:57:51.160 --> 00:57:54.140
So yeah, I guess it's still a modern challenge to some degree, right?

00:57:54.140 --> 00:57:57.060
There's still one thing I think which is missing at the moment.

00:57:57.060 --> 00:58:03.720
And that's dropping support for Python 2 in a nice way for packages that you push to PyPI.

00:58:03.720 --> 00:58:07.000
So, you know, I've got a number of packages that I maintain.

00:58:07.000 --> 00:58:09.140
I'd love to drop Python 2 support.

00:58:09.620 --> 00:58:15.300
But I want people who are still using Python 2 to end up installing the version which still supported it.

00:58:15.300 --> 00:58:26.740
And I think something we're still missing is, you know, if you pip install some package, and you don't pin the version, and it detects that you're running Python 2, it says, oh, okay, you want the legacy version, not the new version.

00:58:26.740 --> 00:58:28.640
And it installs the legacy version.

00:58:28.640 --> 00:58:29.260
That would be nice.

00:58:29.260 --> 00:58:29.960
So you can all maintain.

00:58:29.960 --> 00:58:34.880
Yeah, I think that would be really helpful, you know, to help people start to drop Python 2 support.

00:58:34.880 --> 00:58:41.160
Because at the moment, if you know that 70% of your downloads are in Python 2, you still have to kind of maintain both.

00:58:41.160 --> 00:58:49.580
Yeah, and you don't necessarily want to do like a super hard break and say, we're going to rename the package like request3 or boto3 or something like that.

00:58:49.580 --> 00:58:50.460
Yeah, definitely not.

00:58:50.460 --> 00:58:51.060
Don't want that.

00:58:51.060 --> 00:58:51.960
All right.

00:58:51.960 --> 00:58:54.020
So maybe final thoughts.

00:58:54.540 --> 00:58:59.320
We have Python 3.7 coming out already just recently came out in a beta.

00:58:59.320 --> 00:59:02.360
And we can start playing with things like data classes, right?

00:59:02.360 --> 00:59:09.200
I recommend people go and download the beta and give Python 3.7 a go, try some of your existing code.

00:59:09.200 --> 00:59:14.740
The reason it's in beta is because they're asking people in the community to actually go and test it and give them feedback.

00:59:14.740 --> 00:59:21.720
So, you know, if there's something that's not working quite right, or you've somehow managed to find some bugs, yeah, then definitely raise those.

00:59:21.860 --> 00:59:31.560
And now that Python's source code, CPython's source code anyway, is on GitHub, then you can, you know, you can work with pull requests and issues through a nice friendly interface.

00:59:31.560 --> 00:59:36.320
Yeah, I think the move to GitHub is really actually, we're going to see benefits from that for a long time.

00:59:36.320 --> 00:59:37.160
I think that's a big deal.

00:59:37.160 --> 00:59:38.560
Yeah, definitely a huge deal.

00:59:38.560 --> 00:59:39.040
Yeah, definitely.

00:59:39.040 --> 00:59:41.140
All right, Anthony, this is really, really insightful.

00:59:41.140 --> 00:59:44.440
Thank you for sharing everything you put together around this course.

00:59:44.440 --> 00:59:46.440
People should check it out over on Plutalsight, right?

00:59:46.440 --> 00:59:47.800
Will you give us a link to it?

00:59:47.800 --> 00:59:48.420
Yeah, definitely.

00:59:48.420 --> 00:59:48.940
I'll give you that.

00:59:48.940 --> 00:59:49.380
All right, through that in the show notes.

00:59:49.380 --> 00:59:49.800
Awesome.

00:59:50.640 --> 00:59:53.480
You're going to be at PyCon in the US, 2018, Cleveland?

00:59:53.480 --> 00:59:57.700
Yes, I guess I'm flying all the way from Oz over to Cleveland.

00:59:57.700 --> 01:00:01.720
Halfway around the world to come hang out with all our Python friends, right?

01:00:01.720 --> 01:00:04.640
Yeah, it's going to be like a remake of planes, trains, and automobiles.

01:00:04.640 --> 01:00:06.200
So I'm quite looking forward to it.

01:00:06.200 --> 01:00:06.740
Yeah, very cool.

01:00:06.740 --> 01:00:08.440
I'm looking forward to that whole event as well.

01:00:08.440 --> 01:00:09.640
And I don't think it's sold out yet.

01:00:09.640 --> 01:00:12.080
So people should definitely get their tickets because it's going to.

01:00:12.080 --> 01:00:13.000
All right.

01:00:13.700 --> 01:00:16.580
Let's do the final two questions before you get out of here.

01:00:16.580 --> 01:00:18.800
If you're going to write some Python code, what editor do you use?

01:00:18.800 --> 01:00:20.980
At the moment, I'm using Visual Studio Code.

01:00:20.980 --> 01:00:25.840
It's kind of lightweight enough not to take 10 minutes when I want to open it.

01:00:25.840 --> 01:00:28.080
And it's got some more advanced features.

01:00:28.080 --> 01:00:31.120
Like it highlights all of my mistakes for me, which is nice.

01:00:31.120 --> 01:00:31.760
Yeah, very cool.

01:00:31.760 --> 01:00:35.980
It's definitely pretty cool with the Python plugin, which is now part of like the official

01:00:35.980 --> 01:00:37.040
team, right?

01:00:37.040 --> 01:00:38.520
They hired Don, who was working on it.

01:00:38.520 --> 01:00:39.640
That's also a cool story.

01:00:39.640 --> 01:00:42.260
So we talked a lot about PyPI packages.

01:00:42.260 --> 01:00:46.680
I've got one that is notable you want to recommend to people that they maybe don't know.

01:00:46.680 --> 01:00:47.340
Yeah, sure.

01:00:47.340 --> 01:00:51.020
I'm going to cheat since this is one of the ones that I maintain.

01:00:51.020 --> 01:00:58.440
But if you're using documentation written in restructured text and you would love to use

01:00:58.440 --> 01:01:02.960
Read the Docs, but the documentation that you want to publish is proprietary to your company

01:01:02.960 --> 01:01:08.340
and you have Atlassian Confluence as a wiki, then install Sphinx-Confluence

01:01:08.340 --> 01:01:08.740
Builder.

01:01:08.740 --> 01:01:12.520
And it will actually publish your Sphinx output to Confluence for you.

01:01:12.520 --> 01:01:12.880
Wow.

01:01:12.880 --> 01:01:13.160
Okay.

01:01:13.160 --> 01:01:13.740
Very cool.

01:01:13.740 --> 01:01:14.680
All right.

01:01:14.680 --> 01:01:15.820
So final call to action.

01:01:15.820 --> 01:01:18.580
People out there with Python 2 code, what do they do?

01:01:18.580 --> 01:01:23.100
So people out there with Python 2 code, start to run some linters, start to run some of the

01:01:23.100 --> 01:01:25.360
automated tools to see what you need to change first.

01:01:25.360 --> 01:01:29.020
Then introduce your test suite to Python 3.

01:01:29.020 --> 01:01:32.520
See what things you need to change and just keep working through the application.

01:01:32.520 --> 01:01:34.120
Follow the best practices.

01:01:34.120 --> 01:01:36.340
And of course, watch the course.

01:01:36.340 --> 01:01:37.380
Right on.

01:01:37.380 --> 01:01:38.160
All right.

01:01:38.160 --> 01:01:39.380
Well, it's really great to catch up with you.

01:01:39.380 --> 01:01:40.720
And thanks for being on the show again.

01:01:40.720 --> 01:01:41.080
Yeah.

01:01:41.080 --> 01:01:41.900
Thanks very much, Michael.

01:01:41.900 --> 01:01:42.140
Yeah.

01:01:42.140 --> 01:01:42.300
Bye.

01:01:42.300 --> 01:01:47.060
This has been another episode of Talk Python To Me.

01:01:47.060 --> 01:01:49.240
Today's guest was Anthony Shaw.

01:01:49.240 --> 01:01:52.340
And this episode has been brought to you by Linode and Rollbar.

01:01:52.340 --> 01:01:56.200
Linode is bulletproof hosting for whatever you're building with Python.

01:01:56.200 --> 01:02:00.540
Get four months free at talkpython.fm/Linode.

01:02:00.540 --> 01:02:02.340
That's L-I-N-O-D-E.

01:02:02.340 --> 01:02:05.340
Rollbar takes the pain out of errors.

01:02:05.840 --> 01:02:10.300
They give you the context and insight you need to quickly locate and fix errors that might

01:02:10.300 --> 01:02:13.060
have gone unnoticed until your users complain, of course.

01:02:13.060 --> 01:02:19.320
As Talk Python To Me listeners, track a ridiculous number of errors for free at rollbar.com slash

01:02:19.320 --> 01:02:20.120
talkpythontome.

01:02:20.120 --> 01:02:22.880
Are you or a colleague trying to learn Python?

01:02:22.880 --> 01:02:27.480
Have you tried books and videos that just left you bored by covering topics point by point?

01:02:27.480 --> 01:02:33.480
Well, check out my online course, Python Jumpstart by building 10 apps at talkpython.fm/course

01:02:33.480 --> 01:02:36.100
to experience a more engaging way to learn Python.

01:02:36.640 --> 01:02:40.880
And if you're looking for something a little more advanced, try my Write Pythonic code course

01:02:40.880 --> 01:02:43.420
at talkpython.fm/pythonic.

01:02:43.420 --> 01:02:46.140
Be sure to subscribe to the show.

01:02:46.140 --> 01:02:48.340
Open your favorite podcatcher and search for Python.

01:02:48.340 --> 01:02:49.580
We should be right at the top.

01:02:49.580 --> 01:02:55.700
You can also find the iTunes feed at /itunes, Google Play feed at /play, and direct

01:02:55.700 --> 01:02:58.880
RSS feed at /rss on talkpython.fm.

01:02:59.280 --> 01:03:00.780
This is your host, Michael Kennedy.

01:03:00.780 --> 01:03:02.140
Thanks so much for listening.

01:03:02.140 --> 01:03:03.200
I really appreciate it.

01:03:03.200 --> 01:03:05.160
Now get out there and write some Python code.

01:03:05.160 --> 01:03:25.660
I'll see you next time.

01:03:25.660 --> 01:03:55.640
Thank you.