#57: Python performance from the inside-out at Intel Transcript
00:00 When you think about the performance of your software, there's nothing more low-level and fundamental than how your code executes on the CPU itself.
00:08 Many of us study and try to understand how to maximize performance at this low level, but few are in a position to define what actually happens there.
00:16 That's why I'm thrilled to share the work that Intel, the largest PC chip manufacturer, is doing specifically to make Python faster and to make their chips execute Python code even better.
00:28 This week, you'll meet David Stewart, Engineering Manager in the Intel Data Center Software Technology Group at Intel.
00:34 We'll discuss a wide variety of work Intel is doing in open source and Python.
00:38 This is Talk Python to Me, Episode 57, recorded May 2, 2016.
00:56 This music I constructed line by line, just like when I'm coding another software design.
01:01 In both cases, it's about design patterns.
01:04 Anyone can get the job done.
01:06 It's the execution that matters.
01:07 I have many interests.
01:09 Welcome to Talk Python to Me, a weekly podcast on Python, the language, the libraries, the ecosystem, and the personalities.
01:16 This is your host, Michael Kennedy.
01:18 Follow me on Twitter, where I'm @mkennedy.
01:20 Keep up with the show and listen to past episodes at talkpython.fm.
01:24 And follow the show on Twitter via at Talk Python.
01:26 This episode has been brought to you by SnapCI and now Metis.
01:31 Thank them for supporting the show on Twitter via Snap underscore CI and this is Metis.
01:37 That's right.
01:38 Metis and their data science education team have joined the show to keep the episodes coming.
01:42 Be sure to find them on Twitter and tell them thank you.
01:44 David, welcome to the show.
01:47 Thank you very much, Michael.
01:48 It's great to talk with you.
01:50 Yeah, I'm really excited to have you on the show today.
01:53 And I'm looking forward to looking inside what you guys are doing at Intel with Python.
01:59 It's going to be a lot of fun.
02:00 Yeah, it's really been exciting to me.
02:03 I think people have been really responding when I hear that Intel is doubling down on Python just because of its power and popularity.
02:12 So I think it's exciting to be a part of that for sure.
02:16 Yeah, it's really exciting to be part of Python in general.
02:19 I mean, it's really surprising and a pleasant way to me to continue to see this language and ecosystem that's 25 years old gaining momentum and gaining speed.
02:30 Kind of feel like it would have done whatever it did in the first five or ten years, but that's not the case.
02:35 It's great.
02:35 So nice to see Intel.
02:36 Your listeners probably, you're probably already aware of the fact that the top, the majority of the top CF schools in the U.S. are teaching Python as the first introductory language, right?
02:48 That's often a surprise when I talk, I was just talking to a professor at computer science last week and he was like, really?
02:55 They just didn't know about that.
02:57 So, yeah, it's really taken hold.
02:58 Yeah, I think that's great.
03:00 And, you know, to be honest, it makes me a little bit jealous.
03:02 Yes, I, my very first CS, not my first programming class, but my first CS class that was for programmers was in Scheme and Lisp.
03:12 And I would have definitely preferred.
03:14 Oh, my gosh.
03:14 Yeah, I would have definitely preferred to get Python in there.
03:17 But, you know, you get what you get.
03:18 Yeah, no.
03:19 I know what you're talking about.
03:21 Well, you know, if you were talking about Pity here, my first was with Fortran.
03:25 And, in fact, I had to teach Fortran programming to, you know, to engineering students and the like as a part of a graduate teaching assistantship back in the day.
03:33 So, yeah, I didn't necessarily do a lot of programming in Fortran myself, but it was one of those things where you sort of had to cut your teeth on something.
03:42 And there's still people using Fortran these days, but it's probably more than Lisp.
03:46 But, yeah, it's got to be waning.
03:49 I hope so.
03:50 I think we share some common history.
03:52 Like, I started out in, yeah, I started out in an engineering field, not in computer science.
03:59 And when I got there, they said, you have to have a programming class.
04:03 I said, great, can I take C++?
04:04 No, you have to take Fortran first.
04:06 This is the most important language you will ever learn in your life.
04:09 Then you can go take those other less meaningful ones.
04:11 So I took that and then I went to CS.
04:15 I said, can I please take C++?
04:16 No, you have to take Scheme.
04:18 I'm like, why can't I do a real programming?
04:20 Anyway, I'm going to get some hate mail for saying that, but that's okay.
04:24 I'm sure you will.
04:25 No, I really, I do like C++, but I wish I had Python in the beginning.
04:32 That would have inspired me more, even though I was already inspired to do programming.
04:36 Yeah, and I think that it's been particularly interesting.
04:40 There's so many open source projects that are, you know, kind of advanced by some corporate entity or something like that.
04:47 Python is a phenomenal project.
04:50 It's sort of like the Linux kernel itself, right?
04:53 There are plenty of companies involved with Linux, but it's really, you know, with Linus Torvalds being the, you know, the lead,
05:01 it makes a final decision on things.
05:03 And Python, that's one of the strengths, I think, is the kind of approach that Guido uses.
05:09 Guido Van Ropsen, who is the benevolent dictator for Life for Python.
05:11 I think he's really created, you know, just kind of an ethos and a culture around this project that I think is super unique.
05:20 And, in fact, it's a great, I hold up as a great example for people who aspire to, you know, establishing open source projects.
05:29 I mean, hey, this is one of the ways that you talk about, yeah, 25 years of, you know, experience.
05:34 And Guido's, you know, I think a phenomenal leader for that movement.
05:38 Yeah, for sure.
05:39 Yeah, he's doing a great job.
05:40 And I think there's a lot of positive examples of the Python community being a very positive place for open source.
05:47 And that's great.
05:48 So, before we get into the story about what you guys are doing with Python, maybe you could just sort of tell me how you got into programming in Python.
05:55 You said you took Fortran in college.
05:58 So, how did you get to there?
05:59 Yeah, I mean, you know, when I got my bachelor's and master's in computer science, I mean, C was really the most interesting language.
06:10 to me at the time because I was doing a lot in operating systems.
06:12 And so, I've been probably over the years of my career probably more interested in, you know, operating systems in general instead of languages per se.
06:20 And since C is a great language, you know, initially when I worked with it was Unix and then, you know, Linux, obviously.
06:27 I think that's one of the things that it was just when I think in terms of programming, it's still hard for me not to relate back to C.
06:34 But, you know, the Python work came as a result of a couple of things I got involved with back in 2010.
06:41 What happened was I was asked to start up a new project here at Intel to work on embedded Linux.
06:48 And it was kind of an amazing thing.
06:51 The core of this project had this something called BitBake.
06:56 And if you're not familiar with it, what BitBake does is it's a desktop application that will build a complete Linux distribution.
07:02 Now, this is actually quite interesting because what it does is it builds the compiler first, right?
07:08 And it has to bootstrap several different paths of the compiler building to get a functional compiler, right?
07:13 And then it goes and a Linux distribution could be made up of, like, literally a thousand different projects all over the Internet.
07:22 Individual projects with their own source control systems, their own repositories.
07:26 And this system will actually download the sources from all these different projects, goes and patches the source.
07:33 Of course, if you have an open source system, you want to be able to, you know, do patching of those things, right?
07:38 Then it will configure them, build them, build a package repository, build a running Linux image, and then an SDK.
07:46 And the amazing thing is that it does all of this in an hour, like one hour on a desktop computer.
07:51 And you're going, wait, hold on.
07:53 What magic is this thing doing?
07:56 And I started talking to the architect of this thing, and it's a massive Python program.
08:01 So BitBake is actually completely written in Python.
08:04 I was like, you know, I got to confess.
08:07 I was like, holy cow, this is a scripting language?
08:09 And I heard about, you know, things like, you know, you're able to serialize and deserialize objects.
08:15 And it's like, hang on, this is really powerful stuff.
08:17 So I worked on that project for about five years.
08:21 That was a project called the Yocto Project.
08:23 So anybody who knows that, that was something I was involved with.
08:26 And then, you know, things were going, you know, pretty well.
08:29 And I kind of get, you know, a little bit, you know, sedate, I guess, when things are going too well.
08:34 So I always like to have a challenge.
08:36 And at that point, Intel was really interested in, as I said, doubling down on Python,
08:41 because it's primarily the core of a lot of key things that Intel is invested in in other areas.
08:48 And so, and besides its overall popularity as a, you know, as a programming language.
08:53 So that was when I really, you know, got personally very invested in what was going on in Python.
08:59 So I, you know, started a group and we have engineers that are at the core of working on that.
09:04 So that was where I think I got really a lot of Python religion, obviously, at that point,
09:10 was not only seeing it as a user, incredibly powerful, but also be able to now, I think,
09:16 affect it positively on a community basis.
09:19 We're going to dig into some of the cool stuff you guys are doing on a community basis as well.
09:24 But you sort of started out with the philosophy of it's in Intel's interest to sort of understand
09:32 modern cloud computing and the languages that drive that and make that stuff go really well.
09:38 That's correct.
09:38 I mean, if you think about, you know, Intel, obviously, we try and sell processors in the data center space.
09:46 I think we've had some success there.
09:47 One of the things that if you look at overall, how are people programming those data center processors?
09:53 Well, there's obviously a lot of C++, C#, and Java, no question.
09:58 But as we kind of analyzed things, we said, well, gee, there are 7 million PHP programmers out there.
10:06 There's like 9 million JavaScript programmers, a considerable number on, you know, on data center computers.
10:12 And then Python is huge in that respect, too.
10:14 So if you look at the top languages that people are using our processors to run, right, it sort of says, well,
10:22 how do I, you know, I'm all about customer choice in that respect.
10:25 I'd like to make sure whatever customers are using that they get the best, you know,
10:29 possible experience with our processors.
10:30 And so it just feels like whether, you know, it's OpenStack work or some of the things that people are using Python for in,
10:39 like high-performance computing or big data analytics, machine learning.
10:46 There's just a lot of applications of Python.
10:49 And it just seems like for Intel as a company to not be kind of investing that and making it great on our processors,
10:59 it's just not a good idea.
11:01 So one of the things I love about this is that we're being agile with respect to how we invest in the software that's running on the data center on our processors.
11:10 And we're trying to, again, if our customers are using this stuff, we want to make sure that we have the best sort of, you know,
11:16 the best sort of experience for them.
11:17 So hopefully they'll come along and buy our processors on an ongoing basis.
11:22 Yeah, I hope so.
11:24 It sounds great.
11:25 So can you give me some idea of what, like, what are you studying in Python?
11:31 Are you looking at sort of the CPython implementation and how it's working and understanding how that's running on your chips and things like this?
11:42 Or what's the story there?
11:43 Yeah, yeah, yeah.
11:45 But our philosophy is, you know, it's kind of what I call the core software strategy, which basically says this.
11:52 It's like you can go off and, you know, work on speeding up some, you know, customer's application or some other, you know, kind of system.
12:00 You know, that's great.
12:02 But then, you know, at the end of the day, you know, you've optimized that one piece of software.
12:07 You haven't really, you know, optimized anybody else's Python code, right?
12:11 So, for example, let's take Swift.
12:15 It's an open source object storage subsystem that's part of OpenStack, and it's implemented in Python.
12:22 And if you look at that and you go, well, I could go work on the Swift source code, right, the Python source code to make it faster.
12:29 But then it'd be better if I could actually speed up Python instead, that core software.
12:34 And so, in theory, if I speed up Swift, I'll also speed up anything else that's using that accelerated Python, right?
12:42 So, the way we achieve that core software strategy is we've developed some really amazing abilities.
12:49 I mean, the engineers on the team have just done an amazing job to analyze exactly where the processor is spending its time as it's running, you know, major Python code, right?
12:59 So, we're actually able to go at a very deep level, and we had some really very interesting discoveries that we've made at the microarchitectural level.
13:08 And, you know, I can tell you more about that.
13:10 It's kind of interesting.
13:13 I mean, clearly, we see some things that are some challenges from a microarchitecture standpoint.
13:19 So, we go, well, if we're going to improve the performance, we need to do what we can to address those things with the current products and then give feedback to our chip architects to say, hey, man, you need to change the way that the chip is designed to run these languages better.
13:34 So, this is a – it's like a – not an either-or for us.
13:38 It's like a both-and.
13:39 Understanding the architecture at a deep level, we can go in and really speed up Python, and we can also go tell the chip architects to design better chips.
13:49 So – and along those lines, yeah, our philosophy is totally upstreaming all of these things.
13:56 And so, as much as possible, I sort of go – I mean, philosophically, I like to put the cookies on the low shelf so everyone can get at them.
14:04 As much as possible, I want to make sure that we don't just do some amazing sort of things and that we upstream them.
14:09 But then, you know, I want to make sure people know about them, right, so they can take advantage of them too.
14:14 And I want to make it – I want to be a contributor to the community so that they can, you know, take advantage of the hard work that we're doing as well.
14:20 Yeah, that's really excellent.
14:22 So, it manifests sort of in two ways.
14:24 You can improve CPython, and we're just getting started talking about the possibilities there, right?
14:31 And then, how dramatic of a change can you make at the chip level?
14:35 I mean, I think it's really cool that you're, like, thinking about, okay, well, here's our current architecture and the way things are working.
14:43 But when you run Python code, this could be better.
14:46 Would the chip actually detect what type of code it's running?
14:50 Like, does it work at that level?
14:52 Or are they just – you're like, these types of operations could be better?
14:56 Well, actually, our chip architects have a long practice of looking at how current software runs on current chips.
15:04 And we actually have some great tools to be able to analyze that at an instruction level that says, okay, you're running – oh, I don't know whether it's a database or whether it's, you know, Python or something else.
15:16 And you're able to analyze, you know, okay, here's the series of instructions that get run.
15:19 And then they can play a lot of interesting, you know, conceptual games with that and say, well, what if we added this instruction, a new instruction to optimize this thing?
15:29 Or what if we change the way the cache works to, you know, run this stuff better?
15:34 Or, you know, and, you know, we're actually – by having us involved with them, we're actually able to come up with some really interesting ideas and say, well, hey, maybe we could have this kind of, you know, acceleration
15:48 or this kind of idea about how to run these languages better, right?
15:53 So our hope is that, yeah, you know, and so the current chips are really designed from just years of running code, you know, of existing software against it.
16:05 And so, you know, some of it's making up for bad software practices, bad programming practice.
16:09 And it's like, oh, if people were to code this way, maybe if we organized the chip this way, it would run, you know, sort of poor code faster or something like that.
16:16 Not to say that anyone's code is crap.
16:18 I'm not saying that.
16:18 But there's a lot of crappy code that's out there, right?
16:20 Yeah.
16:21 I'm sure none of your listeners have bad code.
16:23 I'm sure they don't.
16:24 But if they listen to other podcasts, they may.
16:26 I'm just teasing.
16:28 I'm sure they're fixing other people's bad code, not their own.
16:30 But, I mean, you know, that's a key, you know, observation in terms of the chip.
16:35 Now, I couldn't necessarily go into a lot of ideas on the podcast, but we're working on a bunch of really intriguing ideas, in my opinion, about ways that we can really run this stuff much, much better.
16:46 I think it's cool to know that you guys out there are actually thinking about, you know, specifically studying these different runtimes like Python and Node.js and PHP and so on.
16:59 And understanding sort of how they're working and not working on your hardware and then adjusting for that.
17:07 When would the first sort of fruits of these labors show up?
17:11 You said in 2010 you got started with the Linux embedded stuff and then later you got into Python.
17:18 So how far down the road does this stuff land?
17:21 Yeah.
17:22 And by the way, the one thing I was going to add to your observation about the work that we do, we've actually been doing this work with Java now for, I think, 15 years.
17:31 And we have this nice little chart that we compare from generation to generation how much the chip speeds it up, but how much our software, you know, JVM's, you know, changes have accelerated in each of those generations.
17:43 And so we're just playing back the same playbook, basically, that we've been using for years with Java.
17:48 What I would say in terms of the fruits of our labor, actually, when we began the group, we had our first performance patches available to the community, I think, within about like three, four months or so.
18:00 And we're on the order of like a 10% performance boost.
18:03 And, you know, you think, well, 10%, that can't be great.
18:06 But, in fact, here's my experience with data center level code, whether it's, you know, massive database benchmarks like TPCC or this, you know, running Swift or some of these other big, big, big customer workloads on a data center.
18:21 If you can boost something, you know, 5% to 10%, you are just like, you are like golden.
18:26 That's like awesome.
18:27 It's very unusual to find a real customer benchmark that gets, you know, some multiple of, you know, many X speed.
18:35 It just doesn't happen, right?
18:37 And so, realistically, if you can improve the throughput by a few percent, usually people are pretty happy about that.
18:43 So, we came up with our first patches.
18:45 Yeah, I'm pretty sure it was about three, four months after we started analyzing things and look at some low-hanging fruit of places where we could pull in some stuff pretty quickly and get the experience with the community, you know, talking with Guido, talking with other community developers to try and really, you know, hone our, you know, our efforts there.
19:04 And the first time we kind of took a look at PyPy, that's when we said, well, shoot, maybe we could get a much, much higher, you know, return with PyPy.
19:14 Maybe we could get a whole lot more than just a few percent here and there.
19:17 So, that became extraordinarily interesting.
19:20 Thank you.
19:20 Thank you.
19:50 Thank you.
20:20 Yeah, that's awesome.
20:25 And that's kind of what I was hinting at, like we're just getting started.
20:27 So, the majority of people who run Python code, they run it on CPython.
20:32 That's the default thing you get if you go to python.org and you download it.
20:37 But there are many other runtimes or implementations.
20:41 The most popular one around performance is probably one called PyPy, P-Y-P-Y.
20:47 But we also have PyStin coming out of Dropbox and Guido's group there.
20:53 We've got Psython, Jython, IronPython.
20:56 There's a lot of options.
20:58 And so, you turn to the probably one of the more established high-performance alternatives to CPython, PyPy, right?
21:08 That's correct.
21:08 That's correct.
21:08 And, you know, for us, it's like I didn't want to be married to anything until we really got a chance to see, you know, kind of what was going on in the landscape.
21:16 I mean, we looked at all of these, frankly.
21:19 And they all have their pros and cons.
21:22 I mean, things like Psython, et cetera, you know, basically creating C code.
21:26 That's a nice model for performance.
21:29 The challenge is that kind of takes away some of the, you know, development speed you get from an actual interpreted language.
21:35 You know, and PyPy for it, and Pidgin and Piston have interesting qualities.
21:42 I don't, you know, the thing that I like about PyPy is as follows.
21:46 I mean, for one thing, it's been around 10 years, right?
21:49 So, it's got, as far as I can tell, the broadest compatibility of any of these efforts.
21:56 It's focused on both Python 2 and Python 3.
21:58 I think that, you know, it really hasn't received a ton of, you know, broad sort of help from folks, you know, like these performance guys that I work with here at Intel.
22:11 I mean, I think they've done an amazing job, absolutely amazing, stunning.
22:17 I am incredibly impressed.
22:19 We had a great sprint with them, you know, basically in March of this year, and face-to-face effort with their core developers, and it was just really very, very effective.
22:33 And then there was another release of PyPy that came out as a result of the sprint.
22:38 You know, so I think we're just getting started working with it.
22:42 The results have been nothing short of stunning.
22:45 I mean, this is what really impressed me because, like I said, we're working with Swift because it's a part of OpenStack.
22:51 We're doing a lot of stuff with OpenStack at Intel.
22:53 You know, Swift is the part of OpenStack that seems to spend a lot of its time in Python.
22:58 As we analyzed it, 70% of the cycles, 70%, 80% of the cycles when you're running Swift is actually in the Python interpreter itself.
23:06 So, it's like, oh, there's a lot of opportunity there.
23:11 Then we look at, we split that down further, and it's like really being, you know, like 30% of those cycles are just in the main interpreter loop of CPython.
23:19 So, we said, well, let's try PyPy, as you said, P-Y-P-Y, and it's incredible.
23:24 We got 111% throughput improvement.
23:26 Now, pause for a second and say, okay, I was looking at a few percent here and there, and here I've gotten more than double the throughput of using PyPy, right?
23:36 It's like compared to CPython, amazing.
23:38 And then it's like an 87% response time improvement.
23:42 And it's like if something's like a software system, I mean, throughput's great because, right, you can scale up more and more users and you get, you know, great throughput.
23:50 But the response time is what people really respond to, right?
23:54 I mean, this is like it means they're accessing files, whether you're like, for example, Wikipedia, all of the, you know, sort of images that people look at in Wikipedia are all managed by Swift, right?
24:05 So, you can look at something like that, and that just brings up your Wikipedia pages faster.
24:09 And I've talked to various customers who are using Swift for their, you know, object storage system.
24:14 And, yeah, this is a huge deal when you get that kind of improvement.
24:18 So, it's like, well, with such amazing, you know, speedups, why wouldn't we try to, you know, see if we can, you know, provide a little, you know, love to the project to see if we can, you know, really do it, you know, continue to make this maybe even the default of how people use Python.
24:34 Okay, yeah, that's really amazing.
24:36 Do you know if you can run all of OpenStack on PyPy?
24:39 Yeah, we're actually in the process of getting that together.
24:44 We have, let's see, so far Swift, Keystone, Nova, and Neutron ported.
24:51 And I would, you know, I would say trying to get like a proof of concept where we have all the core services running in PyPy.
25:00 And I'd like us to really be able to do that maybe before the next OpenStack Summit in the fall in Barcelona.
25:05 So, I'm hoping that we'll be able to show that off.
25:08 Keystone, I said, is another interesting service that's written in Python.
25:12 Keystone, if you're not familiar with it, is the user authentication part of OpenStack.
25:18 So, essentially, every OpenStack service has to go through Keystone to see if you're authorized to do the things you're saying you want to do, right?
25:27 And so, it's a really, you know, centralized part of the project.
25:30 And we were able to speed it up by 37% using PyPy.
25:33 So, that's, again, pretty darn amazing.
25:36 So, I think I'd like to see the entire, yeah, I'd like to see PyPy as the default.
25:42 In fact, years ago, the OpenStack gate had a requirement that everything had to work with PyPy.
25:48 And a year or so ago, that got dropped.
25:50 And I'd like to bring that back.
25:51 And, if possible, try and drive, you know, something that would have the whole, you know, community basically making use of PyPy within OpenStack.
25:58 Yeah.
25:59 Yeah, that's really cool.
26:00 If you're running something like OpenStack, you're running on a lot of machines.
26:04 And efficiency is going to make a big difference in that type of system, right?
26:10 Absolutely.
26:10 And, you know, if you think about this, in some respect, it's not kind of radical.
26:14 Every other interpreted language, whether it's Java, JavaScript, PHP, you name it, Lua, they have all gone the direction of a JIT in order to improve performance.
26:29 And here, let me tell you how this works, right?
26:33 If you've got two, let's say you're adding two integers together, right, in Python.
26:36 Very simple operation.
26:37 You know, if you're running native code on our processors, right, there's one instruction that adds two integers together, right?
26:44 In some cases, it can be even less than one instruction with opcode fusing some features that we have.
26:50 With Python, CPython in particular, we did a measurement and showed that on average, it takes 76 instructions to add two integers together, right?
27:00 So automatically, you can see a stunning difference.
27:03 And, you know, we've got great processors.
27:05 They run instructions very fast.
27:07 But you're running one instruction versus 76 is definitely going to be faster.
27:11 That's a very hard thing to compensate for.
27:14 Yeah.
27:15 So as a result, you know, the processor spends a lot of it because the code footprint is just huge on something like Python.
27:22 And so, you know, you spend a lot of the processor spends a lot of its time.
27:25 We talk about it being cycles stalled in the front end.
27:29 And what that basically means is the first two stages.
27:33 If you think about it, I'm sorry to, let me just mention this briefly.
27:37 I won't geek out on this too much.
27:39 But it's like if you think about the five-stage pipeline for a processor, a modern processor, you get best performance if you can get all the stages of the pipeline running in parallel, right?
27:49 The problem is our processors are spending like half of their time twiddling their thumbs waiting for new instructions to get fetched and decoded.
27:56 So if we can, you know, make a huge difference to that, and a lot of it, frankly, comes down to simply the size of the instructions, the size of the code footprint that we're trying to process, right?
28:09 And so JIT directly addresses that because what a JIT is going to do is it's going to look at the hot interpreted, you know, code and generate native code, right?
28:20 So instead of that 76 instruction for an ad, if it's a hot ad, it will basically run in one instruction because it will be running native code instead.
28:28 So this is not a huge secret.
28:31 It's not magical.
28:32 It's something that is a really, you know, well-worn technique to make this happen.
28:37 And I'm telling you, every other language has gone this direction.
28:41 And as I've, you know, as Guido and I have talked about this as well, he's like, you know, he's, you know, supportive.
28:46 He's like, he says, I don't have a dog in the fight.
28:48 You know, he's basically, he's more, I think, interested in the advances of the language as opposed to these performance things.
28:54 Usually he's not thinking in terms of the performance side.
28:57 But I'm, you know, I'm thinking about the performance side.
28:59 I said, hey, could we really help the overall project by, you know, having a, you know, a great interpreter for Python that really runs really super fast?
29:11 By the way, they're able to achieve greater performance by several other things in terms of the garbage collection that they're able to do.
29:18 In various ways, they have much lower memory footprint as well.
29:22 And so this is something that several of these techniques they're able to use to get really stunning performance.
29:27 Yeah, the PyPy guys are doing a great job.
29:30 And they definitely do some interesting stuff.
29:32 I talked to them on episode 21.
29:34 One of the things that I thought was interesting is the way that they don't immediately JIT everything, but they kind of wait until they find a hotspot.
29:43 And then they go and JIT that.
29:45 Right.
29:45 Right.
29:46 That's, again, a well-worn technique that has a ton of fruit in the Java space, as an example.
29:52 Right.
29:52 They even called their JIT a hotspot is what it's called.
29:55 Right.
29:55 So this is a common technique, and I think it works really well.
29:59 Yeah.
30:00 Yeah.
30:02 I think the challenges that you run into is languages like Python, they're so flexible and you can change the types so much that it's not as easy as JIT in something like Java.
30:13 Right.
30:15 Oh, yeah.
30:15 And particularly when, you know, because it's one of the really attractive things about the language is how, you know, flexible it can be in terms of typing.
30:23 You don't have to spend a lot of time, you know, coming up with the types and documenting all those things.
30:27 And so, you know, coding that is Pythonic, it really has that, you know, kind of power and flexibility.
30:33 Unfortunately, that power and flexibility also comes with a cost and typically, you know, means that you have to spend a lot of code trying to figure out, now, what is the type of this thing?
30:41 And did it change since the last time I looked at it?
30:44 Right.
30:44 So that's a strong consideration when trying to run this stuff fast.
30:48 Yeah.
30:49 I think there's a lot of attention being given to speed and Python lately.
30:54 I've noticed a big uptick in projects and people focused on trying to make Python faster, be that through PyPy or the Microsoft Pigeon or I just spoke to the Euphoria guys about distributed, compiled Python.
31:10 There's a whole bunch of really interesting things.
31:13 And they're not all mutually exclusive.
31:16 I think they're going to come together and some pretty amazing stuff is going to come out of it.
31:20 Yeah, I agree with you.
31:21 I think that one of the strengths of the Python community is the fact that you do have a lot of freedom for innovation.
31:28 And, you know, the uptick in performance projects, I've noticed that as well.
31:33 I think part of that's just because it has become so well adopted that I think if they don't solve the performance thing, you know, I think people will sort of get frustrated and go on to other things.
31:45 You know, for example, in the high-performance computing space, you have people who are scientists and, you know, people studying data and things of that sort as opposed to being programmers, right?
31:55 And they love Python because they can, you know, implement their code using Python.
32:00 But then they go, well, this is performing slowly, so, you know, let me recode it in something else, right?
32:05 And I think our vision would be that people don't have to, you know, change their language to get better performance.
32:13 I'd love it to be a no-compromises, you know, experience with Python, right?
32:18 And then we can bring to bear the best of our, you know, microarchitectural analysis tools, software optimization tools, and, frankly, analysis tools as well.
32:26 We've got a terrific visual profiler called VTune that lets you, you know, pinpoint the exact area in your Python code that is, you know, causing your problems, performance problems.
32:38 We also have, you know, a set of Python libraries that are, particularly if you're using NumPy, SciPy, Pandas, SciKit Learn, a variety of those packages.
32:50 We have an accelerated version of those as well in a Python product.
32:54 So there's a lot of, you know, but our common vision, you know, goes end-to-end in that sort of thing, whether it's the upstream stuff that we're doing or the Python product.
33:04 We just want to make sure that the experience people have with Python is no compromises relative to performance.
33:11 Yeah, that's great.
33:11 And I want to talk about the NumPy, the data science stuff in a moment.
33:30 This portion of Talk Python to me is brought to you by Metis, offering data science training in New York City, Chicago, San Francisco, and online.
33:37 Led by a deep team of senior data scientists, Metis delivers immersive boot camps, corporate workshops, online training, and part-time professional development courses in data visualization, machine learning, big data, and other data science skills.
33:51 Their full-time boot camp is the only accredited data science boot camp available and includes extensive career support.
33:57 Metis maintains a busy event schedule, so be sure to check them out on Meetup and keep in touch via at ThisIsMetis on Twitter and learn more about them on the web at ThisIsMetis.com.
34:08 So you said that you had worked with the PyPy guys and you actually got them together to do a sprint with some people on your team.
34:23 Is that what the story is?
34:24 Yeah, that's right.
34:25 Yeah, it turns out that all of the core PyPy developers are in Europe, except for Maciek, who's in South Africa.
34:35 And I have a staff of developers working on Python that are in Europe.
34:40 And so we brought everybody together.
34:42 It turns out it's in Bucharest, Romania.
34:46 And so we got everybody together physically in the same room.
34:51 There were actually a couple of folks from the academic world, a couple of universities that wanted to come in and, you know, work.
34:57 Once I got wind of this happening, they said, man, we want to get involved in this, too.
35:02 So, you know, it was a very effective sprint and great to get that teamwork going.
35:08 I feel like we now have a great understanding not only of, you know, PyPy, but also the microarchitectural tools that we have to analyze performance and things of that sort.
35:16 So we can now turn this thing, I think, into a really powerful kind of collaboration with the PyPy project.
35:23 Oh, that's great.
35:24 And I think that's really great that you were able to help get everyone together because that's one of the real big challenges of these open source projects that are somewhat large is just physically getting the people together.
35:37 So there might be people on projects that have never met before, right?
35:42 Absolutely.
35:43 And, you know, I tell you, I've been working in specifically open source projects since like the mid-2000s.
35:49 And it's true that, you know, open source would never have been, you know, wildly successful without the Internet.
35:55 But trust me, when you can get people physically face-to-face, you can get human-to-human contact in a physical, you know, same physical space, you eliminate a lot of inefficiencies and barriers and you get a lot of progress very quickly.
36:09 And so as far as I'm concerned, that's a worthwhile sort of investment to get the smart people together in a room and work on real problems and produce code as a result.
36:18 Yeah, even just getting to know people for a long weekend.
36:21 That can last like a whole year's worth of goodwill from that, right?
36:24 Absolutely.
36:25 No question.
36:26 Yeah, great.
36:27 So that's a huge commitment back to the PyPy group and all of that is open source, of course.
36:33 Have you done anything where you've taken some of this work and research and gotten it back into CPython?
36:40 Yeah, we're trying to be not an either or for me, at least.
36:47 It's like there's still, you know, adoption of PyPy is currently not very high.
36:53 A lot more people are running code on CPython.
36:55 So it's not like an either or for me.
36:57 It's more like a both and.
36:59 And the same thing goes for Python 2 versus Python 3.
37:03 I mean, even though Python 3 is the place where Guido and the rest of the community, the development community, would like everybody just to be doing things on Python 3, reality is core services and OpenStack, a lot of other code is still in Python 2.
37:18 So that's our commitment to the community is as we do performance work, we're doing it for both Python 2 and Python 3.
37:25 And we're really trying to pull off a thing where we can do CPython and PyPy as well.
37:30 So, you know, we have, I think there's some chemical issues that we can, like I said, give a little bit of love to in terms of making it much easier to use PyPy in many of these circumstances.
37:41 So there's some engineering work along those lines as well that we're trying to invest in also to eliminate if there are any deployment issues or any issues with, you know, with the garbage collector or something like that.
37:52 So we're really trying to, you know, really eliminate those.
37:55 And there is a class of code that is challenged partly because PyPy is so fast.
38:03 You know, this is the amazing thing to me is that we've encountered a couple of places where, you know, people have coded a timeout in Python.
38:10 And, you know, the code wasn't exactly correct.
38:13 And so when you switch to PyPy, which is multiple times faster, right, suddenly these timeouts fail because, you know, the code, you know, isn't exactly correct.
38:23 Now, it's not necessarily something I can fix in PyPy.
38:27 I mean, I'd love to figure that out somehow, but I'm not sure that's going to be possible.
38:31 But, you know, we've seen this now in a couple of very interesting instances where people, you know, write their code.
38:36 And, you know, it's, you know, suddenly when you speed it up a lot, it breaks because it made some assumptions about how slow it should be running.
38:45 It got to the weight too quickly, huh?
38:47 It's an odd problem to have.
38:50 But, you know, that is, frankly, when you're going to do a drastic improvement in performance, you may find an issue like that that pops up.
38:57 So in any case, that's something we've seen a little bit of.
39:00 But hopefully people can fix their code because I think that's probably a good thing, right?
39:05 Yeah, absolutely.
39:06 That's obviously the fix.
39:07 Like, hey, let's just not wait so long.
39:09 Anyway, so it sounds to me like you have this multi-pronged approach.
39:13 You're trying to make the processors faster by understanding how they run Python code, trying to make CPython faster, trying to make PyPy faster.
39:20 Another one of the prongs on making things faster or addressing this Python performance story is around something you call the Intel distribution for Python.
39:31 And you kind of hinted at that before, right?
39:33 What's the story there?
39:34 Yeah, and this is, again, we have sort of two-pronged strategy here.
39:40 One strategy says let's do, you know, everything upstream in open source.
39:45 And the other is, is there some way we can kind of pinpoint some of the key pinch points that people have?
39:52 And then it may be that we have some code, you know, internally that will accelerate those things.
39:57 So, you know, in particular when you're running, you know, math functions, this is something that in the evolution of people's Python code, right?
40:05 They write these things initially in Python, and then it's like, wow, this is kind of a performance pinch point.
40:12 Is there something we can kind of do to improve it?
40:14 So a lot of times people recode these things in C, and it's like, okay, and NumPy is a great example of that, right?
40:20 So what we've actually been able to do is say, gee, we have this library here.
40:25 It's called MKL.
40:26 It's called our math kernel libraries, right?
40:28 Where some incredibly smart people have gone at the deepest possible level to have math functions and execute them just as fast as possible, right?
40:38 So they're highly tuned, very, you know, very, very bespoke sort of code, right?
40:45 So, you know, what they've done is done an amazing job hooking up MKL with NumPy, SciPy, and these other popular libraries.
40:55 And so that's code that they actually call the Intel Python distribution or Intel distribution.
41:00 I might have the name wrong, but it's actually in beta now as we're talking, and so that should be something,
41:07 some get in on the beta program, or if they're listening to this at a later time, they may, in fact, be already released.
41:12 So that's another, I think, great alternative, particularly if you're using those scientific libraries to basically make use of that
41:18 and see, you know, if that's going to improve things.
41:21 In some cases, I think they found some code that's, you know, again, it can multithread some of these operations,
41:27 whereas with, you know, regular Python, it's more or less single-threaded.
41:31 And so with multithreading, depending on how many cores you can throw at it,
41:35 they've seen up to 100x on some math functions that they've been able to speed up.
41:39 So that's pretty stunning.
41:40 There's also some data analytics functions that are part of this, and so I think that's a really intriguing option as well.
41:47 Yeah, that does sound intriguing.
41:49 And it sounds to me like what you're actually targeting there is you're targeting the C extensions
41:56 or the C foundations of some of these heavily C-sped-up libraries.
42:03 Yeah, that's exactly right.
42:04 It's a strategy that, again, is like we don't want to have people make a choice between, well, do I want to stick with Python
42:12 or do I want good performance, right?
42:14 We want to be able to make it so that it's a no-compromise situation.
42:17 So, frankly, this is another alternative that lets us, you know, target, you know, some users that really want super high performance
42:24 in this mathematical area.
42:25 And high-performance computing is another area that we've invested in pretty heavily here.
42:30 So this is a way to, you know, get directly into that space and help them out.
42:33 Yeah, and I guess you guys probably understand better than anybody how to exactly line up the math to work on your processors, right?
42:40 Yeah, we really have some amazing tools and analytics to do.
42:46 Yeah, when you sit next to the guys designing the chips, it's usually, you know, much easier to be able to squeeze out that last little bit of performance in any possible case.
42:56 Yeah, yeah.
42:57 Sounds great.
42:58 I'll be sure to link to that in the show notes.
43:00 So one of the things that we talked about last week is you were actually at the OpenStack conference, right?
43:06 Maybe you can give us a...
43:07 That's correct.
43:07 OpenStack Summit.
43:08 Yeah.
43:09 Awesome.
43:09 Give us a rundown on what happened there.
43:10 Yeah.
43:11 This is really exciting because the OpenStack Summit is, you know, every six months.
43:15 This one was in Austin, Texas.
43:18 And what was interesting about this is that OpenStack as a project is going through a lot of really interesting evolution as cloud computing obviously takes over the kind of the fastest growing segment of the data center kind of area, right?
43:32 So software-defined infrastructure is incredibly important.
43:35 And something like 70% of OpenStack is actually written in Python.
43:40 So one of the things that we did was jointly with the project technical leader, or PTL, of Swift, he and I have been collaborating together on PyPy,
43:51 and we were able to jointly present our performance results and showing, hey, here's the effect performance-wise of using PyPy with Swift.
44:01 And again, I think I mentioned the statistics earlier.
44:03 It's like more than double the throughput, right?
44:05 We have data which shows that we can, you know, get 111% throughput improvement and 87% response time improvement.
44:14 You know, that's stuff that we've seen.
44:16 The PTL, John Dickinson, works for a company called SwiftStack.
44:20 And in their product sort of environment, they have a Swift-based product family.
44:25 And so, yeah, they were able to see, you know, similar results.
44:28 And so jointly we were able to present this.
44:31 And people were very excited about this.
44:33 The response was extremely positive, partly because of the opportunity to see the collaboration go on.
44:39 You know, anytime you go in with open source and you can collaborate and talk about how we're working together to make people's lives better,
44:47 I think it's all good, right?
44:48 So that was – but what was funny is I've got to tell you a story.
44:54 When I landed in Austin, so I get to my hotel that I'm staying at for this conference,
44:59 and I'm walking over to the conference venue to get my conference badge.
45:02 The first person I see on the street, I'm sad to say I didn't recognize him,
45:08 but he recognized me and he said, oh, it's Mr. PyPy.
45:13 Somehow, then I found out, okay, why have you recognized me?
45:18 Anyway, so it's – I think the word's kind of getting out that this is something that, you know,
45:24 we see some real value in.
45:25 We'd like to try and help people out through advancing it a little bit.
45:28 Yeah.
45:28 Yeah, you must be being effective if you're getting recognized on the street as Mr. PyPy.
45:34 That's awesome.
45:35 My usual comment is a good reputation takes a long time to build, but a bad reputation is instantaneous.
45:41 So, you know, I think we'll – you know, hopefully we can keep it to be a good reputation as opposed to a bad one.
45:46 Yeah, absolutely.
45:47 Okay, well, that sounds really interesting.
45:50 That's great you were able to share your results with everyone.
45:54 Another thing that you said that was pretty interesting, sort of completely unrelated to this,
45:58 but you've been talking about how the Python community is a really welcoming community.
46:05 And I can tell you from doing training and speaking at conferences that are not just Python conferences
46:12 but have many languages and technologies that when I do Python talks, the group of people in the room are more representative of the group of people out in society,
46:22 which is really great.
46:24 Well, I totally agree with you.
46:25 I think it's amazing to see, particularly because I've spent a number of years in other open source projects.
46:31 And so I know at sometimes a painful level how challenging it can be for some people who are not males in particular.
46:42 And this is one of the cases where I don't want to point to particular projects or something like that,
46:46 but I've been involved in a number of projects.
46:48 And it's hard sometimes to find projects that not only have great diversity but inclusion as well.
46:55 And so one of the things that's been very impressive to me, the first Python conference I went to actually was EuroPython.
47:02 And Guido made a – actually it was before Guido came on as a keynote speaker.
47:08 The very first keynote of the project, of the conference, was a conference given by the founders of Django Girls.
47:15 And it's two female engineers.
47:20 And Django Girls was set up by then as a nonprofit to basically have women teach other women how to code in Python, right?
47:28 And it was a great keynote.
47:30 I mean they actually did a bunch of it as like a fable, you know, kind of talking about –
47:36 essentially talking about how challenging it is for, you know, women to really be considered part of the community, right, of any sort of community.
47:44 And so Django Girls, what's amazing is they started up with pretty much nothing.
47:48 And after the first year, they had had 100 workshops all over the world.
47:54 Again, women teaching women how to program in Python, right?
47:57 And this was incredibly impressive.
47:59 I mean Guido actually in his keynote, he – you know, I think he's part of the reason.
48:06 It's like this.
48:07 I think he's very committed.
48:09 He was wearing a PyLadies T-shirt in the conference.
48:13 PyLadies is another, you know, part of the Python community that works really well to help encourage women be involved in Python.
48:19 He – in his keynote, he said – most of it was Q&A from the audience.
48:23 And he said, let's alternate a man and a woman asking questions, right?
48:26 So he made it super clear that that's a priority for him.
48:30 And I asked him about it.
48:31 Is that, you know, something that he – it's important to him?
48:34 He said, yeah, absolutely.
48:35 And so he's making a difference.
48:36 He's putting his own, you know, kind of stamp behind that.
48:39 And I might – on a personal basis, I had – you know, my daughter is – one of my daughters is 24 and she's still trying to find kind of her career direction.
48:50 And she said, Dad, you know, if I wanted to learn programming, you know, what would you suggest?
48:54 And I said, well, I think you should try Python.
48:58 And I said, in fact, you know, you ought to check out this Django Girls thing to see if there's a workshop.
49:03 Well, it turned out, you know, we live in the Portland area.
49:05 There was a Django Girls workshop in another, like, six weeks.
49:09 And I said, oh, look, you could get involved in that.
49:11 And she submitted an application, explained, you know, what she'd be using her knowledge for and how – you know, why she wanted to learn it.
49:20 They accepted her in the workshop.
49:21 And, you know, she was a little nervous.
49:24 She said, gee, I'm not sure I'm smart enough to actually, you know, do well with this, right?
49:28 And I said, no, no, no.
49:29 You know, and I understand sometimes people can feel that way.
49:32 Typically, they've been made to feel not smart, which, you know, I try and do everything I can with my daughters to make them feel like they're – you know, they're both brilliant.
49:42 They have no worry about that.
49:45 But sometimes the environment is one that wants to, you know, make people who are, you know, women not necessarily feel very smart.
49:51 And so I said, you know, I encouraged her.
49:53 She went through the workshop.
49:55 She was very successful with it.
49:56 And I'm hoping, you know, fingers crossed that it would be something she'd be able to, you know, make a living doing if she digs it.
50:03 So, yeah, I love the fact that Django Girls is – you know, has this available.
50:08 And it's, again, part of – I think a core part of the Python community.
50:12 I think it's one of the things that makes it a very attractive community is how important it is for the community to – for women to help, you know, other women.
50:20 And for the entire community to have better inclusion.
50:23 Now, is it perfect?
50:25 No.
50:25 I think there's some challenges still.
50:28 I interviewed one of the – a fellow of the Python Foundation.
50:33 Her name is Terry Oda.
50:36 And I talked to Terry and I said – I asked her how it is from her perspective as a woman.
50:42 I mean, it's one thing for me as a male to say, gee, this seems really good.
50:46 Well, how about a perspective of a woman?
50:47 And she's part of the, you know, the Intel – or, sorry, the Python Foundation.
50:52 She's also an Intel employee.
50:53 And she said, yeah, you know, she said that it's a good environment.
50:57 It has the – you know, it could do better in terms of having more core developers who are women.
51:03 And so there's more, you know, work that can be done to, you know, get more of that inclusion and diversity within, you know, sort of the core development community.
51:11 But it is a great project for its diversity and inclusion.
51:15 Yeah, that's cool.
51:16 I think the Python community is more welcoming than most.
51:21 And I think this is just, you know, more evidence of it.
51:23 So that's cool.
51:24 Hopefully we can inspire some more people to go out and check out some of these projects that are out there and freely available.
51:30 Very cool.
51:31 I agree, yeah.
51:32 So getting near the end of the show, let me ask you a couple of questions that I always ask my guests while I still have you.
51:40 If you're going to write some Python code, what editor do you open up?
51:44 Oh, yeah.
51:45 I'm an old VI guy.
51:47 So, you know, with my roots back in Linux and back in the old days, or Unix, you know, before there was a Linux.
51:53 So, yes, when I edit, I use VI.
51:55 Okay, excellent.
51:56 Yeah, that's definitely a popular one.
51:58 And of all the PyPI packages, you can include Intel ones if you want, there's a ton of them.
52:04 And nobody could possibly know about them all.
52:07 There's, you know, close to 80,000, maybe over.
52:10 I haven't looked in a while.
52:10 What ones do you like that maybe you'd recommend people check out that are not necessarily the most popular?
52:17 I'd ask the question, though, what's my favorite interpreter?
52:20 My favorite interpreter for Python these days is definitely PyPy.
52:24 Yeah, okay.
52:25 That's great.
52:26 It's a very meta answer, right?
52:28 Like, it's PyPy that runs all the packages.
52:31 I'm not a politician.
52:32 I don't even get paid to be a politician.
52:34 But sometimes I know how to, you know, reframe the question.
52:37 Yeah, absolutely.
52:38 Okay, so do you have a final call to action for listeners, things they should do or check out?
52:46 Yeah, I would absolutely have people take another look at PyPy.
52:51 You probably have in the past.
52:53 In particular, we would love to have people check out, if you're using, you know, OpenStack or Swift,
52:59 we would love to get people to, you know, check it out with the latest version of PyPy
53:03 and really try to give us some feedback in the Swift community or in the Python dev community.
53:10 We monitor Python dev.
53:12 And if you have some observations about PyPy that would be helpful, saying, you know, for example,
53:16 if you have some Python code and if you either find you have some, you know, challenges with it,
53:22 we would love to hear that because one of the things we'd love to do is I've got engineers I can, you know,
53:26 try and ask to, you know, basically say, hey, let's go solve these.
53:29 I've given the call to action to our engineers to say, try and, you know, resolve these,
53:34 any sort of compatibility issues or other deployment issues and let's make, you know,
53:39 try and make this a great experience for people.
53:40 So if you can help us out in terms of, you know, get your hands on PyPy, try it against your Python code
53:45 and give us feedback in terms of what might be missing or could work better, we would love to hear that
53:50 and we'd love to dialogue with you on it.
53:52 Okay.
53:53 So absolutely try your code on PyPy and send these guys some feedback.
53:57 That's great.
53:58 All right, David, it's been a super interesting conversation.
54:02 I'm really excited to see the work that you're doing appear in the Intel chips going forward
54:08 and in both the runtimes in CPython and PyPy.
54:12 Thank you, Michael.
54:14 I love talking about this stuff.
54:16 I'm incredibly passionate about it and I'd love to, you know, I'd see our love for technology be able to be made use for a lot of people.
54:25 So I'm hopeful that people will get excited about this stuff as a result too.
54:29 Yeah, absolutely.
54:30 Thanks for being on the show.
54:31 Talk to you later.
54:31 Thank you.
54:34 Snap.CI is effective in deep training for data scientists.
54:48 Metis is effective in deep training for data scientists.
55:00 With immersive boot camps, in-person and online courses, you can learn the core data science skills you need to take your career to the next level.
55:08 Are you or a colleague trying to learn Python?
55:11 Have you tried books and videos that just left you bored by covering topics point by point?
55:16 Well, check out my online Python course called Python Jumpstart by Building 10 Apps at
55:21 talkpython.fm/course to experience a more engaging way to learn Python.
55:26 You can find the links from this episode at talkpython.fm/episodes slash show slash 57.
55:31 And be sure to subscribe to the show.
55:33 Open your favorite podcatcher and search for Python.
55:36 We should be right at the top.
55:37 You can also find the iTunes feed at /itunes, Google Play feed at /play,
55:43 and direct RSS feed at /rss on talkpython.fm.
55:48 And there's been a few requests for the Ogthera format, a more open format than MP3.
55:53 And I added that link as well in the footer of the website.
55:56 Our theme music is Developers, Developers, Developers by Corey Smith, who goes by Smix.
56:01 You can hear the entire song on talkpython.fm/music.
56:04 This is your host, Michael Kennedy.
56:07 Thank you so much for listening.
56:08 Smix takes us out of here.
56:11 Stating with my voice, there's no norm that I can feel within.
56:15 Haven't been sleeping, I've been using lots of rest.
56:17 I'll pass the mic back to who rocked his best.
56:20 First developers.
56:22 First developers.
56:22 First developers.
56:25 First developers.
56:26 First developers.
56:27 First developers.
56:28 First developers.
56:29 First developers.
56:31 First of all, first of all, first of all, first of all.
56:33 Thank you.