#57: Python performance from the inside-out at Intel Transcript
00:00 When you think about the performance of your software, there is nothing more low level and fundamental than how your code executes on the CPU itself. Many of us study and try to understand how to maximize performance at this low level. But few are in a position to define what happens at this level.
00:00 That's why I'm thrilled to share the work that Intel, the largest PC chip manufacture, is doing specifically to make Python faster and to make their chips execute Python even better.
00:00 This week you'll meet David Stewart, manager in the Intel Data Center Software Technology group at Intel. We'll discuss the wide variety of work Intel is doing in open source and Python.
00:00 This is Talk Python To Me, episode 57, recorded May 2nd, 2016.
00:00 [music intro]
00:00 Welcome to Talk Python To Me, a weekly podcast on Python- the language, the libraries, the ecosystem and the personalities.
00:00 This is your host, Michael Kennedy, follow me on Twitter where I am at @mkennedy, keep up with the show and listen to past episodes at talkpython.fm and follow the show on Twitter via @talkpython.
00:00 This episode is brought to you by Snap CI and Metis. Thank them for supporting the show on Twitter via @snap_ci and @thisismetis. That's right, Metis and their data science education team have joined the show to keep the episodes coming, be sure to find them on Twitter and tell them thank you.
01:46 Michael: David, welcome to the show.
01:47 David: Thank you very much Michael, it's great to talk to you.
01:50 Michael: Yeah, I am really excited to have you on the show today, and I am looking forward to looking inside what you guys are doing at Intel, with Python, it's going to be a lot of fun.
02:01 David: Yeah, I think people have been really responding when I hear that Intel is doubling down on Python, just because of its power and popularity, so I think it's exciting to be a part of that.
02:15 Michael: Yeah, it's really exciting to be a part of Python in general, I mean, it's really surprising and a pleasant way to me is to continue to see this language and ecosystem that's 25 years old gaining momentum and gaining speed, I kind of feel like it would have done whatever I did in the first five or ten years but that's not the case, it's great, so it's nice to see Intel-
02:37 David: Your listeners are probably already aware of the fact that the top majority of the top cs schools in the US are teaching Python as the first introductory language, right, that's often a surprise when I talk- I was just talking to a professor of computer science last week and he was like really- he just didn't know about that, so yeah, it's really taking home.
02:59 Michael: Yeah, I think that's great and to be honest it makes me a little bit jealous, I my very first cs, not my first programming class, but my first cs class that was for programmers was in Scheme and I would have definitely prefer to have Python in there but, yeah, you get what you get.
03:18 David: Yeah, I know what you are talking about, if you were talking about pity here my first was Fortran, and in fact I had to teach Fortran programming to engineering students as a part of a graduate teaching assessment back in the day so, yeah. I did a lot of programming in Fortran myself, but it was one of those things where you sort of had to cut your teeth on something. People still use Fortran these days.
03:48 Michael: Yeah, I think we share some common history, like I started out in engineering field, not in computer science and when I got there they said you have to have a programming class, great, can I take C++? No, you have to take Fortran first, this is the most important language you will ever learn, in your life, and then you can go take those other lessons, meaningful ones. So I took that and then I went to CS and said can I please take C++, no, you have to take Scheme, what, why can't I do real programming...? I am going to get some hate mail from saying that, but that's ok. [laugh]
04:25 David: I'm sure you will.
04:27 Michael: Now, I really like C++ but I wish I had Python in the beginning, that would have inspired me more even though I was already inspired to do programming.
04:38 David: And I think that it's been particularly interesting, so many projects that are kind of advanced by some corporate entity or something like that, Python is a phenomenal project because sort of like the Linux kernel itself it's either plenty of companies involved with Linux but it's really with Linus Torvalds being the lead and makes a final decision, you know, in Python that's one of the strings I think is the kind of approach the Guido uses, Guido Van Rossum I think he has really created just kind of an ethos and a culture around this project that I think is super unique, and in fact it's a great example for people who aspire to establish projects, I mean hey, this is one of the ways you talk about 25 years of experience and Guido I think is a phenomenal leader for that movement, for sure.
05:39 Michael: Yeah, he is doing a great job and I think there is a lot of positive examples of the Python community being a very positive place for open source, that's great. So before we get into the story about what you guys are doing Python, maybe you could just sort of tell me how you got into programming in Python, you said you took Fortran in college, so how did you get to there?
06:00 David: Yeah, I mean, when I got my bachelors and masters in computer science C was really the most interesting language to me at the time because I was doing a lot of operating systems and so I've been probably over the years more interested in operating systems in general instead of languages per se and since C is a great language, initially when I worked with it was Unix and then Linux obviously, I think that's one of the things that- when I think in terms of programming it's still hard for me not to relate back to C, but the Python work came as a result of a couple of things I got involved with back in 2010. What happened was I was asked to start up a new project here at Intel to work on embedded Linux and it was kind of an amazing thing, the core of this project had this something called bitbake and if you are not familiar with what bitbake does, it's a desktop application that will build the complete Linux distribution.
07:03 Now, this is actually quite interesting because what it does is it's it builds the compiler first, right, and it has to bootstrap several different passes of compile building to get a functional compiler, right, and then it goes, a Linux distribution could be made up of like literally a thousand different projects all over the internet, individual projects with their own source control system, their own repositories, and this system will actually download the sources from all these different projects, goes and patches the source of course if you have an open source system you want to be able to do patching of those things, right, then it will configure them, build them, build a package repository, build the Linux image and then an stk and an amazing thing is that it does all this in an hour, like one hour on a desktop computer and you are going wait, hold on, what magic is this thing doing, and I start talking at the architect of this thing and it's a massive Python program.
08:02 So bitbake is actually completely written in Python, I was like I got to confess, I was like holly cow this is a scripting language, and I heard about things like you know, you were able to serialize and deserialize objects and it's like, hang on, this is really powerful stuff. So I worked on that project for about five years, it was the project called the Yoctoproject, so anybody who knows that was something I was involved with. And then, things were going pretty well and I kind of get a little bit sedated, I get some things going too well, I always like to have a challenge, and at that point Intel was really interested in as I said doubling down on Python because it's primarily the core of a lot of key things at Intel is invested in another areas and so, and besides its overall popularity as a programming language; so that was when I really got personally very interested in what was going on in Python, so I started to group, and we had engineers that are at the core of working on that. That was where I think I got really a lot of Python religion obviously that point was not only seeing it as a user incredibly powerful but also be able to now, I think affect a positive way on a community basis.
09:20 Michael: We are going to dig into some of the cool stuff you guys are doing on the community basis as well, but you sort of started out with the philosophy of it's in Intel's interest to sort of understand modern cloud computing and the languages that derive that and make that stuff go really well.
09:38 David: That's correct, I mean, if you think about, Intel obviously we turn self processors in the data center space I think we had some success there, one of the things that if you look at overall how are people programming those processors, well, there is obviously a lot of C++, C Sharp and Java no question, but as we kind of analyze things we said well, there are seven million PHP programmers out there, there is like 9 million Javascript programmers considerable number on data center computers and then Python is huge in that respect too, so if you look at the top languages that people are using our processors to run, it sort of says well, I am all about customer choice in that respect, I want to make sure whatever customers are using that they get the best possible experience of our processors.
10:31 And so it just feels like whether it's open stack work or some of the things that people are using Python for in like high performance computing and big data analytics, machine learning- there is just a lot of applications in Python and just it seems like for Intel as a company to not be kind of investing that and making it great on our processors, it's not a good idea; so, what I love about this is that we are being agile with respect to how we invest in the software that is running on the data center on our processors and we are trying again, if our customers are using this stuff we want to make sure that we had the best sort of experience, hopefully they will come along and buy our processors on an ongoing basis.
11:23 Michael: Yeah, I hope so, it sounds great. So can you give me some idea of what are you studying in Python, are you looking at sort of the CPython implementation and how it's working, and understanding how it's running on your chips, and things like this, what's the story there?
11:43 David: Our philosophy is it's kind of what I call the core software strategy which basically says that it's like you can go often work on speeding up some customers application or some other kind of system, that's great, but then at the end of the day, you've optimized that one piece of software, you haven't really you know, optimized anybody else's Python code, right, so for example, let's take swift, it's open source object storage, it's part of open stack and it's implemented in Python, and if you look at that and go well, I can go work on the swift source code, right, the Python source code to make it faster but then it would be better if I could actually speed up Python instead that core software, and so in theory if I speed up swift, it's usually that accelerated Python right, so the way we achieve that core software strategy is we can develop some really amazing abilities, the engineers on the team I have, they had just an amazing job to analyze exactly where the processor is spending its time as it's running major python code, right.
13:00 So, we are actually able to go at a very deep level, we have some really very interesting discoveries that we made at the micro architectural level. And I can tell you more about that, it's kind of interesting but clearly we see some things that are some challenges from the micro architectural stand point so we go, well, if we are going to improve the performance, we need to do, we can address those things with the current products and then give feedback to our chip architects to say hey man, you need to change the way the chip is designed to run these languages better, so this is like both end, understanding the architecture of the deep level, we can go in and really speed up Python and we can also go tell the chip architects to design better chips. So our philosophy is totally upstreaming all of these things and so as much as possible I sort of go, I like to put the cookies on the low shelf, so everyone can get them, as much as possible I want to make sure we don't just do some amazing sort of things and that we upstream them, but then I want to make sure people know about them, like so they can take advantage of them too, I don't want to make it, I want to be a contributor to the community, so that they can take advantage of the hard work that we are doing as well.
14:21 Michael: Yeah, that's really excellent, so it manifests sort of in two ways, you can improve CPython and we are just getting started talking about the possibilities there, right, and then how dramatic of a change can you make at the chip level? I mean, I think it's really cool that you are like thinking about ok, well, here is our current architecture and the way things are working but when you run Python code, this could be better. Will the chip actually detect what type of code is running, like does it work at that level or there are just these types of operations could be better?
14:56 David: Well, actually, our chip architects have a long practice of looking at how current software runs on current chips. And they actually have some great tools to be able to analyze that at a construction level that says ok, you are running whether it's a database or whether it's you know Python or something else, we are able to analyze here is the series of instructions to get run, and they I can play a lot of interesting conceptual games with that and say what if we added this instruction and new instruction to optimize this thing; or what if we change the way that cash works to run this stuff better, and we are actually by having us involved with them, we are actually able to come up with some really interesting ideas and say well hey, maybe we could have this kind of acceleration or this kind of idea about how to run these languages better, right, so our hope is that yeah- so the current chips are really designed from just years of running code of existing software against it, and so some of it is making up for bad software practicies, bad programming praciticies, like oh, if people code this way maybe if we code it this way it would run poor code faster, or something like that, not to say that anyone's code is crap, I am not saying that, but there is a lot of crappy code that's out there, right.
16:20 Michael: Yeah.
16:21 David: I am sure none of your listeners have bad code.
16:23 Michael: I am sure they don't. But if they listen to other podcast they may.
16:28 David: I am sure that are fixing other people's bad code, but that's a key, you know in terms of the chip, now I couldn't necessarily go into a lot of ideas on the podcast, but we are working on a bunch of really intriguing ideas in my opinion about ways that we can really run this stuff much, much better. Yeah.
16:47 Michael: I think it's cool to know that you guys out there are actually thinking about specifically studying these different runtimes like Python, NoJs and PHP and so on and understanding sort of how they are working and not working on your hardware and then adjusting for that. When would the first sort of fruits of this labor show up, you said in 2010 you got started with the Linux embedded stuff and then later you got into Python sort of how far down the road does this stuff land?
17:21 David: Yeah, and by the way, the one thing I was going to add to your observation about the work that we do we have actually been doing this work with Java and now for I think 15 years, and we have this nice little chart that we compare from generation to generation how much of the chip speed is that but also how much our software JVM changes and have accelerated in these generations, so we are just playing back the same playbook basically that we have been using for years with Java. What I would say in terms of the fruits of our labour, we are actually when we began the group we had our first performance patches available to the community, I think it was about three or four months or so, and we were on the order of like 10% performance boost and you think well like 10%, that can't be great but in fact with- here is my experience with data center level code, whether it's you know, mass of data base benchmarks, or this running swift or some of these other big customer workloads on the data center, if you could boost something 5 to 10% you are like golden, that's like awesome, it's very unusual to find a real customer benchmark to get some 18:32 it just doesn't happen right. And so, realistically if you can improve the throughput by a few percent, usually people are pretty happy about that. So we came up with our first patches, I am pretty sure it was about 3 or 4 months after we started analyzing things and look at some of the low hanging fruit of places we can pull in some stuff pretty quickly and that the experience with the community talking with Guido, talking with other community developers to try and really hone our efforts there. And the first time we kind of took a look at PyPy that's when we said well shoot, maybe we can get much much higher return with PyPy maybe we could get a whole lot more than just a few percent here and there so that became extraordinarily interesting.
17:21 [music]
17:21 This episode is brought to you by SnapCI, the only hosted, cloud based continuous integration and delivery solution that offers the multi stage pipelines as a built in feature.
17:21 SnapCI is built to follow best practices like automated build, testing before integration, and it provides high visibility into who is doing what. Just connect Snap to your GitHub repo and it automatically builds the first pipeline for you, it's simple enough for those who are new to continuous integration, yet powerful enough to run dozens of parallel pipelines. More reliable and frequent releases, that's Snap.
17:21 For a free, no obligation 30 day trial just go to snap.ci/talkpython.
17:21 [music]
20:24 Michael: Yeah, that's awesome and that's kind of what I was hinting at, like we are just getting started. So the majority of people who run Python code, they run it on CPython, that's the default thing you get if you go to Python.org and you download it. But there are many other runtimes or implementations, the most popular one around performance is probably one called PyPy but we also have Pyston coming out of Dropbox and GUido's group there, we've got Scithon, Jython, Iron Python there is a lot of options and so you turn to the probably one of the more established high performance alternatives to CPython PyPy, right?
21:08 David: That's correct. And, you know, for us it's like I do not want to be married to anything, so we really got a chance to see kind of what was going on the landscape, I mean we looked at all of these frankly, and they all have their pros and cons I mean things like Cython, etc, you know basically creating C code, that's a nice model for performance and the challenges, that kind of takes away some of the development speed you get from an actual interpreted language you know, and PyPy, and Pyjion and Pyston have interesting qualities, the thing that I like about PyPy is it follows, for one thing it's been around ten years, right, so it's got as far as I can help the broadest compatibility of any of these efforts, it's focused on both, Python 2 and Python 3. I think that it really hasn't received a ton of broad sort of help from folks like these performance guys that I work with here at Intel, I mean I think they have done an amazing job absolutely amazing, stunning, I am incredibly impressed, we had a great sprint with them, basically in March of this year, and face to face effort with their core developers and it was just really very very effective.
22:34 And then there was another release of PyPy that came out as a result of the sprint, you know, so I think we are just getting started working with it, the results have been nothing short of stunning, I mean this is what really oppressed me because it's like I said we are working with swift because it's a part of open stack, we don't do a lot of stuff with open stack at Intel, you know swift is a part of open stack that seems to spend a lot of its time in Python, as we analyzed it 70%, 80% of the cycles when you are running swift is actually in the Python interpreter itself, so it's like oh, there is a lot of opportunity there, then we look at, we split that down further and it's like really being like 30% of the cycles are just in the main interpreter loop of CPython so we said well, try PyPy and it's incredible, we got a 111% throughput improvement.
23:26 Now, pause for a second and say ok, I was looking at a few percent here and there and here I've got more than double the throughput of using PyPy right, and it's like compared to CPython amazing, and then it's like an 87% response time improvement and then it's like, if some things like a software system and throughput is great because you can scale at more and more users and you get great throughput but the response time is what people really respond to, right, I mean this is like it means they are accessing files whether you are like for example Wikipedia, all of the sort of images that people look at in Wikipedia are all managed by swift, right, so you can look at something like that and that just brings up your Wikipedia pages faster. I talked to various customers who are using swift for their object storage system and yeah, this is a huge deal when you get that kind of improvement, so it's like while we are such amazing speed why wouldn't we try to see if we can provide a little love to the project to see if we can really continue to make this maybe even a default of how use Python.
24:34 Michael: Ok, yeah, that's really amazing. Do you know if you can run all of open stack on PyPy?
24:40 David: Yeah, we are actually in a process of getting that together, we have let's see so far Swift,Keystone, Nova and Neutron ported and I would say trying to get like a proof of concept where we have all the core services running in PyPy, I'd like to really be able to do that maybe even for the next open stack summit in the fall in Barselona. I am hoping that we will be able to share that off, Keystone I said is another interesting service that's written in Python; keystone if you are not familiar with it is the user authentication part of open stack so essentially every open stack service has to go through keystone, to see if you are authorized to do the things you are saying you want to do right, and so it's a really centralized part of the project, we were able to speed it up by 37% using PyPy, so that's again pretty darn amazing, so I think I'd like to see the entire, I'd like to see PyPy as the default; in fact, years ago the open stack gate had requirements that everything had to work with PyPy and year or so ago that got dropped and I'd like to see bring that back and if possible try and derive something that would have the whole community base we are making use of PyPy within open stack.
26:00 Michael: Yeah. that's really cool, if you are running something like open stack, you run it on a lot of machines, and efficiency is going to make a big difference in that type of system, right?
26:10 David: Absolutely. And, you know, if you think about this, in some respect it's not kind or radical, every other interpretive language whether it's java, Javascript, PHP, you name it, they have all gone the direction of JIT, in order to improve performance, and let me tell you how this works, right, if you got two, let's say you are adding two integers together, right, in Python, very simple operation, you know if you are running native code on our processors, right, there is one instruction that adds to integers together, right, in some cases it can be even less than one instruction with the opt codes using some features we have; with Python, CPython in particular, we get a measurement and show that on average it takes 76 instructions to add to integers together, so automatically you can see a stunning difference in- we got great processors, they run instructions very fast, but one instruction versus 76 is definitely going to be faster.
27:13 Michael: That's a very hard thing to compensate for.
27:15 David: Yeah. So as a result, you know the processor spent a lot of because the code footprint is just huge on something like Python and so you spend a lot of the- the processor spend a lot of its time and says we talk about it being cycle 27:28 front end and what that basically means is the first two stages, if you think about it- I am sorry and let me just mention this briefly I won't geek on this too much but it's like if you think about the five stage pipeline for a processor modern processor, you get best performance if you can get all the stages the pipeline running in parallel, right, the problem is our processors are spending like half of their time twiddling their thumbs waiting for new instructions to get fetched and decoded, so if we can make a huge difference to that and a lot of it frankly comes down to the simply the size of the instructions, the size of the code footprint, they were trying to process, right. And so JIT directly addresses that because what the JIT is going to do is it's going to look at the hot interpreted code and generate native code, right, so instead of that 76 instruction for an ad if it's a hot ad, it will basically run in one instruction until because it will be running native code instead.
28:28 So, this is not a huge secret, it's not magical, it's something that is a really well worn technique to make this happen and I am telling you every other language has gone this direction and as Guido and I have talked about this as well, he is supported, he is like I don't have a dog in the fight, he is more of I think interested in the advances of the language as opposed to these performance things, usually he is not thinking in terms of the performance side, but I am thinking about the performance side, hey, could we really help the overall project by having a great interpreter for Python that really runs really super fast. By the way, they are able to achieve greater performance by several other things, in terms of the garbage collection, they are able to do various ways they have much lower memory footprint as well, and so this is something several of these techniques are able to use against really stunning performance.
29:27 Michael: Yeah, the PyPy guys are doing a great job and they definitely do some interesting stuff, I talked to them on episode 21. One of the things that I thought was interesting is the way that they don't immediately JIT everything, but they kind of wait until they find hot spot and then they go and JIT that.
29:45 David: Right. That's again a well worn technique that it has a lot ton of fruit in the Java space as an example, right, they even call their JIT a hotspot, it's what it's called, right, so this is a common technique and I think that works really well.
30:00 Michael: Yeah. I think the challenges that you run into is with languages like Python there is so flexible and you can change the types so much, that it's not as easy as JIT 30:13 something like Java, right?
30:15 David: Oh yeah, and particularly when you know, because it's one of the really attractive things about the language is how flexible it can be in terms of typing, you don't have to spend a lot of time coming up with the types and documenting all those things and so coding that is pythonic it really has that kind of power and flexibility. Unfortunately, that power and flexibility also comes with the cost and typically means that you have to spend a lot of code trying to figure out, what is the type of this thing and did it change since the last time I looked at it, and so that's a strong consideration when we are trying to run this stuff fast.
30:49 Michael: Yeah, I think there is a lot of attention being given to speed in Python lately, I have noticed a big uptake in projects and people focused on trying to make Python faster, be that through PyPy, or the Microsoft Pyjion or I just spoke to the Euforia guys about distributed compiled Python, there is a whole bunch of really interesting things, and they are not all mutually exclusive, I think they are going to come together and some pretty amazing stuff is going to come out of it.
31:21 David: Yeah, I agree with you, I think that one of the strengths of the Python community is the fact that you do have a lot of freedom for innovation, and you know, the uptake in performance projects I've noticed that as well, I think part of that is just because it has become so well adopted that I think don't solve the performance thing I think people will sort of get frustrated and go for other things. You know, for example, in the high performance computing space, you have people who are scientists, and people studying data and things of that sort, as opposed to being programmers, right, and they love Python because they can implement their code using Python, but then they go well, this is performing slowly, only recode it in something else, right, and I think our vision would be the people don't have to change their language to get better performance, I'd love it to be a no compromise as experience with Python, right, and then we can bring the best of our micro architectural analyses tool, software optimization tools and frankly analyses tells us as well we've got terrific visual profiler called vtune that lets you you know pinpoint the exact area in your Python code that is causing you problems, performance problems, we also have a set of Python libraries that are particularly for using numpy, scipy, pandas, scikitlearn, the variety of those packages, we have an accelerated version of those as well, and a Python product, so there is a lot of- but our common vision goes end to end in that sort of thing whether it's the upstream stuff that we are doing, or the Python product, we just want to make sure that the experience we have with Python is no compromised as relative to performance.
33:10 Michael: Yeah, that's great, and I want to talk about the Numpy, the data science stuff in a moment.
33:10 [music]
33:10 This portion of Talk Python To Me is brought to you by Metis, offering data science training in New York city, Chicago, San Francisco and online. Led by a deep team of senior data scientists, Metis delivers immersive bootcamps, corporate workshops, online training and part time professional development courses in data visualization, machine learning, big data and other data science skills. Their full time bootcamp is the only accredited data science bootcamp available and includes extensive career support. Metis maintains a busy event schedule so be sure to check them up on meetup and keep in touch via @thisismetis on twitter and learn more about them on the web at thisismetis.com.
33:10 [music]
34:16 Michael: So you said that you had worked with the PyPy guys and you actually got them together to do a sprint with some people on your team, is that what the story goes?
34:25 David: Yeah, that's right, it turns out that all of the core PyPy developers are in Europe except for 34:34 who is in South Africa, and I have a staff of developers working on Python that are in Europe and so we brought everybody together it turns out it's in Bucharest Romania, and so we got everybody together physically in the same room, there are actually a couple of folks on the academic world, a couple of universities that wanted to come in and work, once they got this was happening they said man we want to get involved in this too, so you know, it was a very effective sprint and it was great to get that team going. I feel like we now have a great understanding not only of PyPy but also the micro architectural tools that we have analyzed performance and things of that sort. So we can now turn this thing, I think into a really powerful kind of collaboration with the PyPy project.
35:25 Michael: Oh that's great and I think that's really great that you are able to help get everyone together because that's one of the real big challenges of these open source projects that are somewhat large, is just physically getting the people together so there might be people on projects that have never met before, right?
35:43 David: Absolutely, and you know, I had been working in specifically open source projects like the mid 2000s and it's true that the open source would never have been widely successful without the internet but trust me when you have people physically face to face you can get human to human contact in the same physical space, you eliminate a lot of inefficiency and barriers and you get a lot of progress really quickly and so as far as I am concerned that's a worthwhile sort of investment to get smart people to get together in a room and work on real problems and produce code as a result.
36:20 Michael: Yeah, even just getting to know people for a long weekend, that can last like a whole year's worth of good will from that, right?
36:26 David: Absolutely, no question
36:26 Michael: Yeah, great. So, that's a huge commitment back to the PyPy group and all of that is open source of course, have you done anything where you have taken some of this work and research and gotten it back into CPython?
36:41 David: Yeah, we are trying to be, it's not an either or for me at least, it's like there is still adoption of PyPy is currently not very high, lot more people are running code on CPython so it's not like an either/ or for me, it's like a both end and same thing goes for Python 2 versus Python 3 I mean we all know Python 3 is the place where Guido and the rest of the community the development community would like everybody to just do things in Python 3, reality is core services in open stack, a lot of the code is still in Python 2, so that 37:19 into the community is we need performance work we are doing it both for Python 2 and Python 3, and we are really trying to pull off a thing where we can do CPython and PyPy as well, so you know, we have, I think there is some issues we can give a little love to in terms of making much easier to use PyPy in many of these circumstances, so there is some engineering work along those lines as well that we are trying to invest and also to eliminate if here are any deployment issues or any issues with the garbage collector, something like that, we are really trying to really eliminate those. And, there is a class of code that is challenge partly because PyPy is so fast, we've encountered a couple of places where people have coded the timeouts in Python and you know the code wasn't exactly correct and so when you switch to PyPy which is multiple times faster, right, suddenly these timeouts fail because the code isn't exactly correct. Now, it's not necessarily something I can fix in PyPy, I mean I'd love to figure that out somehow but I'm not sure that's going to be possible but you know, we've seen this now in a couple of very interesting instances where people write their code and it's suddenly when you speed it up a lot it breaks because it makes some assumptions about how it should be running.
38:46 Michael: It got to the way too quickly.
38:48 David: It's an odd problem to have, but that is frankly, when you are going to do a drastic improvement in performance you may find an issue like that that pops up. So, that's something we've seen a little bit up but hopefully people can fix your code because I think that's probably good thing, right?
39:05 Michael: Yeah, absolutely, that's obviously the fix, like hey let's just not wait for so long. Anyway, so, it sounds to me like you have this multipronged approach, you are trying to make the processors faster by understanding how they run the Python code trying to make CPython faster trying to make PyPy faster. Another one of the prongs on making things faster or addressing this Python performance story is around something you call the Intel distribution for Python and you kind of hinted at that before, right, what's the story there?
39:35 David: Yeah, and this is, again, we have sort of two prong strategy, one strategy says let's do everything upstream and open source, and the other is there some way we can kind of pinpoint some of the key pinch points that people have and then maybe that we have some code internally that will accelerate those things, so you know, in particular when you are running math functions, this is something that in the evolution of people's Python code right, they write these things initially in Python and then it's like wow this is kind of a performance pinch point, this is something we kind of improve, it's a lot of times people will code these things in C, and it's like ok, and Numpy is a great example of that right, so what we have actually been able to do is say we have this library here it's called mkl it's called math kernel library, right, where some incredible smart people have gone at the deepest possible level to have math functions and execute them just as fast as possible.
40:41 They are highly tuned, what they have done is they have done an amazing job hooking up mkl with numpy, scipy and these other popular libraries and so that's code they actually called the Intel Python distribution or Intel distribution, I forgot I might have named it wrong, but it's actually beta analyze we are talking and so that should be some getting innovative program where if they listen to this at later time they may in fact be already released, so that's another I think great alternative particularly if you are using scientific libraries to basically make use of that in C if that is going to improve things. In some cases I think they found some code that's again, it can multithread some of these operations whereas with the regular Python it's more of a single threaded and so with multithreading depending on how many cores you can throw at it they 41:37 at some math functions they have been able to speed up so that's pretty stunning. There is also some data analytics functions that are part of this and so I think that's a really intriguing option as well.
41:48 Michael: Yeah, that sounds intriguing, and it sounds to me like what you are actually targeting in there is you are targeting the C extension through the C foundations of some of these heavily C sped up libraries.
42:03 David: Yeah. That's exactly right, it's a strategy that again is like we don't really want to have to have people make a choice between well do I want to stick with Python or do I want to quit performance, right, we want to be able to make it so that it's a no compromise situations, so frankly this is another alternative that lets us target some of the users that really want super high performance in this mathematical area and high performance computing is another area that we have invested in pretty heavily here so this is the way to get directly to that space and help them out.
42:33 Michael: Yeah, and I guess you guys probably understand better than anybody how to exactly line up the math to work on your processors, right?
42:41 David: yeah, we really have some amazing tools and analytics, yeah, when you are sitting next to the guys designing the chips it's usually much easier to be able to squeeze that last little bit of performance in any possible case.
42:58 Michael: Yeah, yeah, it sounds great. I'll be sure to link to that in the show notes. So one of the things that we talked about last week is you were actually at the open stack conference, right? Maybe you can give us a rundown on what happened there?
43:10 David: Yeah, this was really exciting because the open stack summit is every six months, this one was in Austin Texas, and what was interesting about this is that open stack is the project that is going through a lot of really interesting involution as the cloud computing obviously takes over the kind of the fastest growing segment of data center kind of area, right; so software defined infrastructure is incredibly important and something like 70% of open stack is actually written in Python so one of the things that we did was jointly with the project technical leader or PTL of Swift, he and I have been collaborating together on PyPy and we were able to present our performance results and showing hey here is the effect performance using PyPy with Swift and again I think I mentioned the statistics earlier it's like more than double than of throughput, right, we have data which shows that we can 111% throughput improvement and 80% response time improvement, you know that stuff that we've seen the PTL John Dickinson works for a company called Swift Stack and in their product sort of environment they have a Swift based product family and so they are able to see similar results and so conjointly we were able to present this and people were very excited about this and the response was extremely positive partly because of the opportunity to see the collaboration going on, any time you go in with open source you can collaborate and talk about how we are working together to make people's lives better I think it's all good, right, so that was- but what was funny is I've got to tell you the story when I landed in Austin, so I get to my hotel that I am staying at this conference and I am walking over to the conference venue to get my conference badge, the first person I see on the street I am sad to say I didn't recognize him but he recognized me and said, "Oh, it's Mister PyPy" so, [laugh] somehow, then I found out ok, why have you recognized me... Anyway, so I think the word is kind of getting out that this is something that we see some real value and we would like to try and help people out through advancing a little bit.
45:29 Michael: You must be being effective if you are getting recognized on the street, here is Mister PyPy, that's awesome.
45:36 David: My usual comment is a good reputation takes a long time to build but a bad reputation is instantaneous, so you know, I think we'll, you know I hopefully will keep good reputation as opposed to a bad one.
45:47 Michael: yeah, absolutely. Ok, well that sounds really interesting, that's great you are able to share results with everyone. Another thing that you said that was pretty interesting sort of completely unrelated to this but you had been talking about how the Python community is a really welcoming community and I can tell you from doing training and speaking at conferences that are not just Python conferences but have many languages and technologies that, when I do Python talks the group of people in the room are more representative of the group that people out in society which is really great.
46:25 David: I totally agree with you, I think it's amazing to see particularly because I spent a number of years in other open source projects and so I know that sometimes a painful level how challenging it can be for some people who are not males in particular, and this is one of the cases where I don't want to point to particular project or something like that but I've been involved in number of projects and it's hard sometimes to find projects that are not only have great diversity but inclusion as well, and so one of the things that is very impressive to me, the first Python conference I went to actually was EuroPython and Guido made, actually it was before Guido came on as a keynote speaker, the very first keynote to the project of the conference was the conference given by the founders of Django girls, and it's two female engineers and Django girls was setup by them as a nonprofit to basically have women teach other women how to code in Python, right, and it was a great keynote, they actually did a bunch of things like a fable kind of talking about, essentially talking about how challenging it is for women to really be considered a part of the community, of any sort of community.
47:46 So Django girls what is amazing is they started up with pretty much nothing and after the first year they had had a 100 workshops all over the world, again, women teaching women how to program in Python, right. And this was incredibly impressive, and Guido actually in his keynote, I think he is part of the reason it's like this I think he is very committed, he was wearing a Py Ladies T shirt int he conference, Py Ladies is another part of the Python community that works really well, that helps encourage women be involved in Python, in his keynote he said most of it was Q&A from the audience and he said let's 48:24 a man and a women asking questions, right, so he made it super clear that that's priority for him and I asked him about it, is that something that is important for him and he said yes, absolutely and so he is making a difference.
48:40 On a personal basis I had my daughter is, one of my daughters is 24 and she is still trying to find kind of her career direction and she said, dad if I want to learn programming what would you suggest and I said well, I think you should try Python and I said in fact, you've got to check out this Django girls and see if there is a workshop, and it out we live in a Portland area, there was a Django girls workshop in like six weeks and I said oh, look you could get involved in that and she submitted an application, explained what she had been using her knowledge for and why she wanted to learn it, they accepted her in the workshop and you know, she was a little nervous she said, I am not sure I am smart enough to actually do well with this and I said no, I understand sometimes people can feel that way they had been made to feel not smart which I try everything I can with my daughters to make them feel like they are brilliant, sometimes the environment is one that wants to make people who are women not necessarily feel very smart. And so I encouraged her, she went through the workshop, she was very successful with it and I am hoping fingers crossed to be something she would be able to make a living doing if she digs it.
50:03 So yeah, I love the fact that Django girls is available and again, it's a core part of the Python community I think it's one of the things that makes it a very attractive community is how important is for the community to have women help other women and for the entire community to have better inclusion. Now, is it perfect, no, I think there are some challenges still, I interviewed a fellow of the Python foundation her name is 50:36 and I talked with Terry and I said, I asked her how it is from her perspective, one thing for me as a male to say is che this is really good, but how about from a perspective of a woman, and she is part of the Python foundation and she is also an Intel employee, and she said that it's a good environment, it could do better in terms of having more core developer women and so there is more work that can be done there to get more of that inclusion and diversity within sort of a core developer community but it is a great project for its diversity and inclusion.
51:16 Michael: Yeah, that is cool, I think that Python community is more welcoming than most, and I think is just more evidence of it so that's cool, hopefully we can inspire some more people that go out and check out some of these projects that are out there and freely available, very cool.
51:32 David: I agree, yeah,
51:34 Michael: So, we are getting near the end of the show, let me ask you a couple of questions that I always ask my guests, while |I still have you. If you are going to write some Python code, what editor do you open up?
51:45 David: Oh yeah, I am an old VI guy so you know, I have my roots back in Linux and back in the old days, before there was a Linux so yes, when I edit I use VI.
51:56 Michael: Ok, excellent, that is definitely a popular one. And of all the PyPi packages you can include Intel ones if you want, there is a ton of them and nobody could possibly know about them all, there is close to 80 000 maybe over apps, what ones do you like that maybe you would recommend people check out that are not necessarily the most popular?
52:17 David: I asked a question what is my favorite interpreter my favorite interpreter for Python these days and it's definitely PyPy.
52:26 Michael: Yeah, ok, that's great it's a very meta answer, right, like it's PyPy that runs all the packages.
52:33 David: I am not a politician, I don't even get paid to be a politician but sometimes I know how to reframe the question. [laugh]
52:38 Michael: Yeah, absolutely. Ok, so do you have a final call to action for the listeners, things they should do or check out?
52:46 David: Yeah, I would absolutely have people take another look at PyPy, you probably have in the past. In particular we would love to have people check out if you are using open stack or Swift we would love to get people to check it out and look at latest version of PyPy and really try to give us some feedback int he Swift community or in the Python dev community, and if you have some observations about PyPy that would be helpful of saying, for example if you have some Python code and if you either find you have some challenges with it, we would love to solve these, any sort of compatibility issues or other deployment issues and let's try to make this a great experience for people; so if you can help us out in terms of get your hands on PyPy try against your Python code and give us feedback in terms of what might be missing or could work better we would love to hear that, we'd love the dialogue with you on it.
53:53 Michael: Ok, so absolutely try your code on PyPy and send these guys some feedback that's great. All right David, it's been a super interesting conversation, I am really excited to see the work that you are doing up here in the Intel chips going forward and in both the runtimes the CPython and PyPy.
54:13 David: Thank you Michael I love talking about this stuff, I'm incredibly passionate about it and I'd love to see our love for technology be able to make use for a lot of people, so I am hopeful that people will get excited about this stuff.
54:29 Michael: Yeah, absolutely. Thanks for being on the show, talk to you later.
54:32 David: Thank you.
54:32 This has been another episode of Talk Python To Me.
54:32 Today's guest was David Stewart and this episode has been sponsored by SnapCI and Metis. Thank you guys for supporting the show!
54:32 Snap CI is modern continuous integration and delivery. Build, test, and deploy your code directly from github, all in your browser with debugging, docker, and parallelism included. Try them for free at snap.ci/talkpython
54:32 Metis is effective and deep training for data scientists. With immersive bootcamps, in-person, and online courses, you can learn the core data science skills you need to take your career to the next level.
54:32 Are you or a colleague trying to learn Python? Have you tried books and videos that left you bored by just covering topics point-by-point? Check out my onlne course Python Jumpstart by Building 10 Apps at talkpython.fm/course to experience a more engaging way to learn Python.
54:32 You can find the links from the show at talkpython.fm/episodes/show/57
54:32 Be sure to subscribe to the show. Open your favorite podcatcher and search for Python. We should be right at the top. You can also find the iTunes feed at /itunes, Google Play feed at/play and direct RSS feed at /rss on talkpython.fm.
54:32 Our theme music is Developers Developers Developers by Cory Smith, who goes by Smixx. You can hear the entire song at talkpython.fm/music.
54:32 This is your host, Michael Kennedy. Thanks for listening!
54:32 Smixx, take us outta here.