#74: Past, Present, and Future of IronPython Transcript

00:00 Have you heard of IronPython and Jython? These two alternate implementations of Python were created by Jim Hugunin, they run on top of the .Net and JVM runtimes. On this episode, we are going to look at the story of IronPython. It's been around for many years, although the last few years it's been somewhat stagnant, that's why I am thrilled to introduce you to Alex Earl who along with Benedict Eggers has become the maintainer of the IronPython project. It's great to see IronPython getting the attention that it deserves; we'll talk about IronPython past, present and future in this episode of Talk Python To Me.

00:00 This is episode number 74, recorded August 31st 2016.

00:00 [music intro]

00:00 Welcome to Talk Python To Me, a weekly podcast on Python, the language, the libraries the ecosystem and the personalities. This is your host, Michael Kennedy, follow me on Twitter where I am at @mkennedy, keep up with the show and listen to past episodes at talkpython.fm, and follow the show on Twitter via @talkpython.

00:00 This episode is brought to you by Hired and SnapCi. Thank them for supporting the show on Twitter via @hired_hq and @cnap_ci.

01:33 Michael: Hi Alex, welcome to the show.

01:33 Alex: Hey, thanks for having me on, Michael.

01:34 Michael: Yeah, I am excited to talk to you about IronPython and its history, and its future more importantly, where is it going, right? So that's going to be a great conversation.

01:43 Alex: Yeah, definitely.

01:43 Michael: Yeah, I am looking forward to share with everyone.. Before we do though, let's get to your story, like how did you get into programming, in Python?

01:48 Alex: As probably most people my age, in their thirties or so, I kind of got started programming back in the days, there was a game that came with the computer called Gorillas there were two gorillas and they threw bananas at each other, and I like to go in and modify that and make the bananas like purple or just change things about it, and so that was kind of my first introduction to programming. From there I didn't really do much until high school when I took computer programming class for C++ and it was really fun and I enjoyed it. But my first introduction to Python was in college, I had a buddy who I was in several classes with, his name was Travis Heartwell, he introduced me with Python, I had never even heard of scripting languages or even like that, I had only done C or C++.

02:30 Michael: You were like where is the compiler?

02:32 Alex: Yeah, exactly. I can just run these files, you know. So it was pretty awesome, and it was so easy to do things that were like way more complex in C++, the lists and dictionaries were awesome, functions you could use to sort or find things in them, and then he introduced me to list comprehensions and stuff like that, my mind was blown, about how easy it was to do things in Python compared to C or C++. And so, I've just had a lot of the Python ever since then, I didn't use it at that time in college, except for, in one class our professor actually let us he said if you can write program to do your homework for you, go ahead and do that, it was a math class and so I used Python and basically wrote some easy functions to do all my homework for me, for certain assignments and so that made me love Python even more, it could do my homework for me, so that was pretty awesome.

03:27 Michael: Credit to that professor who was like look if you can think outside of the box and solve this problem, do it, that's really cool.

03:33 Alex: Yeah, it was like math for algorithms type of class, it was kind of breaking down algorithms into math and it was really cool to apply Python to that, because I mean, Python is great at that sort of thing.

03:46 Michael: Yeah, sure. Was that like graph theory type stuff, and those kinds of things?

03:49 Alex: Exactly. Yeah.

03:51 Michael: Yeah, it seems like you could totally [3:52] that for a lot of them, find the most efficient path here, well guess what, we'll try them. Beautiful, ok, so that was how you got into programming in Python; what do you do today, what are you focused now?

04:03 Alex: I work for a validation group actually, so most of my work is actually done in assembly or C, I write code that runs on embedded chips. As part of my job as well I also develop some software tools that we use to automate our tests and so forth, and that's where I really got into IronPython. But, so I write software tools and develop a software infrastructure for embedded processor testing.

04:33 Michael: Ok, that sounds fun. Right, so you mentioned IronPython, and I think IronPython is definitely one of the interesting alternative Python interpreters or runtimes out there. And, we've talked about alternate runtimes before on the show like PyPy for example, but I think it's maybe worth just touching on the concept, that there are other interpreters or runtimes out there because maybe not everybody has heard what the options are, can you give us like a little bit of rundown of what the options are?

05:03 Alex: Sure. So as you mentioned, you mentioned PyPy which is a JIT enabled fast interpreter for Python. There is also what we call it from Iron Python and I don't know if other Python interpreters call this but CPython which is the main one developed, you can go to python.org and download the installation or it comes on if you are running like Linux or something it usually comes on Linux. We call that CPython because it's based on the C language.

05:30 Michael: Yes, and if people don't really know what version of Python they have.

05:33 Alex: Yeah, it's definitely true, you type like Python something, and that's usually the CPtyhon, but there are other Pythons. Jython is another option, it's developed on the Java framework, so it runs on the JVM, the Java Virtual Machine, and then there is us, Iron Python which is developed on the .Net runtime. So those are kind of the main ones that I know about, we kind of looked it, those ones in particular as kind of the main areas where we can focus on compatibility and optimization so we can perform well.

06:07 Michael: Yeah, those are definitely the man ones. That's great, and so if you don't realize that there is a bunch of interpreters or runtimes, however you want to think of them, for Python, there may be a better choice in the default Python, the CPython implementation for your project, depending on what you are trying to do. So, let's talk about IronPython because that one has been around for a while, a lot of the ones you mentioned, they are not brand new, there are some new things coming out that maybe we'll touch on a little bit later, some of the JIT based ones, like Pyjon and so on, but let's talk about IronPython, it's been around for a while, when did it get started, do you know?

06:42 Alex: It got started in Microsoft, I think it was around 2006, it's been around for about ten years, right now it has an Apache license, Microsoft kind of took it to a certain point and ten they decided to open it up to be run by the community, and so they open sourced it as Apache license, and so that's the license we currently have.

07:06 Michael: Yeah, I remember it was licensed under a different license like the Microsoft open source license or something that people like what is this, right, and then finally they switched it to the Apache license, like here is something you totally are familiar with, already approved, go with that.

07:20 Alex: Yeah. It was mspl I think, ms public license or something like that, and it had some weird stuff within the license that people were like I don't really want to touch this because it's a Microsoft license so moving it to Apache was a huge benefit to the project.

07:36 Michael: Yeah, that's great. So it was started by Jim Hugunin, is that right?

07:39 Alex: Yeah, he was the one who did Jython as well, so it's kind of the funny story, the whole way the IronPython came around is Jim had started working on Jython and he saw this .Net framework come out and he said I am going to go show that a Python implementation would be horrible to run on the .Net framework. And so he kind of started out IronPython as a way to show that .Net was a horrible platform to work with, but it ended up being great, he said it's a wonderful platform, it's great to write these languages for, and they run really well on top of it. He started out as kind of a trying to attack on that and ending up loving it, even was hired by a Microsoft to continue development on IronPython.

08:19 Michael: Yeah, I remember that and he I think he worked there like six years before he moved on to Google. It's really funny, you still have to prove something that is totally inadequate, and like wait, it's not totally inadequate for this thing, that's pretty interesting, yeah. Do you know how similar Jython and IronPython are? I've never really done a lot of work with Jython because I don't do tons of job, so...

08:34 Alex: They kind of aim at similar ideas, kind of bridging the gap between either Java and Python or the .Net framework and Python. The implementation is very different, mainly because there is while Java and the .Net framework or C# are similar in ideas because they have garbage collector, they are all object oriented, and that sort of thing, really the way that the different platform solve things can be quite different, the way that they implement things in Jython is actually I believe quite different than how things are implemented in IronPython. like their object structure, how they interact with the actual Python modules and then represent those in the Java runtime, you know that can be very different from how we do it in IronPython. I forgot one detail I just wanted to make sure people understood why it's called IronPython, Iron stands for implementation runs on .Net.

09:39 Michael: Right, right, you have a couple of languages, you have Iron Ruby as well. Well, they did when they started out at least, and it is under the project that you are maintaining. Before we get to that though, maybe tell us a little bit about why would I choose IronPython over CPython?

09:55 Alex: The .Net framework has a lot of capabilities, and there is a lot of things, there is the wpf, there is Windows forms for writing GUI apps, there is network libraries that may not be available in Python as a module, and so really the place that I use IronPython the most is creating a scripting environment for my applications, when I write my tools, I generally develop in C# and I use like Windows form or wpf, to create these tools, and to provide a scripting interface for those tools IronPython is very good at embedding into a C# application and accessing the components that you give it access to. It's very easy to provide it access to like a text output window, and so it can write to a text output window very easily that is developed in C#. I've tried to do so many things with CPython I've used Boost Python to try embedding Python into an application, and I ran into some issues with wanting to re-run scripts over and over and over. I got weird assertions and segfaults and stuff like that, so for me, embedding Python in application is very easy with IronPython, and that's my biggest usecase as a software engineer, using IronPython.

11:16 Michael: Ok, so I guess, yeah, you can embed CPython pretty easily, especially if you are doing the C based application, than that probably makes a lot of sense. But if you are already writing in C#, this might be better because you can share information across it better, right, like your classes and libraries in one language are available to the other; does it go both ways?

11:37 Alex: Yeah, you can definitely access IronPython objects and stuff from C#. One of the tools I actually developed was we have a hardware team, they developed boards, with FPGA and stuff on there and they wanted the way to test their hardware platform, and so I created a GUI the interfaces with IronPython and I write scripts to test these boards, and I used the unit test module and so they write their tests and the standard unittest format but I can parse those objects and find all the unit tests for my C# gui, and present those and they can select which ones they want to run, and run those tests to test the hardware boards out.

12:17 Michael: Oh that's nice, so you'd like load the Python modules and inspect what tests are there, and then present some kind of custom runner on top of it?

12:24 Alex: Exactly. Yes.

12:26 Michael: Ok, that sounds really cool. So we were talking about C# and a lot of people say you know, C# means Windows only, and a lot of people doing Python are not interested in Windows. Does it run outside of Windows or is that required?

12:41 Alex: Yes, so we do have pretty basic support for running on Linux under mono, right now, we definitely want to improve that, because we want to be everywhere that people want to use C# and Python, so whether that is Linux or OSX or Windows, we really want to provide the solution to everybody, so we are going to put some focus on improving our support for mono as well as .Net core which runs on Linux right now.

13:10 Michael: Yeah, that's really interesting, I think that that's going to unlock a lot of things like this, that used to be 13:14 more or less to Windows to set them free, to run on Linux machines in the data center, something like that which definitely opens up the appeal I think. What about performance, like is there a performance benefits, doing this versus CPython, are there performance penalties but you get the integration or what's the story?

13:38 Alex: So one I think, so performance benefits we do outperform CPython in several benchmarks, I don't have the details right now, recently someone ran that and they compared CPython, PyPy and IronPython, and we outperformed CPython in many cases, but PyPy definitely outperforms us right now. I think that really probably is related to global interpreter lock, because we don't have that IronPython, it can help us in a lot of those benchmarks. One of the things that we do have, something that we definitely look into optimization and performance problems is startup time, and it takes us a lot longer than CPython to get started, you know, actually executing the Python code and so that's something that we are planning on looking at.

14:20 Michael: Sure. Yeah, if you are just going to run a little bit of code and then shut back down maybe it doesn't make so much sense, and I think PyPy also has that challenge that it takes a lot longer to get rolling at a high speed but once it's up and it sort of like a web app type of environment where it starts and runs for a long time than it's better.

14:39 Alex: Yeah, definitely.

14:39 Michael: Ok, great. Now, one of the things I thought was really intriguing about IronPython and you brought it up is the lack of the GIL or the global interpreter lock, and that seems to be one of the challenges for doing real parallel computational parallelism in Python. You can do it and you can use the multi process support and things like that, but straight up threading, when you are just doing computational stuff is challenging, right. Is that more doable in IronPython?

15:09 Alex: Yeah, I would say so because we don't have that the gil in IronPython, when we have something that can be spread across multiple cores, which .Net supports very well, you can really take advantage of multiple cores and not running into the, the gil which can slow you down.

15:24 Michael: Ok, so people out there may have some project and they are trying to use parallelism, how or what hard would it be for them to switch from running CPython to try out IronPython with their code, like what's the chances on taking what they have and running with the different interpreter and having it work?

15:42 Alex: It depends quite a bit on what modules are using, because we are running as a .Net application, we really can't use C modules, C based modules, so if you have something that is based on a C module it's hard for us to interact with that right now. We do have a side project that had started a while ago, when we were working on Python 2.6 compatibility called IronClad which allowed us to interact with C modules, it was kind of abandoned for a while and now we are going to try and bring it back so that we can reuse some of these C modules that CPython provides. So the more things are compatible under IronPython.

16:23 Michael: Ok. Yeah, that seems really important to me, I think one of the lessons form PyPy is it's really great but it had these huge challenges around some of the popular packages that relied heavily on C, like Numpy for example. And they went so far to try to implement their own version of Numpy, in PyPy or in Python, in pure Python, and that just seems like such a hard battle, to like look at all the open source projects that are success and then go let's just reimplement all of those. It seems almost impossible.

16:57 Alex: Yeah, it feels like you always play catch up and that's the hard position to be in, for one thing you are trying to maintain compatibility with the language itself which can change slightly from rev to rev even on just 2.7 baseline. But then you have the open source projects that provide modules that are native, or written in C and those can be difficult to keep up with as well, so it's definitely, a trial that we are trying to work on and figure out-

17:24 Michael: Yeah, it sounds like IronClad would be really important or something filling that role at least. How did you end up becoming one of the maintainers? And, you have a co maintainer as well, right?

17:33 Alex: Yeah. So my co-maintainer is Benedict Eggers, he is located in Germany, he was kind of what really kind of kicked off the IronPython I would say, ever since Microsoft open sourced it I had been working doing some stuff, because I was using it in my job so I wanted it to be a good piece of software, and Jeff Hardy was the maintainer at the time and he's gotten with all of us, we go through hills and valleys of how busy we are with our jobs and families and stuff, and he just reached the point where his job and his family were taking up a lot of time as they should, and he just didn't have the time that he thought he could put into helping IronPython be successful and so Benedict kind of pushed some buttons using some Twitter conversations between Jeff and myself and Benedict to kind of push in and Jeff finally realized that he could pass it up to myself and Benedict. Myself because I had some history with the project and so and forth, Benedict because he really had some excitement about it, to get it going again. That's kind of how I got in there and you know, Benedict is good at making people excited about tings and so I got excited about IronPython again and we started making plans for the future.

18:52 Michael: Yeah, that's cool. And, it's interesting that was over Twitter, I feel like people outside the development space don't really appreciate how special Twitter is, in connecting you with people who you would otherwise not be connected to.

19:07 Alex: Yeah. I mean you can go to [19:10] project and see like an email address for a developer but you don't know always is it current, are they using that email address, but Twitter is so easy to connect real fast.

19:19 Michael: Yeah, and the messages are small enough you are likely to get a response, if you write some 500 word email, people are just going to ignore you most of the time if they are busy or whatever, but yeah, so I think it's really cool that started over Twitter.

19:19 [music]

19:19 This portion of Talk Python To Me is brought to you by Hired. Hired is the platform for top Python developer jobs. Create your profile and instantly get access to 3500 companies who will work to compete with you.

19:19 Take it from one of Hired users who recently got a job and said, "I had my first offer on Thursday, after going live on Monday and I ended up getting eight offers in total. I've worked with recruiters in the past but they've always been pretty hit and miss, I've tried LinkedIn but I found Hired to be the best. I really like knowing the salary upfront, privacy was also a huge seller for me."

19:19 Sounds awesome, doesn't it? Wait until you hear about the signing bonus- everyone who accepts a job from Hired gets a $1000 signing bonus, and as Talk Python listeners it gets way sweeter. Use the link hired.com/talkpythontome and hired will double the signing bonus to $2000. Opportunity is knocking, visit hired.com/talkpythontome and answer the door.

19:19 [music]

20:41 Michael: So do you want to talk a little bit about the internals about how this works? I know how it works in CPython, we have Python bytecode and we have this eval loop that will go through and pull out this bytecode instead of JIT compiling it it will just interpret it, and piece it all together. What happens in IronPython? How is it same or different?

21:01 Alex: Sure. It's kind of a similar concept. We parse the Python code into an abstract syntax tree and then we kind of do some optimization or yeah, you can call it optimization, so when something is executed, we generate what is called the call site for the operation, so that in the future, if we execute the same code, or similar code, we have an optimized version of that code that we can then call. So we basically take the Python and we convert that in abstract syntax tree and then that gets converted into like a lambda basically in the .Net world. Something that we can execute, and we cash off the compiler optimized version of that call so that if we get another git [21:48] call for instance we have an optimized version that we can call and we don't have to recompile it into the lambda format, from the abstract syntax tree.

21:55 Michael: Ok, that's interesting. So, every instruction when I run python code in IronPython is JIT compiled, eventually?

22:05 Alex: Yes, because it gets converted into a .Net IL, which is the intermediate language, the JIT will run on that code when it's executed and can get cashed off.

22:16 Michael: All right, that's a really interesting difference than say PyPy, because when I talk to the PyPy guys they said it initially runs through CPython, but if it runs enough they are like all right, this is the hotspot we are going to JIT this part but it doesn't always JIT which is interesting, I think it makes it a little less predictable, perhaps, in the way that PyPy works, so that's pretty cool. So I guess that means like things like garbage collection apply as well, right? The standard garbage collection that you would know from there.

22:47 Alex: Yeah, definitely the .Net framework includes garbage collection and so we don't- unlike the CPython which pass the reference counting on objects in the internals we don't have to do that, it's all done automatically for us and the gs takes care of anything that is not around anymore.

23:03 Michael: Ok. That definitely makes it easier on you as an implementer, what are the benefits or drawbacks of that? Versus say reference counting in CPython?

23:11 Alex: In CPython it's deterministic because you are telling the runtime when something is no longer needed, and so it can clean it up right then in running under .Net with IronPython it's the garbage collection is none deterministic. So it can run even during a critical point in your app and slow things down slightly. Now, because we are running on .Net we can have multi core stuff so the garbage collector may be running on one core and so you still have like three cores out of four that could actually execute code but there is non-deterministic garbage collectors and Jython would have the same issue, as java runtime garbage collector is also non-deterministic, so-

23:53 Michael: Right, it's quite similar, yeah. Interesting. Yeah, I remember playing around with some of those ideas and checking out the performance realizing that allocation was actually faster a lot of the times in some of the JC spaces but you paid for later when it had to be cleaned up because that was way more complicated, because the compacting JCs and you don't have to like find an empty slot and the free space, it usually gets compacted. Ok, that's pretty cool. So you brought up some of the different types of apps that you can build with IronPython- do you want to give us a bit of a rundown on that? So you said you can build wpf apps which is sort of the standard Windows UI framework these days, what else?

24:38 Alex: So we actually have a module, a wpf module so you can import wpf and it provides kind of higher level ways of interacting with wpf that are more Python like so it makes it easier to develop those sorts of things, Benedict actually does quite a bit with wpf on IronPython. We also have you can write Windows forms which is kind of the older format of Windows applications. Wpf is supposed to replace Windows forms but it hasn't happened yet, people still write windows forms apps, you can write windows forms apps under IronPython. Now, you have to start them differently than you would like a command line application on Windows when you are running a Windows application, it has to have an event loop, n order for it to process Windows events and things like that, the normal command line client doesn't have that message loop so it can't run Windows forms apps or wpf apps, and so we actually have a separate interpreter executable, which allows or has a event loop started up so that your forms applications and stuff work correctly.

25:49 Michael: Oh, that's interesting, because it has to plug in basically to that already running the event loop or something like that, right?

25:55 Alex: Exactly. Yes.

25:56 Michael: Ok, and can you build- what do you distribute I guess, like do people have to have IronPython installed in the system, can you give them like an executable, self contained, what thing do you give the people to run?

26:11 Alex: We do have what is called the IronPython compiler, which is basically what it does is takes any .py files that you want, and you can actually also include Python standard library, and you can build a standalone executable that is basically a has the Python code as a resource embedded inside and then some wrapper code to pull out those resources and execute as if you are running it from the command line using the PyPy interpreter. So you can crate pretty standalone executables, they are pretty large because you are embedding everything inside, but you can distribute a single executable that has all the data inside for running your application.

26:53 Michael: Oh that is pretty awesome. Does that take external packages like if I am using requests or passlib or some of these things? Will it get sucked in and become part of my project?

27:04 Alex: Right now we don't have automatic dependency checking for that, so yo would have to specify the path of the .py file on your system, but that is something we are looking into, I recently created a patch for model finder for IronPython because we don't have bytecode and model finder uses bytecode, the Python bytecode to determine what's being imported, but I created a patch that uses ast module to find all the imports within your project as well, that is something I am going to roll into the IronPython compiler so that it can automatically find the dependencies that you are using. And include them.

27:43 Michael: Ok, what's the timeline on that?

27:44 Alex: Probably in the next major release of the 2.7 branch that we are working on, we are working on IronPython 3 which would be 3.x support but we are still maintaining the 2.7 release track as well and so probably the next major release we are going to do another minor release pretty soon because we had some [28:03] it was our first releases as project owners and we are humans, so we made a few mistakes, but we are working to fix those and getting all kind of minor release but I think in the next major release we can include that types of work.

28:18 Michael: So what is the story with Python 3 versus Python 2 since you brought it up?

28:23 Alex: Sure, so in .Net all strings are unicode, so we've had some Python 3 support for a while now because of that, it also causes problems with some of the 2.7 modules but that's kind of a different story, but we are working on implementing all of the features for the most recent 3.x series, it's in a separate code repository from the IronPython 2, because we are going to make some fairly significant changes to the parsing logic for the new language features and things like that. But it will also allow us, you know, a lot of those unicode issues will go away with IronPython 3 which because Python 3 switched to unicode as well.

29:05 Michael: Sure. So is that something I can use today or is it still being worked on?

29:10 Alex: It's still being worked on, we don't have all the language features in yet, we are working on trying to get out a first release in the end of October, that has language features implemented and then if we are going to work on fixing bugs and making sure all the tests are passing and we are really trying to push standard library features upstream, so that you can use the normal Python standard library. Right now we have some changes in the standard library and have a local version of it, so you can't just go and grab the standard library of CPython and use it with IronPython and have it work, but we want to push those changes upstream so that there you can use the normal CPython standard library with IronPython 3.

29:52 Michael: Ok. Yeah, it seems like a great plan. You touched on threading, and one thing I did want to ask you about, since we are talking about Python 3 is there is some coroutine parallelism features in Python 3 with the async and await keywords. And C# also has async and await keywords, so is there some special [30:14] just like async and await do anything extra cool on IronPython versus regular Python 3?

30:20 Alex: The async and await stuff is something we haven't implemented yet, that I think it's going to be fairly easy because C# and .Net have the async and await already built in, but that's something we haven't implemented yet, in IronPython 3 yet.

30:34 Michael: Ok, and speaking of implementation, we said that CPython is not mostly Python, mostly it's C code in the actual low level parts, right, so it's some of the standard library but what is IronPython written in?

30:48 Alex: IronPython is written in C# and so the native modules or some of the built in modules are implemented in C#, we replicate the functionality of the native C modules, things like _hashlib, _ssl, things that are imported in the Python module like sll.py or hashlib.py. We implement those and try and create them as feature complete as possible in C#.

31:15 Michael: Ok, and how hard is it to match those things up, do you have like tons of tests, do you run the standard Python test to verify your code or anything like that?

31:23 Alex: Yeah, we definitely use all the tests that are the already there for the CPython int he standard library, we use those heavily, in addition we have some features that we tests for like COM, which is the Windows technology, the Common Object model, we have some stuff that makes it easier to work with COM objects and so we have some tests that do that, we have tests for interaction between C# and IronPython and make sure that if you are hosting IronPython in your application it continues to run well, so we do use the CPython tests and then we add additional tests on top of that.

31:58 Michael: Right. Because you are trying to add additional functionality I guess. Ok. Also, you talked about Windows apps, can you build web apps, which is maybe where Python is used more?

32:09 Alex: Yeah. So Benedict has been working on something that he recently imported into the Iron languages repository on organization on github, which is ironpythonasp.net nbc and basically what it does is it allows you to write asp.net nbc apps, web apps in IronPython or in Python. So you can define your routes and that sort of thing in Python. So it's a really exciting thing and that's one of the things we really want to get running out of .Net core as well.

32:37 Michael: Yeah, that sounds awesome, if you can get that running on .Net core, so I could develop that on my Mac and deploy it on Linux, that's pretty interesting.

32:47 Alex: Yeah, and you still get the power of some of the .Net libraries that may not have a Python module to do.

32:53 Michael: Sure. Ok, well yeah, that sounds pretty exciting, I am looking forward to that. What about managing like local environments CPython has great support for virtual environments, especially in Python 3, it's got venv built in, do you guys have like this concept of virtual environments with what you build?

33:13 Alex: We had some patches upstream to virtual env because there is some things that we were having some issues with, they requested us to make some changes and all that kind of happened back when development was really kind of dying on IronPython, and so it's something we need to resurrect and re-look at, but it's definitely something we want to support, because people love that as part of CPython so- we'd like to provide that as part of IronPython as well.

33:38 Michael: Yeah, it's definitely pretty important. Ok, cool, and then you also have something called PyLauncher, how is that really- is that like the thing that will sort of associate with Python files and run them or what's the story there?

33:49 Alex: It is a launcher, I don't know if you are familiar with Visual Studio at all, Visual Studio has a bunch of different formats for their project files, and so they have a what is called a Visual Studio launcher, Visual Studio launcher will get these format of the file that it's received, and determine which version of Visual Studio that you have on your system you should open it with. And that's kind of what PyLauncher does as well, it allows you to have multiple different Python versions on your system and launch a different one based on different things within the files.

34:19 Michael: Right. Ok. So, you mentioned Visual Studio, and Visual Studio is fine, if you are on Windows, but if you are not than you don't really have it, and I am also a fan of PyCharm as people who listen the show know, could I say open PyCharm and write IronPython apps with it?

34:35 Alex: Yeah, I mean, we do run on mono right now so which will install on OSX or on Linux and so you know, some people I know who are working on IronPython itself use like Emacs or Vim or their favorite editor and they write the Python and then they execute it from the command line using the ipy.exe so if you are developing the Python itself you can write it anywhere, and run it under the interpreter. Now, there is an awesome tool from Microsoft called the Python tools for Visual Studio, which is basically a free Python IDE if you are running on Windows, they do an awesome java, and that kind of started out of IronPython initially and so that is kind of what I like. Yeah, you can use basically any IDE you want to write the Python and then you can just execute it on the IronPython interpreter.

35:27 Michael: All right, yeah, that sounds really cool. Yeah I remember the IronPython tools for Visual Studio just became the Python tools for Visual Studio at some point, right?

35:36 Alex: Yeah. And so they work with IronPython, they work with CPython, I don't know if it works with PyPy, I've never tried that, but as I've created this tools here at work, that use IronPython I had to recommend you know, GUIs for people who were hardware engineers and stuff like that to write they can get syntax highlighting, intellisense types that are auto-completion stuff of methods and parameters, documentation, so yeah, Python tools is very good for that. Most people here run Windows so it's easy for me to recommend one of those tools to them.

35:36 [music]

35:36 We all want to ship our software faster, respond to user request quicker and build an edge on the competition. But the faster you go, the more likely a bug or issue will slip by and cause your users' grief. I have the same trade-offs with my Talk Python websites and infrastructure. That's why I use Snap CI. Every time I check into github, Snap CI springs into action. A clean copy of code at github, a Python 3 virtual environment is created all the packages and dependencies are installed, the unittests are run. Finally, I am notified in Slack and in many bar on my Mac about the outcome. That how I know it's safe to ship a shiny new version of my web app. Ship your software faster with the 30 days free trial at snap.ci/talkpython.

35:36 [music]

37:06 Michael: Yeah, there is a whole let's say vibrant debate that we could start about whether you should use IDEs or you should use something minimalistic like Emacs or Sublime, relatively speaking. Maybe we shouldn't really go into it but I kind of lean a little more towards the IDE side myself on the PyCharm space, and I think the tools if you are on Windows I think the Visual Studio tools are pretty sweet as well, I played with them a little. So one thing that I thought was interesting that I saw on the mailing list going around was about the time that you took over I think, there was a meeting that said there is a bunch of urgent issues and we are going to sort of reset the direction of IronPython. When was that?

37:55 Alex: So we had our first meeting a couple months ago, we've been trying to have them every two weeks on Thursday, and we have them on gettr, I don't know if you are familiar with gitter it's basically allows you to use your gihub credentials and organizations to have meeting rooms in a webpage.

38:12 Michael: Yeah, and they are generally like a round or project on github, right?

38:15 Alex: Exactly. So we have room for our organization which is the Iron languages organization.

38:20 Michael: Sure, so tell people what the github url is, of course I'll put it in the show notes but-

38:24 Alex: Sure, we are at github.com/ironlanguages/main is currently the main repository for the Python 2.7 branch, then there is github.com/ironlanguages/ironpython3 for the work we are doing for our Python 3 support.

38:45 Michael: all right, excellent. Cool, I'll definitely link to those. So the meeting, the first meeting some of the issues that people brought up were this idea of IronRuby, like what is it doing here, does it make sense in IronPython, like do you want to talk about that a little?

39:00 Alex: Sure, so when Microsoft open sourced the code, they had two languages are basically implemented on top of what is called the dynamic language runtime which is kind of a set of classes or objects that make writing dynamic languages on top of .Net little easier. It does all that stuff about the call sites that I talk about cashing things and converting abstract syntax tree into the lambdas for calling the code and so forth. So they had IronPython which is implementation of Python and they had IronRuby which was an implementation of the Ruby language on top of the blr. So what we kind of did was we split out IronRuby into its own repository under the organization, and I submitted the message to the Ruby mailing list, say if there is anybody interested in taking up IronRuby and keeping it alive. And so a couple of people responded and we were willing to help them, in any way we can, but we really kind of just split it out so that they could lead their own destiny wherever they want to take it.

40:03 Michael: Yeah. That makes a lot of sense to me, because while there is some similarities there is a lot of differences between those two communities and languages and I feel like, maybe correct me if I am wrong, but I feel like the mission of IronPython these days is no longer to say we are going to be this cool thing that shows off the ability to do dynamic languages on .Net, it's all about making the Python as an alternate implementation special, right, and so that makes the IronRuby being tied up in it make a lot less sense, right?

40:33 Alex: We don't want to tie them down neither with the decisions that we make going forward, we don't want to impact them in a negative way if they want to keep it alive and keep it going. So allowing them to have their own space is I think a good way to let them what they need to do and let us do what we need to do to make both of our languages the best we can.

40:56 Michael: Yeah, excellent. Ok, I'd say that's a good choice. And then, when you moved over from where it used to be hosted to github there was something like 960 issues that were just sort of stale out there, and you were like what do we do with these? Did you end up throwing them all away, did you keep them, like how did you manage that transition?

41:15 Alex: So, and all those issues were originally filed on code plugs which is Microsoft's open source competitor if you will, to github. I actually wrote a Python script that ported them, did a screen scrape using beautiful soup on code plug because I didn't have an API to do this, and then importing them into github using one of the github Python modules. So, that was a lot of fun.

41:37 Michael: Yeah, who says they don't have an API, they do, they just don't know it.

41:40 Alex: That's right. So anyway, there were these 960 odd issues, some of them were as old as 2006 or 2007, pretty old, and so what we decided to do in the meeting is to go through and we marked them all as what we called untriaged where we said we we didn't know what was going on, what needed to happen, or anything like that. And we have about 10 people who were in the community meeting that said that they would go through issues and try and triage whether they are a valid issues still, whether it's something that doesn't exist anymore, so we could just close it down, or things along those lines, and so we've been working on that, I think we're down to about 800, so we haven't made a huge amount of progress, but we are making progress on triaging the issues and seeing, you know, we need to fix these issues, some of them that we've triaged we fixed as well, so some of them were very simple fixes to make, we just had to know that they were still an issue and then go on and fix it. But we are slowly making our way through those and porting things from the untriage state to either closed or yes this is still an issue to I don't know what do we do with this, let's talk about this one further.

42:54 Michael: Sure. I think it highlights one of the real big challenges the non glamorous side of running a large open source project, right, like probably you could have your dream, whatever you want to do technically for your time, what do you want to dream about is probably not triaging 900 old issues that might not even be relevant, right? But, it's important work.

43:18 Alex: Yeah. It can be very satisfying to see that count go down.

43:21 Michael: Yes, I'm sure.

43:23 Alex: Seeing it go down is very satisfying.

43:28 Michael: Sure. Well, when I look at open source projects you know, on github or whatever, that is one of the indications of whether or not I want to trust this thing still being around, how old are the issues are they getting responded to, also poll request, like you'll see projects that have year old poll requests, it seems just so sad because somebody went to the extra effort of fixing code for you and you didn't bother to pull it in or even respond, you know.

43:54 Alex: Yeah, completely.

43:55 Michael: Yeah, it's tough. You also talked about release schedule, what's the story there, how often, what versions 2, 3 and so on?

44:02 Alex: Sure. Well, we are going from having like one release every four years to we'd really like to get on more regular release schedule. So we did 3 beta releases, before we did the most recent 2.7.6.3 release, that was our most recent final release, and so we really want to do that every couple of months, do a pull release with the fixes and stuff, so people know that we are actively working on it and we are actively fixing issues.

44:37 Michael: Yeah, ok, cool.

44:38 Alex: Every couple of times we like to do a final release for you know, either a minor release or major release.

44:44 Michael: Yeah, that's way better than every four years.

44:45 Alex: Yeah.

44:47 Michael: Nice. All right, then you also talked about continuous integration, IronPython 3 which we touched on a little bit, and then IronClad or the ability to support C extensions, and I definitely want to just put my vote in to say from what I've seen, the challenges the PyPy ran into, if you can support C extensions in super clear automatic way, that could seriously unlock the adoption.

45:12 Alex: Yeah. Definitely, it would be a huge benefit for people using IronPython and we wouldn't have to implement as many of the native modules in C# or whatever and there may be cases where we want to because we can optimize it better without the gil but because the native libraries would assume there is a gil there, so there are some things we probably still keep in C# or implemented C# because we would want to avoid the gil.

45:39 Michael: Yeah, that makes sense, but it would be nice to let that be a choice and not a necessity.

45:43 Alex: Definitely.

45:45 Michael: Yeah, awesome. All right Alex, we are getting near the end of the show, at the end of the show I always ask people a couple of questions, so let me put them to you now. On PyPi, there is over 80 thousand packages, so many out there that you can use and just, so many that people probably have never even heard of, like what is an amazing one that you would like to sort of shine a light on?

46:07 Alex: It's not a super like glamorous module I would say, but I use Python with the pyserial module, we actually wrote an application to allow us to interact with body bugs, I don't know if you are familiar with body bug, it's a wrist or an armband e-ware that tracks different things like your pulse, and it's got a step counter and stuff like that, and you have to have subscription on their website to interact with it, and so we kind of looked around on the web and found some information to interact with it over the serial port, and so we used pyserial to interact with our body bug and get the same data. It's not a glamorous model but it's a lot of fun to be able to reverse engineer and do stuff like that with our body bugs.

46:52 Michael: Sure, it's a cool story, you have these companies with the weird restrictions and business models, whatever, you are like that's my data, is that my arm, come on.

47:00 Alex: Exactly, yeah.

47:01 Michael: Nice. I could probably guess what you are going to give me for the next question as an answer, but what's your favorite editor when you write Python code?

47:08 Alex: Actually, I use Vim for most of my Python, I know I said Python tools for Visual Studio is awesome that a lot of times I am doing some quick edits to the Python code or something like that so I got introduced to Vim years ago in college and I use that quite a bit, I've tried to use Spacemax which kind of has a Vim mode for Emacs, but Vim is my go to editor.

47:33 Michael: Oh right on. Very cool, all right, well, any final call to action, are you looking for people to help out on the project, people to try it out, what do you guys need?

47:42 Alex: Oh absolutely, we would love to have anybody who wants to you don't even need to write C#, there is all sorts of documentation we need to figure out, we do reuse a lot of the Python but there are some things specific to IronPython, testing things out, see if there are still issues, we are still in that issue disposition, anybody is welcome to come to guthub and look at the issues and see if they can reproduce it or give us a little bit more information, to kind of push us towards getting a fix faster, so anybody who wants to get involved, we'd love to have them on there, they can add on the mailing list, which we have through python.org ironpython users, yeah, we would welcome anybody who wants to help out.

48:23 Michael: That sounds great, all right, if it sounds interesting to you, get out there and check it out, Alex, it’s been fun to talk about IronPython, I'm excited to see this coming back to life, it plays an interesting role in the ecosystem, so nice work and keep it up.

48:36 Alex: Thanks for having me on it's been a blast.

48:39 Michael: You bet, talk to you later.

48:39 This has been another episode of Talk Python To Me.

48:39 Today's guest was Alex Earl and this episode has been sponsored by Hired and Snap CI. Thank you both for supporting the show!

48:39 Hired wants to help you find your next big thing- visit hired.com/talkpythontome to get five or more offers with salary inequity presented right upfront and a special listeners' signing bonus of $2000.

48:39 Snap CI is modern continuous integration and delivery. Build, test, and deploy your code directly from github, all in your browser with debugging, docker, and parallelism included. Try them for free at snap.ci/talkpython

48:39 Are you or a colleague trying to learn Python? Have you tried books and videos that left you bored by just covering topics point-by-point? Check out my online course Python Jumpstart by Building 10 Apps at talkpython.fm/course to experience a more engaging way to learn Python. If you're looking for something a little more advanced, try my write pythonic code course at talkpython.fm/pythonic.

48:39 You can find the links from the show at talkpython.fm/episodes/show/???

48:39 Be sure to subscribe to the show. Open your favorite podcatcher and search for Python. We should be right at the top. You can also find the iTunes feed at /itunes, Google Play feed at /play and direct RSS feed at /rss on talkpython.fm.

48:39 Our theme music is Developers Developers Developers by Cory Smith, who goes by Smixx. Cory just recently started selling his music on iTunes so I recommend you check it out at talkpython.fm/music. You can browse his music there and listen to the full-length version of Developers Developers Developers.

48:39 This is your host, Michael Kennedy. Thanks for listening!

48:39 Smixx, take us out of here.