#74: Past, Present, and Future of IronPython Transcript

00:00 Have you heard of IronPython and Jython?

00:01 These two alternate implementations of Python were created by Jim Hugenin.

00:05 They run on top of the .NET and JVM runtimes.

00:08 On this episode, we're going to look at the story of IronPython.

00:12 It's been around for many years, although the last few years it's been somewhat stagnant.

00:16 That's why I'm thrilled to introduce you to Alex Earle, who, along with Benedict Eggers, has become the maintainer of the IronPython project.

00:24 It's great to see IronPython getting the attention that it deserves.

00:27 We'll talk about IronPython past, present, and future in this episode of Talk Python to Me.

00:32 This is episode number 74, recorded August 31, 2016.

00:54 Welcome to Talk Python to Me, a weekly podcast on Python, the language, the libraries, the ecosystem, and the personalities.

01:09 This is your host, Michael Kennedy.

01:11 Follow me on Twitter, where I'm @mkennedy.

01:13 Keep up with the show and listen to past episodes at talkpython.fm.

01:17 And follow the show on Twitter via at Talk Python.

01:21 This episode is brought to you by Hired and SnapCI.

01:23 Thank them for supporting the show on Twitter via at Hired underscore HQ and at Snap underscore CI.

01:30 Hi, Alex. Welcome to the show.

01:33 Hey, thanks for having me on, Michael.

01:34 Yeah, I'm excited to talk to you about IronPython and its history and its future, even more importantly.

01:40 Where is it going, right?

01:41 So that's going to be a great conversation.

01:42 Yeah, definitely.

01:42 Yeah, I'm looking forward to share it with everyone.

01:44 Before we do, though, let's get to your story.

01:46 Like, how did you get into programming in Python?

01:47 You know, as of probably most people my age, mid-30s or so, I kind of got started programming back in the Q&A days.

01:53 I like, there's a game that came with a computer called Gorillas, where there are two gorillas and you threw bananas at each other.

02:00 And I like to go in and modify that and make the bananas like purple or just change things about it.

02:06 So that's kind of my first introduction to programming.

02:09 From there, I didn't really do much until high school when I took a computer programming class for C++.

02:15 And it was really fun and I enjoyed it.

02:17 But my first introduction to Python was in college.

02:20 I had a buddy who I was in several classes with.

02:23 His name is Travis Hartwell.

02:24 He introduced me to Python.

02:26 I'd never even heard of scripting languages or anything like that.

02:28 I'd only done C or C++.

02:30 You're like, where's the compiler?

02:31 Exactly.

02:32 Yeah.

02:32 I can just run these files, you know.

02:34 So it was pretty awesome.

02:35 And it was so easy to do things that were like way more complex in C++.

02:41 You know, the lists, dictionaries are awesome.

02:46 Functions you can use to sort or find things in them.

02:49 And then he introduced me to lists, comprehensions and stuff like that.

02:53 My mind was blown about how easy it was to do things in Python compared to C or C++.

02:58 And so I've just had a love of Python ever since then.

03:02 I didn't use it a ton in college except for in one class our professor actually let us.

03:08 He said, if you can write a program to do your homework for you, go ahead and do that.

03:12 It is a math class.

03:14 And so I used Python and basically wrote some easy functions to do all my homework for me for certain assignments.

03:21 And so that made me love Python even more.

03:23 I could do my homework for me.

03:24 So that was pretty awesome.

03:26 Credit to that professor who's like, look, if you can think outside of the box and solve this problem, do it.

03:32 That's really cool.

03:32 Yeah.

03:33 It was like a math for algorithms type class.

03:37 It was kind of breaking down algorithms into math.

03:40 And it was really cool to apply Python to that because, I mean, Python is great at that sort of thing.

03:46 Yeah, sure.

03:46 Was that like graph theory type stuff and those kinds of things?

03:49 Exactly.

03:50 Yeah.

03:50 Yeah.

03:50 Yeah.

03:51 It seems like you could totally automate that for a lot of them.

03:53 Find the most efficient path here.

03:55 Well, guess what?

03:56 We'll just try them.

03:56 Beautiful.

03:57 Okay.

03:58 So that was, yeah, that's how you got into programming in Python.

04:01 What do you do today?

04:02 Where do you focus now?

04:03 I work for a validation group, actually.

04:06 So most of my work is actually done in Assembly or C.

04:11 I write code that runs on embedded chips.

04:14 As part of my job as well, I also develop some software tools that we use to automate our tests and so forth.

04:23 And that's where I really got into Iron Python.

04:27 So I write software tools and develop a software infrastructure for embedded processor testing.

04:32 Okay.

04:33 Yeah, that sounds fun.

04:34 All right.

04:34 So you mentioned Iron Python.

04:35 And I think Iron Python is definitely one of the interesting alternative Python interpreters or runtimes out there.

04:44 And we've talked about alternate runtimes before on the show, like PyPy, for example.

04:50 But I think it's maybe worth just touching on the concept that there are other interpreters or runtimes out there because maybe not everybody's heard of what the options are.

05:00 Could you give us like a little bit of a rundown of what the options are?

05:03 Sure.

05:03 So as you mentioned, you mentioned PyPy, which is a JIT-enabled fast interpreter for Python.

05:09 There's also what we call it from Iron Python.

05:11 And I don't know if other Python interpreters call it this, but CPython, which is the main one developed.

05:17 You know, you can go to python.org and download the installation or it comes on if you're running like Linux or something, it usually comes on on Linux.

05:25 We call that CPython because it's based on the C language.

05:28 Yep.

05:29 And I feel like if people don't really know what version of Python they have.

05:32 Yeah, it's definitely true.

05:33 You know, you type like Python something and that's usually the CPython, but there are other Pythons.

05:39 JIT-enabled Python is another option.

05:42 It's developed on the Java framework.

05:45 So it runs on the JVM, the Java virtual machine.

05:48 And then there's us, Iron Python, which is developed on the .NET runtime.

05:52 So those are kind of the main ones that I know about.

05:56 We've kind of looked at those ones in particular as kind of the main areas where we can focus on compatibility and optimization so that we can perform well.

06:07 Yeah, those are definitely the main ones.

06:08 That's great.

06:09 And so if you don't realize that there's a bunch of interpreters or runtimes, however you want to think of them, for Python, you know, there may be a better choice than the default Python, the CPython implementation for your project, depending on what you're trying to do.

06:24 So let's talk about Iron Python, because that one's been around for a while.

06:29 A lot of the ones you mentioned, they're not brand new.

06:31 There are some new things coming out that maybe we'll touch on a little bit later, some of the JIT-based ones like Pigeon and so on.

06:37 But let's talk about Iron Python.

06:40 It's been around for a while.

06:40 When did it get started?

06:41 Do you know?

06:42 It got started at Microsoft, I think it was around 2006.

06:47 It's been around for about 10 years.

06:50 Right now it has an Apache license.

06:53 Microsoft kind of took it to a certain point and then they decided to open it up to be run by the community.

06:59 And so they open sourced it as Apache license.

07:03 And so that's the license we currently have.

07:06 Yeah, I remember it was licensed under a different license, like the Microsoft open source license or something that people are like, what is this?

07:13 Right?

07:13 And then finally they switched it to the Apache license.

07:16 Just go, here's something you're totally familiar with.

07:18 Already approved.

07:19 Go with that.

07:19 Yeah.

07:20 It was MSPL, I think.

07:22 The MS public license or something like that.

07:24 And it had some weird stuff within the license that people were like, I don't really want to touch this because, you know, it's a Microsoft license.

07:31 So moving it to Apache was a huge benefit to the project.

07:35 Yeah, that's great.

07:36 So it was started by Jim Hugenin, is that right?

07:39 Yeah, he was the one who did Jython as well.

07:41 So it's kind of a funny story.

07:43 The whole way that Iron Python came around is Jim had started working on Jython and he saw this .NET framework come out.

07:50 And he said, oh, I'm going to go show that a Python implementation would be horrible to run on the .NET framework.

07:57 And so he kind of started out Iron Python as a way to show that .NET was a horrible platform to work with.

08:03 But it ended up being great.

08:04 He said, it's a wonderful platform.

08:06 It's great to write these languages for.

08:08 And they run really well on top of it.

08:11 He started out as kind of trying to attack .NET and ended up loving it.

08:15 And he even was hired by Microsoft to continue development on Iron Python.

08:19 Yeah, I remember that.

08:20 And I think he worked there for like six years before he moved on to Google.

08:23 It's really funny.

08:24 You start out to prove something that is totally inadequate.

08:28 You're like, wait, it's not totally inadequate for this thing.

08:30 That's pretty interesting.

08:31 Yeah.

08:32 Do you know how similar Jython and Iron Python are?

08:35 I've never really done a lot of work with Jython because I don't do tons of Java.

08:40 They kind of aim at similar ideas, kind of bridging the gap between either Java and Python or the .NET framework and Python.

08:47 The implementation is very different, mainly because there's while Java and the .NET framework or C Sharp are similar in ideas because they have garbage collector.

08:58 They're all object-oriented and that sort of thing.

09:01 Really, the way that the different platforms solve things can be quite different.

09:05 The way that they implemented things in Jython is actually, I believe, quite different in how things are implemented in Iron Python.

09:12 Like their object structure, how they interact with the actual Python modules and then represent those in the Java runtime.

09:21 That can be very different from how we do it in Iron Python.

09:25 Okay.

09:27 I forgot one detail.

09:28 I just wanted to make sure people understood why it's called Iron Python.

09:31 Iron stands for implementation runs on .NET.

09:35 Okay.

09:36 So that's why it's in Python.

09:38 Right, right.

09:39 You have a couple of languages.

09:40 Like you had Iron Ruby as well.

09:42 Well, they did when they started out at least.

09:44 And it is under the project that you're maintaining.

09:46 Before we get to that, though, maybe tell us a little bit about why would I choose Iron Python over CPython?

09:54 The .NET framework has a lot of capabilities.

09:57 And there's a lot of things.

09:58 There's the WPF.

10:00 There's Windows Forms for writing GUI apps.

10:03 There's network libraries that may or may not be available in Python as a module.

10:09 And so really the place that I use Python the most or Iron Python the most is creating a scripting environment for my applications.

10:18 When I write my tools, I generally develop in C Sharp and I use Windows Forms or WPF to create these tools.

10:24 And to provide a scripting interface for those tools, Iron Python is very good at embedding into a C Sharp application and accessing the components that you give it access to.

10:39 It's very easy to provide it access to like a text output window.

10:44 And so it can write to a text output window very easily that's developed in C Sharp.

10:49 I've tried doing similar things with CPython.

10:52 I've used Boost Python to try embedding Python into an application.

10:56 And I ran into some issues with wanting to rerun scripts over and over and over.

11:01 I got weird assertions and seg faults and stuff like that.

11:05 So for me, embedding Python in an application is very easy with Iron Python.

11:10 And that's my biggest use case as a software engineer using Iron Python.

11:15 Okay.

11:16 So I guess, yeah, you can embed CPython pretty easily, especially if you're doing a C-based application.

11:22 Then that probably makes a lot of sense.

11:24 But if you're already writing in C Sharp, this might be better because you can share information across it better, right?

11:32 Like your classes and libraries in one language are available to the other.

11:36 Does it go both ways?

11:37 Yeah, you can definitely access Iron Python objects and stuff from C Sharp.

11:42 One of the tools I actually developed was we have a hardware team that they develop boards with FPGAs and stuff on there.

11:51 And they wanted a way to test their hardware platforms.

11:54 And so I created a GUI that interfaces with Iron Python.

11:58 And they write scripts to test these boards.

12:01 And I use the unit test module.

12:02 And so they write their tests in the standard unit test format.

12:05 But I can parse those objects and find all the unit tests from my C Sharp GUI and present those.

12:12 And they can select which ones they want to run and run those tests to test the hardware boards out.

12:17 Oh, that's nice.

12:17 So you, like, load the Python modules and inspect what tests are there and then present some kind of custom runner on top of it?

12:24 Exactly.

12:25 Yep.

12:25 Okay.

12:25 Oh, that sounds really cool.

12:27 So we were talking about C Sharp.

12:29 And a lot of people say, you know, C Sharp means Windows only.

12:34 And a lot of people doing Python are not interested in Windows.

12:37 Does it run outside of Windows or is that required?

12:40 Yeah.

12:40 So we do have pretty basic support for running on Linux under Mono right now.

12:46 We definitely want to improve that because, you know, we want to be everywhere that people want to use C Sharp and Python.

12:53 So whether that's Linux or, you know, OSX or Windows, we really want to provide a solution to everybody.

13:01 So we're going to put some focus on improving our support for Mono as well as .NET Core, which runs on Linux right now.

13:10 Yeah, yeah.

13:10 That's really interesting.

13:11 I think that that's going to unlock a lot of things like this that used to be tethered more or less to Windows to set them free to run on, you know, Linux machines in a data center or something like that, which definitely opens up the appeal, I think.

13:25 What about performance?

13:27 Like, is there a performance benefit doing this versus CPython?

13:33 Are there performance penalties, but you get the integration?

13:37 What's the story?

13:38 So one, I think, so performance benefits, we do outperform CPython in several benchmarks.

13:43 I don't have the details right now.

13:46 Recently, someone ran that.

13:47 They compared CPython, PyPy, and IronPython.

13:50 And we outperform CPython in many cases, but PyPy definitely outperforms us right now.

13:56 I think that really probably is related to the global interpreter lock because we don't have that in IronPython.

14:03 It can help us in a lot of those benchmarks.

14:06 One of the things that we do have, something that we definitely need to look into optimization and performance problems is startup time.

14:12 It takes us a lot longer than CPython to get started, you know, actually executing the Python code.

14:18 And so that's something that we're planning on looking at.

14:20 Sure.

14:20 Yeah.

14:21 If you're just going to run a little bit of code and then shut back down, maybe it doesn't make so much sense.

14:25 I think PyPy also has that challenge that it takes a lot longer to get rolling at a high speed.

14:31 But once it's up in a sort of like a web app type of environment where it starts and runs for a long time, then it's better.

14:38 Yeah, definitely.

14:39 Okay, great.

14:40 Now, one of the things I thought was really intriguing about IronPython, and you brought it up, is the lack of the GIL or the global interpreter lock.

14:48 And that seems to be one of the challenges for doing real parallel, computational parallelism in Python.

14:55 You can do it and you can use the, you know, multi-process support and things like that.

15:00 But, you know, straight up threading when you're just doing computational stuff is challenging, right?

15:05 Is that more doable in IronPython?

15:08 Yeah, I would say so.

15:09 Because we don't have that, the GIL in IronPython, when we have something that can be spread across multiple cores, which .NET supports very well, you can really take advantage of multiple cores and not running into that, the GIL, which can slow you down.

15:23 Okay, so people are out there and maybe they have some project and they're trying to use parallelism.

15:28 How hard would it be for them to switch from running, you know, CPython to try out IronPython with their code?

15:35 Like, what's the chances of them taking what they have and running it with a different interpreter and having it work?

15:41 It depends quite a bit on what modules they're using.

15:44 Because we're running as a .NET application, we really can't use C modules, C-based modules.

15:52 So if you have something that's based on a C module, it's hard for us to interact with that right now.

15:58 Now, we do have a side project that had started a while ago when we were working on 2.6, Python 2.6 compatibility called Ironclad, which allowed us to interact with C modules.

16:11 It was kind of abandoned for a while, and now we're going to try and bring it back so that we can reuse some of those C modules that CPython provides so that more things are compatible under IronPython.

16:22 Okay, yeah, I mean, that seems really important to me.

16:26 I think one of the lessons from PyPy is it's really great, but it had these huge challenges around some of the popular packages that relied heavily on C, like NumPy, for example.

16:37 And they went so far as to try to implement their own version of NumPy in PyPy or in Python, in pure Python.

16:44 And that just seems like such a hard battle to, like, look at all the open source projects that are successful out there and go, let's just reimplement all of those.

16:52 And instead have to date, right?

16:55 Yeah.

16:55 It seems almost impossible.

16:56 Yeah, it's hard to play catch up.

16:58 It feels like you're always playing catch up, and that's a hard position to be in.

17:01 For one thing, you're trying to maintain compatibility with the language itself, which can change slightly from rev to rev, even on just the 2.7 baseline.

17:10 But then you have the open source projects that provide modules that are native or written in C, and those can be difficult to keep up with as well.

17:19 So it's definitely a trial that we're trying to work on and figure out.

17:23 Yeah, it sounds like Ironclad would be really important, or something filling that role at least.

17:27 How do you end up becoming one of the maintainers?

17:30 And you have a co-maintainer as well, right?

17:33 Yeah.

17:33 So my co-maintainer is Benedict Eggers.

17:36 He's located in Germany.

17:37 He was kind of what really kind of kicked off the resurgence of IronPython, I would say.

17:44 Ever since Microsoft open sourced it, I had been working, doing some stuff, because I was using it in my job.

17:51 So I wanted it to be a good piece of software.

17:54 And Jeff Hardy was the maintainer at the time.

17:58 And he's gotten, as with all of us, we go through hills and valleys of how busy we are with our jobs and family and stuff.

18:06 And he just reached a point where his job and his family were taking up a lot of time, as they should.

18:12 And he just didn't have the time that he thought he could put into helping IronPython be successful.

18:17 And so Benedict kind of pushed some buttons using some Twitter conversations between Jeff and myself and Benedict to kind of push.

18:27 And Jeff finally realized that he could pass it off to myself and Benedict.

18:32 Myself, because I had some history with the project and so forth, and Benedict, because he really had some excitement about it to get it going again.

18:41 So that's kind of how I got in there.

18:43 And Benedict is good at making people excited about things.

18:47 And so I got excited about IronPython again.

18:49 How nice.

18:50 We started making plans for the future.

18:52 Yeah, yeah, that's cool.

18:53 And it's interesting that was over Twitter.

18:55 I feel like people outside the development space don't really appreciate how special Twitter is in connecting you to people you would otherwise not be connected to.

19:07 Yeah, I mean, you can go to a site for a project and see like an email address for a developer, but you don't know, is it current?

19:13 Are they using that email address?

19:16 But Twitter, it's so easy to connect real fast.

19:18 Yeah, and the messages are small enough, you're likely to get a response.

19:21 If you write some 500-word email, people are just going to ignore you most of the time if they're busy or whatever.

19:27 But yeah, so I think it's really cool that that started out over Twitter.

19:31 This portion of Talk Python is brought to you by Hired.

19:45 Hired is the platform for top Python developer jobs.

19:48 Create your profile and instantly get access to 3,500 companies who will work to compete with you.

19:53 Take it from one of Hired's users who recently got a job and said, I had my first offer on Thursday after going live on Monday and I ended up getting eight offers in total.

20:01 I've worked with recruiters in the past, but they've always been pretty hit and miss.

20:05 I tried LinkedIn, but I found Hired to be the best.

20:07 I really like knowing the salary up front.

20:09 Privacy was also a huge seller for me.

20:12 Sounds awesome, doesn't it?

20:13 Well, wait until you hear about the sign-in bonus.

20:16 Everyone who accepts a job from Hired gets $1,000 sign-in bonus.

20:19 And as Talk Python listeners, it gets way sweeter.

20:22 Use the link Hired.com slash Talk Python to Me and Hired will double the signing bonus to $2,000.

20:27 Opportunity's knocking.

20:29 Visit Hired.com slash Talk Python to Me and answer the door.

20:32 So do you want to talk a little bit about the internals, about how this works?

20:44 I know how it works in CPython.

20:46 We have Python bytecode and we have this eval loop that will go through and pull out this bytecode.

20:53 Instead of a JIT compiling it, it will just interpret it and piece it all together.

20:57 What happens in IronPython?

20:59 How is it the same or different?

21:00 Sure.

21:01 It's kind of a similar concept.

21:04 We parse the Python code into an abstract syntax tree.

21:08 And then we kind of do some optimization or, yeah, you can call it optimization.

21:14 So when something is executed, we generate what's called a call site for that operation.

21:20 So that in the future, if we execute the same code or similar code, we have an optimized version of that code that we can then call.

21:28 So we basically take the Python and we convert that into the abstract syntax tree.

21:33 And then that gets converted into like a Lambda, basically, in the .NET world.

21:38 Something that we can execute.

21:40 And we cache off the compiled or optimized version of that call so that if we get another get adder call,

21:48 for instance, we have an optimized version of that that we can call and we don't have to recompile it into the Lambda format from the abstract syntax tree.

21:55 Okay. That's interesting.

21:56 So every instruction when I run Python code in IronPython is JIT compiled eventually?

22:04 Yes. Because it gets converted into a .NET IL, which is the intermediate language,

22:10 the JIT will run on that code when it's executed and can get cached off.

22:15 All right. That's a really interesting difference than, say, PyPy.

22:18 Because when I talked to the PyPy guys, they said it initially runs through CPython,

22:24 but if it runs enough, they're like, all right, this is the hotspot.

22:27 We're going to JIT this part.

22:29 But it doesn't always JIT, which is interesting.

22:32 I think it makes it a little less predictable, perhaps, in the way that PyPy works.

22:37 So that's pretty cool.

22:38 So I guess that means things like garbage collection apply as well, right?

22:43 The standard garbage collection that you would know from there.

22:47 Yeah, definitely.

22:48 So the .NET framework includes garbage collection.

22:50 And so we don't, unlike the CPython, which has to do reference counting on objects in the internals,

22:56 we don't have to do that.

22:58 It's all done automatically for us.

23:00 And the GC takes care of anything that's not around anymore.

23:03 Well, that definitely makes it easier on you as an implementer.

23:06 What are the benefits or drawbacks of that versus, say, reference counting in CPython?

23:11 In CPython, it's deterministic because you're telling the runtime when something is no longer needed.

23:18 And so it can clean it up right then.

23:20 In running under .NET with IronPython, the garbage collection is non-deterministic.

23:26 So it can run even during a critical point in your app and slow things down slightly.

23:32 Now, because we're running on .NET, we can have multi-core stuff.

23:36 So the garbage collector may be running on one core.

23:39 And so you still have, like, three cores out of four that can actually execute code.

23:44 But there is a non-deterministic garbage collector.

23:47 And Jython would have the same issue.

23:49 Java runtime garbage collection is also non-deterministic.

23:52 So it's a similar issue or problem between the two managed frameworks.

23:58 Interesting.

23:58 Yeah, I remember playing around with some of those ideas and checking out the performance,

24:03 realizing that allocation was actually faster a lot of the times in some of the GC spaces.

24:09 But you paid for it later when it had to be cleaned up because that was way more complicated

24:13 because the compacting GCs and simplifying.

24:16 You don't have to, like, find an empty slot in the free space.

24:20 Those usually get compacted.

24:22 Okay, that's pretty cool.

24:23 So you brought up some of the different types of apps that you can build with IronPython.

24:29 Want to give us a bit of a rundown on that?

24:31 So you said you could build WPF apps, which is sort of the standard Windows UI framework these days.

24:37 What else?

24:38 So we actually have a module, a WPF module.

24:41 So you can import WPF, and it provides kind of higher-level ways of interacting with WPF that are more Python-like.

24:49 So it makes it easier to develop those sort of things.

24:52 Benedict actually does quite a bit with WPF on IronPython.

24:56 We also have, you can write Windows Forms, which is kind of the older format of Windows applications.

25:03 WPF is supposed to replace Windows Forms, but it hasn't happened yet.

25:08 People still write Windows Forms apps.

25:10 You can write Windows Forms apps under IronPython.

25:13 Now, you have to start them differently than you would a, like a command line application.

25:19 On Windows, when you're running a Windows application, it has to have an event loop in order for it to process Windows events and things like that.

25:28 The normal command line client doesn't have that message loop, so it can't run Windows Forms apps or WPF apps.

25:37 So we actually have a separate interpreter executable, which allows or has an event loop started up so that your forms applications and stuff work correctly.

25:48 Oh, that's interesting.

25:49 Yeah, because it has to plug in basically to that already running event loop or something like that, right?

25:54 Exactly.

25:55 Yep.

25:56 Okay.

25:56 And can you build, what do you distribute, I guess?

26:00 Like, do people have to have IronPython installed in the system?

26:05 Can you give them, like, an executable that's self-contained?

26:09 What thing do you give to people to run?

26:11 We do have what's called the IronPython compiler, which is basically what it does is takes any .py files that you want,

26:19 and you can actually also include the Python standard library, and you can build a standalone executable that is basically a, has the Python code as a resource embedded inside,

26:32 and then some wrapper code to pull out those resources and execute it as if you were running it from the command line using the IPy interpreter.

26:40 So you can create pretty standalone executables.

26:44 They are pretty large because you're embedding everything inside, but you can distribute a single executable that has all of the data inside for running your application.

26:54 Oh, that's pretty awesome.

26:54 Does that take, like, external packages?

26:57 Like, if I'm using requests or passlib or some of these things, will it get sucked in and become part of my project?

27:04 Right now, we don't have automatic dependency checking for that, so you would have to specify the path to the .py file on your system.

27:14 But that is something we're looking into.

27:16 I recently created a patch for Module Finder for IronPython because we don't have bytecode,

27:22 and Module Finder uses bytecode, the Python bytecode, to determine what's being imported.

27:27 But I created a patch that uses the AST module to find all the imports within your project as well.

27:34 And that's something I'm going to roll into the IronPython compiler so that it can automatically find the dependencies that you're using and include them.

27:42 Okay. What's the timeline on that?

27:43 Probably in the next major release of the 2.7 branch that we're working on.

27:49 We are working on IronPython 3, which would be 3.x support, but we're still maintaining the 2.7 release track as well.

27:58 And so probably the next major release.

27:59 We're going to do another minor release here pretty soon because we had some regressions.

28:04 It was our first release as project owners, and we were human, so we made a few mistakes.

28:08 But we're working to fix those and get out kind of a minor release.

28:13 But I think in the next major release, we can include that type of support.

28:18 So what's the story with Python 3 versus Python 2 since you brought it up?

28:22 Sure. So in .NET, all strings are Unicode.

28:26 So we've had some Python 3-esque support for a while now because of that.

28:31 It also caused us problems with some of the 2.7 modules, but that's kind of a different story.

28:36 But we are working on implementing all of the features for the most recent 3.x series.

28:43 It's in a separate code repository from the IronPython 2.

28:47 Because we're going to make some fairly significant changes to the parsing logic for the new language features and things like that.

28:56 But it also allows, you know, a lot of those Unicode issues will go away with IronPython 3, which, because Python 3 switched to Unicode as well.

29:05 Sure. So is that something I can use today or is it still being worked on?

29:10 It's still being worked on. We don't have all the language features in yet.

29:13 We're working on trying to get out a first release in the end of October that has language features implemented.

29:21 And then we're going to work on fixing bugs and making sure all the tests are passing.

29:26 And we're really trying to push standard library features upstream so that you can use the normal Python standard library.

29:35 Right now we have some changes in the standard library and have a local version of it.

29:39 So you can't just go and grab the standard library of CPython and use it with IronPython and have it work.

29:44 But we want to push those changes upstream so that you can use the normal CPython standard library with IronPython 3.

29:52 Okay. Yeah. It seems like a great plan.

29:54 You touched on threading. And one thing I did want to ask you about, since we're talking about Python 3,

29:59 is there's some coroutine parallelism features in Python 3 with the async and await keywords.

30:06 And C Sharp also has async and await keywords.

30:10 So is there some special tie-in, like does async and await do anything extra cool on IronPython versus regular Python 3?

30:20 The async and await stuff is something we haven't implemented yet, but I think it's going to be fairly easy because C Sharp and .NET have the async and await already built in.

30:30 But that's something we haven't implemented yet in the IronPython 3 yet.

30:34 Okay. And speaking of implementation, we said that CPython is not mostly Python.

30:39 Mostly it's C code in the actual low-level parts, right?

30:43 So I had some of the standard library. But what is IronPython written in?

30:47 IronPython is written in C Sharp.

30:49 And so the native modules or some of the built-in modules are implemented in C Sharp.

30:54 We replicate the functionality of the native C modules, things like underscore hashlib, underscore SSL,

31:03 things that are imported in the Python module like SSL.py or hashlib.py.

31:09 We implement those and try and create them as feature-complete as possible in C Sharp.

31:14 Okay. And how hard is it to match those things up? Do you have like tons of tests? Do you run the standard Python tests to verify your code or anything like that?

31:23 Yeah, we definitely use all the tests that are already there for the CPython in the standard library.

31:28 We use those heavily. In addition, we have some features that we test for like COM, which is a Windows technology, the common object model.

31:37 We have some stuff that makes it easier to work with COM objects. And so we have some tests that do that.

31:42 We have tests for interaction between C Sharp and IronPython to make sure that if you're hosting IronPython in your application, it continues to run well.

31:53 So we do use the CPython tests and then we add additional tests on top of that.

31:58 Right, because you're trying to add additional functionality, I guess, in addition to what they're providing.

32:01 Okay. So also you talked about Windows apps. Can you build web apps, which is maybe where Python is used more?

32:09 Yeah. So Benedict has been working on something that we recently imported into the Iron Languages repository or organization on GitHub, which is IronPython ASP.NET MVC.

32:20 And basically what it does is allows you to write ASP.NET MVC apps, web apps in IronPython or in Python.

32:28 So you can define your routes and that sort of thing in Python. So it's a really exciting thing.

32:34 And that's one of the things we really want to get running under .NET Core as well.

32:37 Yeah, that sounds awesome. If you could get that running on .NET Core so I could develop that on my Mac and deploy it on Linux, that's pretty interesting.

32:46 Yeah, and you still get the power of some of the .NET libraries that may not have a Python module to do.

32:53 Sure. Okay. Well, yeah, that sounds pretty exciting. I'm looking forward to that.

32:56 What about managing local environments?

33:00 CPython has great support for virtual environments, especially in Python 3. It's got VEMD built in.

33:06 Do you guys have this concept of virtual environments with what you built?

33:10 We had some patches upstream to virtual ENV because there are some things that we were having some issues with.

33:17 They requested us to make some changes and all that kind of happened back when development was really kind of dying out on IronPython.

33:26 So it's something we need to resurrect and re-look at, but it's definitely something we want to support because people love that as part of CPython.

33:33 So we'd really like to provide that as part of IronPython as well.

33:37 Yeah, yeah. It's definitely pretty important. Okay, cool.

33:40 And then you also have something called PyLauncher.

33:42 How does that relate? Is that like the thing that will sort of associate with Python files and run them, or what's the story of that?

33:49 It is a launcher. I don't know if you're familiar with Visual Studio at all.

33:53 Visual Studio has a bunch of different formats for their project files, and so they have what's called a Visual Studio Launcher.

33:59 Visual Studio Launcher will get the format of the file that it's received and determine which version of Visual Studio that you have on your system you should open it with.

34:08 And that's kind of what PyLauncher does as well.

34:10 It allows you to have multiple different Python versions on your system and launch a different one based on different things within the files.

34:19 Right, okay.

34:19 So you mentioned Visual Studio, and Visual Studio is fine if you're on Windows, but if you're not, then you don't really have it.

34:27 And I'm also a fan of PyCharm, as people listening to the show know.

34:30 Could I, say, open PyCharm and write IronPython apps with it?

34:34 Yeah, I mean, we do run on Mona right now.

34:38 So, which we'll install on OSX or on Linux.

34:41 And so, you know, some people I know who are working on IronPython itself use like Emacs or Vim or, you know, their favorite editor.

34:51 And they write the Python and then they execute it from the command line using the ipy.exe.

34:56 So if you're developing the Python itself, you can write it anywhere and run it under the interpreter.

35:03 Now, there is an awesome tool from Microsoft called the Python Tools for Visual Studio, which is basically a free Python ID if you're running on Windows.

35:12 They do an awesome job of a lot of stuff.

35:15 And that kind of started out of IronPython initially.

35:18 And so that's kind of why I like it.

35:20 But, yeah, you can use basically any ID you want to write the Python.

35:23 And then you just execute it under the IronPython interpreter.

35:27 All right.

35:27 Yeah, that sounds really cool.

35:29 Yeah, I remember the IronPython tools for Visual Studio just became the Python tools for Visual Studio at some point, right?

35:36 Yeah, and so they work with IronPython.

35:38 They work with CPython.

35:39 I don't know if it works with PyPy.

35:41 I've never tried that.

35:43 But as I've created this tools here at work that use IronPython, I've had to recommend, you know, GUIs for people who are, you know, hardware engineers and stuff like that to write Python.

35:53 They can get syntax highlighting, IntelliSense type stuff or auto-completion stuff of methods and parameters, documentation.

36:01 So, yeah, Python tools is very good for that.

36:04 Most people here run Windows.

36:06 So, it's easy for me to recommend Windows tools to them.

36:08 Yeah, it's an easy sell if it already runs there.

36:10 We all want to ship our software faster, respond to user requests quicker, and build an edge on the competition.

36:30 But the faster you go, the more likely a bug or issue will slip by and cause your users grief.

36:35 I have the same trade-offs with my Talk Python websites and infrastructure.

36:38 That's why I use SnapCI.

36:40 Every time I check in at GitHub, SnapCI springs into action.

36:44 A clean copy of code is pulled from GitHub.

36:46 A Python 3 virtual environment is created.

36:48 All the packages and dependencies are installed.

36:50 The unit tests are run.

36:52 Finally, I'm notified in Slack and a menu bar on my Mac about the outcome.

36:55 That way, I know it's safe to ship a shiny new version of my web app.

36:59 Ship your software faster with a 30-day free trial at snap.ci slash Talk Python.

37:13 Yeah, there's a whole, let's say, vibrant debate that we could start about whether you should use IDEs or you should use something minimalistic like Emacs or Sublime, relatively speaking.

37:25 Maybe we shouldn't really go into it.

37:27 I kind of lean a little more towards the ID side myself on the PyCharm space.

37:31 And I think the tools, if you are on Windows, I think the Visual Studio tools are pretty sweet as well.

37:37 I've played with them a little.

37:38 So one thing that I thought was interesting that I saw on the mailing list going around was about the time that you took over, I think,

37:46 there was a meeting that said there's a bunch of urgent issues and we're going to sort of reset the direction of IronPython.

37:53 Yeah, when was that?

37:54 So we had our first community meeting a couple months ago.

37:58 We've been trying to have them every two weeks on Thursday.

38:01 And we have them on Gitter.

38:03 I don't know if you're familiar with Gitter.

38:04 It basically allows you to use your GitHub credentials and organizations to have meeting rooms in a web page.

38:12 Yeah, and they're generally like around a project on GitHub, right?

38:15 Exactly, yeah.

38:16 So we have one for our organization, which is the Iron Languages organization.

38:19 Sure, so tell people what the GitHub URL is.

38:21 Of course, I'll put it in the show notes.

38:23 Sure, so we're at github.com forward slash iron languages forward slash main is currently the main repository for the Python 2.7 branch.

38:36 Then there's github.com forward slash iron languages forward slash iron Python 3 for the work we're doing for Python 3 support.

38:44 All right, excellent, yeah.

38:46 Cool, I'll definitely link to those.

38:48 So the meeting, the first meeting, some of the issues that people brought up were this idea of Iron Ruby.

38:54 What is it doing here?

38:55 Does it make sense in Iron Python?

38:57 Do you want to talk about that a little?

38:59 Sure.

39:00 So when Microsoft open source the code, they had two languages are basically implemented on top of what's called the dynamic language runtime, which is kind of a set of classes or objects that make writing dynamic languages on top of .NET a little easier.

39:15 It does all that stuff about the call sites that I talked about, caching things and converting an abstract syntax tree into the lambdas for calling the code and so forth.

39:26 So they had Iron Python, which is an implementation of Python, and they had Iron Ruby, which was an implementation of the Ruby language on top of the DLR.

39:35 So what we kind of did is we split out Iron Ruby into its own repository under the organization.

39:41 And I submitted a message to the Ruby mailing list, say if there's anybody interested in taking up Iron Ruby and keeping it alive.

39:51 And so there were a couple of people who responded.

39:53 And, you know, we're willing to help them in any way we can.

39:57 But we really kind of just split it out so that they could lead their own destiny wherever they want to take it.

40:02 Yeah, that makes a lot of sense to me because while there's some similarities, there's a lot of differences between those two communities and languages.

40:09 And I feel, I mean, maybe correct me if I'm wrong, but I feel like the mission of Iron Python these days is no longer to say we're going to be this cool thing that shows off the ability to do dynamic languages on .NET.

40:23 It's all about making the Python as an alternate implementation special, right?

40:29 And so then that makes Iron Ruby being tied up in it make a lot less sense, right?

40:33 Yeah, and it really, we don't want to tie them down either with the decisions that we make going forward.

40:39 We don't want to impact them in a negative way if they want to keep it alive and keep it going.

40:43 So allowing them to have their own space is, I think, a good way to let them do what they need to do and let us do what we need to do to make both of our languages the best we can.

40:55 Yeah, excellent.

40:56 Okay, I'd say that's a good choice.

40:59 And then when you moved over from where it used to be hosted to GitHub, there was something like 960 issues that were just sort of stale out there.

41:08 And you're like, what do we do with these?

41:10 Did you end up throwing them all away?

41:12 Did you keep them?

41:13 Like, how do you manage that transition?

41:15 So, and all those issues were originally filed on CodePlex, which was Microsoft's open source competitor, if you will, to GitHub.

41:24 I actually wrote a Python script that ported them, did a screen scrape using Beautiful Soup on CodePlex because they didn't have an API to do this.

41:31 And then importing them into GitHub using one of the GitHub Python modules.

41:35 Nice.

41:36 That was a lot of fun.

41:37 Yeah, who says they don't have an API?

41:38 They do, they just don't know it.

41:40 That's right.

41:40 So anyway, there were these 960-odd issues.

41:44 Some of them were as old as 2006 or 2007, pretty old.

41:52 And so what we decided to do in the meeting is to go through and we marked them all as what we called untriaged,

41:59 where we said we just basically didn't know what was going on, what needed to happen, or anything like that.

42:05 And we have about 10 people who were in the community meeting that said that they would go through issues and try and triage whether they're a valid issue still,

42:14 whether it's something that doesn't exist anymore so we can just close it out, or things along those lines.

42:19 And so we've been working on that.

42:21 I think we're down to about 800.

42:22 So we haven't made a huge amount of progress, but we are making progress on triaging the issues

42:28 and seeing, you know, we need to fix these issues.

42:30 Some of them that we've triaged, we've fixed as well.

42:33 So some of them were very simple fixes to make.

42:36 We just had to know that we're still an issue and then go in and fix it.

42:40 But we're slowly making our way through those and porting things from the untriage state to either closed or,

42:48 yes, this is still an issue, to I don't know what we do with this.

42:52 Let's talk about this one further.

42:54 Sure.

42:54 I think that highlights one of the real big challenges, the unglamorous side of running a large open source project, right?

43:03 Like, probably if you could have your dream, whatever you want to do technically for your time, what do you want to dream about?

43:10 It's probably not triaging 900 old issues that might not even be relevant, right?

43:15 But it's important work.

43:17 Yeah.

43:18 It can be very satisfying to see that count go down.

43:21 Yes, I'm sure.

43:22 You know, it's been sitting around this 906 time.

43:25 And so seeing it go down is very satisfying.

43:28 Sure.

43:28 Well, when I look at open source projects, you know, on GitHub or whatever, that's one of the indications of whether or not I want to trust this thing still being around, right?

43:38 How old are the issues?

43:40 Are they getting responded to?

43:41 Also, pull requests.

43:42 Like, you'll see projects that have year-old pull requests, which seems just so sad because somebody went to the extra effort of fixing code for you,

43:50 and you didn't bother to pull it in or even respond, you know?

43:53 Yeah.

43:54 I agree completely.

43:54 Yeah, it's tough.

43:56 You also talked about release schedules.

43:57 What's the story there?

43:58 How often?

43:59 What versions?

44:00 Two, three, so on?

44:02 Sure.

44:02 Well, we're going from having like one release every four years to we'd really like to get on a more regular release schedule.

44:11 So we did three beta releases or RC releases, release candidate releases, before we did the most recent 2.7.6.3 release.

44:22 That was our final or our most recent final release or stable release.

44:27 And so we really want to do that, you know, every couple months, do a full release with the fixes and stuff so people know that we're actively working on it and we're actively fixing issues.

44:37 Yeah, okay, cool.

44:38 Every couple months, we'd like to do a final release or, you know, either a minor release or a major release.

44:43 Yeah, that's way better than every four years.

44:45 Yeah.

44:45 Yeah.

44:46 Nice.

44:48 All right.

44:48 Then you also talked about continuous integration, IronPython 3, which we touched on a little bit, and then IronCloud or the ability to support C extensions.

44:57 And I definitely want to just put my vote in to say, from what I've seen, the challenges that PyPy ran into, if you can support C extensions in a super clear, automatic way, that could seriously unlock the adoption.

45:12 Yeah.

45:12 Yeah, definitely.

45:13 It would be a huge benefit for people using IronPython.

45:16 And we wouldn't have to implement as many of the native modules in C Sharp or whatever.

45:22 Now, there may be cases where we want to because we can optimize it better without the GIL.

45:27 But because the native libraries would assume there's a GIL there.

45:33 So there are some things we'd probably still keep in C Sharp or implement in C Sharp because we would want to avoid the GIL.

45:39 Yeah, that makes sense.

45:39 But it would be nice to let that be a choice and not a necessity.

45:42 Definitely.

45:43 Definitely.

45:44 Yeah, yeah.

45:45 Awesome.

45:45 All right, Alex.

45:47 We're getting near the end of the show.

45:48 At the end of the show, I always ask people a couple of questions.

45:51 So let me put them to you now.

45:53 On PyPI, there's over 80,000 packages, so many out there that you can use and just, you know, so many people probably haven't even heard of.

46:03 Like, what's an amazing one that you'd like to sort of shine a light on?

46:06 It's not a super, like, glamorous module, I would say.

46:10 But my brother and I use Python with the PySerial module.

46:14 We actually wrote an application that allowed us to interact with our body bugs.

46:18 So I don't know if you're familiar with body bug.

46:20 Like, it's an armband that you wear that tracks different things, like your bolts, and it's got a step counter and stuff like that.

46:28 Oh, nice.

46:29 And you have to have a subscription on their website to interact with it.

46:33 And so we kind of looked around on the web and found some information on how to interact with it over the serial port.

46:39 And so we use PySerial to interact with our body bug and get the same data that the website would get.

46:44 Oh, that's awesome.

46:46 It's not a glamorous module, but it was a lot of fun to be able to reverse engineer and do stuff like that with our body bugs.

46:51 Sure.

46:52 No, but it's a cool story.

46:53 And, you know, you have these companies with weird restrictions and business models or whatever.

46:57 You're like, you know, that's my data.

46:58 It's on my arm.

46:59 Come on.

46:59 Exactly.

47:00 Yeah.

47:00 Yeah.

47:01 Nice.

47:01 I could probably guess what you're going to give me for the next question as an answer.

47:05 But what's your favorite editor when you write Python code?

47:08 Actually, I use Vim for most of my Pythons.

47:11 I know I said Python tools from Visual Studio is awesome, but a lot of times I'm doing some quick edits to the Python code or something like that.

47:18 So I got introduced to Vim years ago in college, and I use that quite a bit.

47:23 I've tried using SpaceMax, which has a Vim mode for Emacs.

47:29 But plain old vanilla Vim is kind of my go-to editor.

47:33 Oh, right on.

47:34 Very cool.

47:35 All right.

47:35 Well, any final call to action?

47:37 Are you looking for people to help out on the project, people to try it out?

47:40 What do you guys need?

47:41 Oh, absolutely.

47:41 We'd love to have anybody who wants to.

47:44 You don't even need to, you know, write C Sharp.

47:47 There's all sorts of documentation we need to figure out.

47:50 We do reuse a lot of the Python docs, but there are some things specific to Iron Python.

47:56 Testing things out, see if there's still issues.

47:59 We're still in that issue disposition time.

48:01 So anybody's on to GitHub and look at the issues and see if they can reproduce it or give us a little bit more information.

48:08 They'll kind of push us towards getting a fix faster.

48:10 So anybody who wants to get involved, we'd love to have them on there.

48:14 They can add on the mailing list, which we have through Python.org.

48:18 It's Iron Python users.

48:19 Yeah, we would welcome anybody who wants to help out.

48:22 That sounds great.

48:23 All right.

48:23 If it sounds interesting to you, get out there and check it out.

48:26 Alex, it's been fun to talk about Iron Python.

48:28 I'm excited to see this coming back to life.

48:30 It plays an interesting role in the ecosystem.

48:34 So nice work and keep it up.

48:36 Hey, thanks for having me on.

48:37 It's been a blast.

48:38 You bet.

48:38 Talk to you later.

48:40 This has been another episode of Talk Python to Me.

48:43 Today's guest was Alex Earl.

48:45 And this episode has been sponsored by Hired and SnapCI.

48:48 Thank you both for supporting the show.

48:50 Hired wants to help you find your next big thing.

48:53 Visit Hired.com slash Talk Python to Me to get five or more offers with salary and equity presented right up front.

48:59 And a special listener signing bonus of $2,000.

49:02 SnapCI is modern, continuous integration and delivery.

49:06 Build, test, and deploy your code directly from GitHub, all in your browser with debugging, Docker, and parallels included.

49:12 Try them for free at snap.ci slash Talk Python.

49:15 Are you or a colleague trying to learn Python?

49:18 Have you tried books and videos that just left you bored by covering topics point by point?

49:23 Well, check out my online course, Python Jumpstart by Building 10 Apps at talkpython.fm/course to experience a more engaging way to learn Python.

49:32 And if you're looking for something a little more advanced, try my WritePythonic code course at talkpython.fm/Pythonic.

49:39 You can find the links from this episode at talkpython.fm/episodes slash show slash 74.

49:46 Be sure to subscribe to the show.

49:49 Open your favorite podcatcher and search for Python.

49:51 We should be right at the top.

49:52 You can also find the iTunes feed at /itunes, Google Play feed at /play, and direct RSS feed at /rss on talkpython.fm.

50:01 Our theme music is Developers, Developers, Developers by Corey Smith, who goes by Smix.

50:06 Corey just recently started selling his tracks on iTunes, so I recommend you check it out at talkpython.fm/music.

50:13 You can browse his tracks he has for sale on iTunes and listen to the full-length version of the theme song.

50:19 This is your host, Michael Kennedy.

50:20 Thanks so much for listening.

50:22 I really appreciate it.

50:23 Smix, let's get out of here.

50:25 Outro Music.

50:46 *outro music*