#340: Time to JIT your Python with Pyjion? Transcript

00:00 Is Python slow.

00:01 We touched on that question with Guido and Mark in the last episode.

00:04 This time we welcome back friend of the show, Anthony Shaw.

00:07 He's here to share the massive amount of work that he's been doing to answer that question and speed up things where the answer is yes, and he just released version 1.0 of the Pyjion project, Pyjion is a drop in Git compiler for Python 310.

00:23 It uses the .NET 6 cross platform JIT to compile and optimize Python code on the fly with zero changes to your source code.

00:31 It runs on Linux.

00:32 Macos, and Windows, both X 64 and Arm 64.

00:36 It's a cool project and I'm excited.

00:38 Anthony is here to tell us all about it.

00:40 This is Talk Python to me.

00:41 Episode 340, recorded November 3.

00:44 2021.

00:58 Welcome to Talk Python to Me, a weekly podcast on Python.

01:01 This is your host, Michael Kennedy.

01:03 Follow me on Twitter where I'm @mkennedy and keep up with the show and listen to past episodes at 'Talkpython.FM' and follow the show on Twitter via at Talk Python.

01:12 We started streaming most of our episodes live on YouTube, subscribe to our YouTube channel over at Talkpython.FM.

01:19 Youtube to get notified about upcoming shows and be part of that episode.

01:24 This episode is brought to you by Shortcut and Linode and the transcripts are sponsored by 'assemblyAI' Anthony Shaw.

01:32 Welcome to Talk Python to Me.

01:34 Hi, Michael. Good to see you again.

01:35 Yeah. It's great to have you back your at least yearly appearance here, if not by name, just at least appear yourself once. But I think you get mentioned a bunch of times with all the stuff you're doing. Yes.

01:47 We're just trying to work out when it was the last time over a year ago.

01:51 Yeah, it was May 2020.

01:54 That was when the whole to thing was about to end. Like it's just a couple of months. We'll be through this. It'll be fine.

02:00 Everyone will wear their mask and get their shots and it'll be totally normal.

02:03 I had to pause for a second. I was trying to work out what year it was.

02:08 I know.

02:09 Well, you're in Australia, so it's probably the next year.

02:13 Yeah.

02:14 You guys are always ahead by a little bit. Yeah.

02:16 Awesome.

02:17 Welcome to the show.

02:19 This time we're here to talk about a project that you've been spearheading for the last year or so.

02:23 Pyjion, which is a JIT compiler for Python, which is pretty awesome.

02:29 Yeah.

02:30 It was sitting on the shelf for a few years and I decided to pick it up. It was related to a thread of things I've been working on, looking at Python performance and also when I kind of finished working on the book.

02:43 CPython internals book.

02:44 Yeah. CPython internals book.

02:46 Yeah. It was really interesting to dig into a compiler, and I really wanted to put some of that theory into practice and work on my own compiler. So.

02:55 Yeah, that's kind of what led to it.

02:57 It looks like no small feat, as people will see as we get into it. But how much did the diving into the C internals make you feel like, all right, I'm ready to actually start messing with this and messing with how it runs and compiling machine instructions.

03:11 Yeah. Massively.

03:11 I don't know where I would have started. Otherwise, it's a pretty steep learning curve.

03:15 Yeah. If someone gave me that job, I'd be like, I have no idea how to do this.

03:19 Yes.

03:19 It was one of those projects I got started on because it just had my curiosity.

03:23 I didn't understand how it could work, and I just really wanted to learn, and it seemed like a really big challenge. So yeah, I looked at it and thought, this is an interesting thing.

03:33 I know that Brett, and Dino were no longer working on it and just decided to pick it up and see how far I could take it.

03:38 Yes, absolutely.

03:40 I think you've taken pretty far, and it's about to go to 1.0. Is that right?

03:45 Yeah.

03:45 So it could be launching version 1.0 Pyjion in a few days.

03:51 So I'm waiting for .NET6 to be released, and I'll explain why, probably a bit later.

03:56 Wait, hold on.

03:58 This is a Python project on a Python podcast. And you're waiting for .NET6. Oh, my gosh. I don't understand.

04:05 Yes.

04:05 This bit confuses people.

04:06 And it did originally. I think when the project was written.

04:09 So it's a jit compiler for Python written in C++, and you could write your own Jit compiler, which is no small feat.

04:20 Yes. And then you'd be done in, like, ten years.

04:22 Exactly.

04:23 Or you can take a Jit compiler off the shelf.

04:25 So a jit compiler compiles some sort of intermediary language into machine code assembly.

04:33 A lot of jit compilers are working with basically, like, which registers to use which operations and instructions are supported on different CPUs.

04:42 There's a whole bunch of stuff to work very involved.

04:47 And .NETcore what was called Net Core now just .NET has a Jit compiler in it.

04:53 And yes, you can actually use just the Jit compiler.

04:57 So that's what the project originally did was to basically use the JIT compiler for Net Core.

05:04 Other jits for Python, some of them use the LLVM jit, and there's a few other jits as well. You can get off the shelf.

05:11 So I started to use the Net one that was originally Dino and Brett when they built this.

05:17 It was actually before .Net Core was even released. It was still in beta back then, but yeah, they used .Net Core Jit.

05:22 Yeah.

05:22 I think it was a much earlier version of Net Core back then, right. Like, it's come a long way.

05:28 I think back then you probably know this better than I do now.

05:32 But back then, I think there was like a fork in the road to pass for the .Net world. You could do traditional Net on Windows only.

05:41 But that was like the 15 year polished version of .Net.

05:45 And then there was this alternative, funky opensource .Netcore thing that sort of derived, but was not the same thing as that. And so I think probably when Brett and Dino we're working, it was really early days on that version of the JIT compiler.

06:00 Yeah, it was version 0.9 of Net core, right.

06:02 So yeah, I did actually get involved back then just helping out upgrading it from version 0.9. I've got a call to version one.

06:10 So I did, like, help on the original Pyjion project with some of the builds and stuff. But that early version of Pyjion required a fork of CPython and a fork of Net core, so it required you to compile both projects from sort with patches and stick a whole bunch of stuff together and very tricky to set up.

06:32 And that's one of the things when I kind of came in a year ago to pick this project up again that I really wanted to tackle was let's make it easy to install this, which means it should just be pip installable, so you can just pip install Pyjion right on Python.

06:46 And that's the second thing I had to do was to upgrade it to the latest versions of Python, the latest versions of Net.

06:52 So yeah, it's running on Net6 and CPython 3.10

06:58 It's basically a check compiler for Python 3.10.

07:01 Right.

07:02 So now we have Net.

07:04 I think they've closed that fork because it's a little bit like the two to three boundary that we've crossed. It's just back to this Net thing, and it's open source, which is cool.

07:13 So that closes the .Net jit side of things from a beta thing.

07:18 On the other hand, there's a Pep, and I'm sure you know the number. I don't know the number off the top of my head that allowed for extensions into standard CPython, so you don't have to fork it and reprogram CFLC.

07:33 There's an extensible and extensibility layer for this kind of stuff now, right? Yes.

07:38 Maybe to backtrack a little bit. But when you write Python code and then execute it in CPython, which is the most popular Python interpreter, the one you get from Python.org

07:47 When you compile the you don't compile the Python code.

07:50 Python does it for you.

07:52 When you compile Python code, it compiles down into an abstract syntax tree, and then the next level down is bytecode sequence, which you can see if you import the disk module and you run the disk function on your code.

08:08 I've got to talk about Pyjion, which I did in PyCon this year.

08:12 We'll link that in the show notes.

08:14 It gives some explanations, an example.

08:16 So that's kind of the bite code.

08:18 And basically this looks at how can the bytecode gets handed off onto an evaluation loop in Python, which is called .

08:29 Is that one of the first things you looked at when you started diving into the C source code?

08:34 Is that the first file you went to?

08:36 It was one of yeah, it's such a big one.

08:38 I remember on Python Bytes last week, you mentioned that Lucas had been analyzing bits of Python that changed the most, and that was like, the most changed bit of Python. It's kind of like the brain of Python. Really.

08:50 It's a loop that evaluates all the instructions and then calls all the different CAPIs to actually make your code do things.

08:58 And what you can do in this Pep, which Brett proposed originally when he was working on Pyjion, was that you can actually tell Cpython to not use its own evaluation loop, but use a replacement one. So this 523 it is you can basically write an extension module for Python and then say, okay, from now on, I will evaluate all Python code.

09:23 Well, Python compiles it for you and then just give it to you as a bytecode code object with bytecodes in it, and then you can write custom function, which will evaluate those byte codes.

09:34 Right. Exactly.

09:35 Normally, there's just a switch statement that says, if I get this byte code, here's what I do. If I get that by code, here's what I do.

09:41 One of the drawbacks of that that makes it super hard to optimize, among other things. Is it's one statement at a time, right.

09:49 Like the Ceval that switch statement, that loop doesn't go here's a series of possibly related opcodes make that happen. It goes, no, you need to load this variable now create this object, and there's just not a lot of room for optimization there.

10:07 You're not going to inline a function or do other types of things when it's instruction by instruction.

10:12 Yeah.

10:12 Exactly.

10:13 So what Pyjion does, essentially, is it implements that API.

10:18 When you install Pyjion and activate Pyjion, which you do by importing Pyjion and then just doing Pyjion enable, it will tell CPython that Pyjion will be the function to evaluate all Python code from now on.

10:31 And when it sees a new function for the first time, instead of interpreting all the bytecode instructions, when you execute the function, it will basically interpret those ahead of time and then compile them into machine code instructions, and then store that machine code in memory and then reexecute it every time you run the function.

10:52 So it basically compiles the function down into assembly, essentially.

10:59 And then puts the assembly object in memory.

11:02 And then when you run the Python function, if it's already been compiled, it will then just run those instructions, right.

11:08 As standard jit stuff. It has to do at once.

11:10 But then once it's hit one method, it's like, okay, this one here's the machine instructions for it. We're just going to reuse that, right.

11:17 Yeah. Because the computer needs machine code to do anything.

11:22 So something has to have compiled it down into machine code. And in the case of normal CPython, C, Python is written in C, and the C compiler has compiled that down into machine code, but it's a loop that it runs through for each bitecode instruction to go. Okay, this is an ad operator, so I'm going to take two objects off the stack, the left hand side on the right hand side, and then I'm going to call the add function in the CAPI.

11:49 Exactly.

11:50 I kind of got you diving down deep a little too quick. I do want to set the stage just a moment before we get into how all of this works, because understanding where you're coming from and understanding some of the problems you're trying to solve are really going to be helpful to see the value here. So back in April 2020, whenever PyCon was that virtual, the first virtual PyCon you had that talk called Why Python is Slow. And you talked about some interesting things that really set the stage for.

12:20 Well, if you had a jit and you had all sorts of control over it as you do, how could it be faster? What could we do?

12:26 Right.

12:26 So one of the things you talked about was this in body problem.

12:29 How C++ relative, say, Python?

12:33 There's a bit of a difference there, but also Net core was a lot faster and really importantly, JavaScript was faster, right?

12:38 Yeah.

12:38 So that was 2019.

12:42 My gosh.

12:43 Okay.

12:44 This is part of this lovely pandemic.

12:46 See, I covered the N-body problem, which is interesting because the N-body problem is a mathematical formula that calculates the position of the Jovial planet.

12:59 As soon as you have three or more, it starts to get super complicated, right?

13:02 Yeah.

13:03 It's basically like a big mathematical formula, and it just loops through the iterations to work out the position of different planets. So it's kind of the difference between C is 7 seconds it takes to run the algorithm in C and 14 minutes it takes to run it in Python. Python is even slower than Perl, which is embarrassing.

13:22 Yeah. It's actually pretty much the worst case scenario for all the reasonable languages.

13:28 Yeah.

13:28 In that talk, I dug into the details about why some of the reasons why that is and kind of the core of the N- body algorithm is this.

13:37 There's a few lines of code which basically look at floating point numbers, and it calculates does a big calculation.

13:43 There's a lot of mathematical operations there's, like minus divide power add, which is great and can all be done in line.

13:52 Cpus are very efficient at doing this because CPUs natively understand floatingpoint numbers.

13:58 Yes.

13:58 But a number in C and a number in Python. These are not equivalent. Right. A floating point number in C is probably eight bytes on the stack.

14:06 A floating point number in Python is what is that? A Py float object that's 50 bites and is out on the heat, probably separated in space from the other numbers in terms of like, it will cause more cache misses and flushes and all sorts of stuff. Right? Yeah.

14:21 A floating point number in Python as an immutable object, and it's basically a wrapper around a double.

14:30 So basically you have to create a Python object to store the floating point number, and then if the value changes, you have to create a new one.

14:38 So the issue in N-Body is that you have to create for one line of Python that just does a whole bunch of work to get a single answer.

14:47 Like a single floating point number.

14:49 All the interim values in that calculation create like 18 objects, which are immediately discarded.

14:56 Right. So the memory management kicks in just constantly.

15:00 Yeah.

15:00 And Python is pretty efficient at allocating small objects.

15:04 But when you magnify that to the level that is seen in the N-Body problem, then yeah. That's why it's so slow effectively because it's creating all these temporary objects and then destroying them in the next operation.

15:20 This portion of Talk Python to Me is brought to you by Shortcut.

15:23 Formerly known as Clubhouse.IO.

15:25 Happy with your project management tool.

15:27 Most tools are either too simple for a growing engineering team to manage everything or way too complex for anyone to want to use them without constant prodding.

15:35 Shortcut is different, though, because it's worse.

15:38 Wait, no, I mean it's better.

15:39 Shortcut is project management built specifically for software teams. It's fast, intuitive, flexible, powerful, and many other nice positive adjectives.

15:48 Key features include team based workflows.

15:51 Individual teams can use default workflows or customize them to match the way they work.

15:56 Orgwide goals, and roadmaps.

15:58 The work in these workflows is automatically tied into larger company goals.

16:02 It takes one click to move from a roadmap to a team's work to individual updates and back. Hype version control integration.

16:09 Whether you use GitHub.

16:10 GitLab or Bitbucket Clubhouse ties directly into them so you can update progress from the command line, keyboard friendly interface. The rest of Shortcut is just as friendly as their power bar, allowing you to do virtually anything without touching your mouse.

16:25 Throw that thing in the trash. Iteration-planning, set weekly priorities, and let Shortcut run the schedule for you with accompanying burn down charts and other reporting.

16:34 Give it a try over at 'Talkpython.FM/Shortcut'.

16:39 Again, that's 'Talkpython.FM/shortcut'.

16:42 Choose Shortcut because you shouldn't have to project manage your project management.

16:49 One of the things you can do is maybe understand that there are numbers there and treat them.

16:53 Keep say the intermediate values as floating points and only return the result to Python.

16:58 Right?

16:58 Like, okay, this Python runtime is going to need a Py Int or Py Long or Py Float or whatever, but we don't need to do all the intermediate steps that way.

17:08 Right.

17:09 We could compute those in a lower level thing because we again understand the whole function, not just understand multiply two numbers, multiply two numbers, add two numbers.

17:19 But that whole thing as a group, right?

17:21 Yeah. Exactly.

17:22 So the principle behind some of the design ideas in Pyjion is that and in lots of other compilers, this is not something I came up with, but the idea is to try and keep things as efficient as possible by just carrying values on the CPU registers and then not allocating memory in the heap.

17:41 And a floating point number fits in a CPU register.

17:44 So a 64 bit integer or a flighting point number fit in a CPU register.

17:49 So let's just carry those values on the registers and then do low level instructions to do addition and minus multiplication as well, but not divide because I found out Python has a whole bunch of, like, custom rules for division, so I can rely on the CPU instructions to do that.

18:09 So these are the types of things that well, maybe we could use this Pep 5.23 and some kind of compiler and turn it loose on this.

18:19 I just earlier this week interviewed Guido and Mark Shannon about just general Python performance. They're working on a parallel branch of making Python faster, which is great.

18:31 So out of the live stream, Josh Peak asks.

18:33 Mark and Guido tease the potential of addition of Jit CPython 3.13 3.14.

18:39 Would this potentially intersect with that project?

18:43 Is this totally separate?

18:44 Do you have any visibility there?

18:46 Yeah.

18:46 I haven't asked to be involved in that yet. I don't know if Mark Shannon's experience with compilers is like miles ahead of mine.

18:56 And to be Frank, Guido has invented the language so that their knowledge surpasses quite substantially.

19:03 Yeah.

19:04 I hope that the work done in this project will be insightful when they're designing the jit.

19:11 And I've already spoken to both of them about this project and walk through like, what's working and what isn't because it's I guess it's quite a bit ahead, and it's dealing with some challenges which they're probably going to hit when they come to this, then. Yeah, it will stir them in that direction.

19:26 Cool.

19:27 Let's talk about compiling just for a bit, because I did a lot of C++.

19:32 I remember pressing compile the build button or the run, which would build and run, and you'd see it grind.

19:38 And actually they came back. That was when computers actually made noises.

19:42 They would like their hard drive would like make noises there.

19:47 You would hear it compiling, even also in C# .net

19:50 Compile, but less and much faster.

19:53 But in Python, it runs.

19:55 It feels like it just runs. But I don't remember this compile stuff.

19:58 And yet there is an aspect of compiling, right?

20:01 Yes, it happens.

20:02 You just don't see it, see, it happens behind the scenes that compiles it, but it doesn't compile it into machine code. It compiles it into byte code.

20:09 Right. Which is the same as Net and Java. But the difference is what happens to that by code.

20:13 Next. Right.

20:14 Yeah.

20:15 It is similar, but the Python bytecode is much higher level.

20:19 So there's, like, single operations for just as two objects, for example.

20:24 Right. Or put this thing in a list.

20:25 Yeah.

20:26 Like, add this thing to a list or merge two dictionaries. It's like DICT.

20:29 Merge is a single bytecode instruction, whereas Net uses a specification.

20:35 It's an open specification called ECMA 335, and this specification describes different stack types. So it says there's, like, 32 bit integer, 64 bit integer, 16 bit et cetera.

20:51 There's floating point numbers, which come in the form of four or eight bytes floating point numbers.

21:00 And there's also things like booleans and then how branches and evaluations work. So it's closer to assembly.

21:08 But the reason you don't want to write things in assembly is because assembly is specific to a CPU, and you often find yourself writing instructions, which would only work on that particular CPU. And then when you ship it to the real world like that doesn't work.

21:22 Right.

21:22 One of the big benefits of Jit is it can look exactly at what you're running on and say, oh, this has this vectorized hardware thing. So let's use that version here or this has this type of threading.

21:33 So we're going to do some sort of memory management around that C is C++ ahead of time compilers. They will interpret your code, parse it and compile it down into machine code instructions, and then put it in a binary format, like a shared library or a standalone executable.

21:50 Net, Java and other languages that have a Jit.

21:55 They have both a compiled VM, which is something which has actually been compiled into a standard executable, which is the framework, so that the Java.exe, for example.

22:07 And then it could compile down the code into an intermediary language and then evaluate that just in time and then typically cache the machine code onto a disk or into memory.

22:19 And it does that using a Jit and CPython interprets everything runtime essentially.

22:27 So it does cache the Byte code, but it doesn't cache up the machine code because it doesn't compile to machine code.

22:32 And that's what Pyjion does.

22:34 Right. Exactly.

22:34 If you've seen PYC files at the Dunderpie cache right down in theirs, that's the compiled output of Python. But like you said, it's a much higher level and it gets interpreted after that.

22:46 So part of the insight is like, well, let's take that compile step.

22:52 And instead of outputting Python bytecode, what if we output intermediate language bytecode?

22:58 Because there's a nice compiler hanging around that can compile that if you could somehow feed it, that Il instead of PYC content.

23:05 Right.

23:06 Yeah.

23:06 So the steps are it's quite involved.

23:10 The steps are and I do go to this in the talk.

23:13 But Python code abstracts syntax tree code object, which has Python bytecode, and then Pyjion will basically compile Python bytecode into .Net intermediary bytecode, and then .net will compile the intermediary bytecode into assembly into a machine code.

23:31 And then you attach that to say the function object or a class or something like that. Right.

23:35 Yeah.

23:35 And then that bytecode that machine code. Sorry, is essentially an executable which lives in memory.

23:41 And then when you want to call it, you just call that memory address and it runs the function just in the same way that you would load a shared library and just call the address.

23:52 Right.

23:52 You might have a .so file and you import it and run it. And as far as you're concerned, magically, it just runs. Right. Exactly.

23:58 Okay.

23:59 Is it slow the compilation stuff in particular?

24:02 Not necessarily.

24:03 We'll get to the performance of the overall system, but is this jit step?

24:07 Is this a big deal?

24:08 Does it take a lot of memory?

24:09 What's it like? Yeah.

24:10 I haven't actually focused too hard on the performance of the compilation step because a lot of the problems I'm looking at are compile once execute 50,000 times, and the overhead doesn't really matter that much, although it's pretty fast.

24:25 I've been really impressed.

24:27 Pyjion is written in C++, and the compilation steps are pretty quick.

24:32 The overhead is 10% to 15% of the execution time on the first pass, depending on the complexity of the function.

24:39 But yeah, if the function takes a second to run, then .15 of a second is around how much it will take to compile it.

24:48 Yeah.

24:48 That's not bad.

24:49 And then it goes faster.

24:50 Yeah.

24:50 And then once it's done at once, that's it.

24:53 With the exception that Pyjion has a feature called Profile guided compilation, which is kind of something that I designed to get around how dynamic Python is.

25:05 Jit compilers are brilliant when you've got statically Typed languages.

25:09 So if you know that this variable is an integer and this variable is a string and this variable is an object, then you can compile all the correct instructions.

25:17 But in Python variable, A could be assigned as a string and then change to an integer, and then you can assign it to the return of a function, which could be anything.

25:28 So one of the challenges I kind of looked at was how do you actually make a jit is only going to be faster if you've got optimizations, and you can't make optimizations if you have to generalize everything.

25:38 What it does is a feature called PGC, which it will compile a profiling function.

25:44 So the first time it runs the Python code, it's basically sort of look at what variables are.

25:51 It's almost like you're doing a C profile on itself.

25:54 Yes.

25:54 Basically, it compiles a function that runs, and then when that function is running, it kind of captures a whole bunch of information about what's actually happening.

26:03 And then it makes them assumptions and says when you were adding these three variables last time, they were all integers. So let's optimize that for integers next time.

26:12 And if they do change, then it depends.

26:16 It won't crash.

26:17 What happens if they change?

26:18 Okay. A crashing is an option.

26:19 You probably don't totally want to go with a crashing part, but that might be an intermediate.

26:25 Like we're building it. It's getting dialed in.

26:27 Yeah.

26:28 Some options that come to mind that you could have an alternate compiled version that says, okay, we've also seen this come as a string intend.

26:36 So we're going to compile a separate one and then do, like a look up on the arguments and go from there.

26:41 Yes.

26:42 They're called specializations, and that's something that Mark Shannon talked about. And I think when CPython does its own jit, they will definitely have specializations.

26:51 The downside is extremely, and the downside is that you have a lot of memory overhead.

26:56 If there are lots of specializations, and a good example would be in the unit test module, the Assert equal function.

27:04 This is pretty much the first one I kind of slammed into, like Pyjion tried to optimize the assert equal function, which could take anything like it could take two strings, a string and a number.

27:15 Probably. The first question is, are they the same type or can they be coerced in the same type and then just keep going down like these different cases? That can't be simple.

27:23 Yes.

27:23 And it was actually a conversation with Guido.

27:26 He suggested looking at typeguards.

27:28 So the type guard is before I go into the optimized code, it will check to see has the variable change from what it was last time they got profiled. And then if it has changed type, then it will default back into a generic path.

27:45 So essentially how it deals with different types.

27:49 Yeah. One of the follow through paths could be just.

27:51 Well, let Python have it.

27:53 Right.

27:53 Let Python just run the Byte code.

27:55 Yeah.

27:55 There are some things that Pyjion doesn't support async and await is one major major feature.

28:00 If it comes across asynchronous generators, then it will just hand them back to Python and Python executes them.

28:06 It should be in there, though, right. I mean, C#.

28:09 .Net also have async await. I know that means quite a bit differently, but theoretically, in the future, down the road when you have more time, maybe it's not completely out of the world possible.

28:18 I actually kind of started implementing it and put most of it together.

28:22 I need to realize that the APIs for Asynchronous generators are all private in CPython, so I can't import them, which makes it technically impossible to implement, which is a bit of a shame, but yeah, that's one of the drawbacks at the moment is you can't do async and await

28:39 Right. But if this were super successful, I can see that. That's like, okay, well, let's go ahead and expose that because Anthony is so close.

28:47 Yes, they could probably maybe be coerced.

28:49 It's like a one line code change.

28:51 I think this doesn't apply to the profile guided optimizations. But one of the things that these frameworks have, I know that .Net has had it at several levels, like they've got this Ngine utility that will take a Net Assembly in IL, and you can pre compile it, like ahead of time, compile it and generate a native image on your machine.

29:13 And then Xamarin had that because they had to have something like this to get onto iOS where they ran the J compiler in advance and saved it.

29:23 Is that something that could potentially be done here, or is it too much?

29:27 I watch this space. I've been researching that that's kind of. One of the things I've been looking into is can you compile it down into a format which can be stored and then loaded or Marshalled, or can it be stored into a portable executable format or some other binary format?

29:47 Lots of security implications there as well. So that's one thing I'm cautious of, but yeah, I want to look into.

29:53 Yeah. I hadn't even thought about all the challenges you got there.

29:55 Yeah, that could be interesting.

29:58 But if it comes along that this is pretty good. But there's this slower startup, and I know something that the cordevs and have been very protective of is the startup speed of Python, right. That they don't want it to start super slow, because often it's sort of run on a little tiny bit of do this tiny thing, and then we're going to drop back and then maybe run Python again on this tiny thing or even multi processing.

30:23 Forget, run these things, drop out of it.

30:26 So I'm just thinking of, like, how do you protect how do you still achieve that goal and gain these advantages?

30:32 Yeah.

30:32 I think there probably could be work done to make the compiler more efficient.

30:36 Also, you can set the threshold of how many times should a function be called before you JIT compile it.

30:41 Right.

30:41 So that's a threshold setting. So if you call the function once, there's probably no need to JIT compile it.

30:48 There is no need to JIT compile it because you're compiling it and then just running it straight afterwards.

30:53 Most of it gets caught a lot. Then you would want to call it a lot.

30:57 That's kind of where you get these things for hot functions, which is a function which is run a lot.

31:02 You want to specialize and make more efficient.

31:06 Essentially.

31:07 So.

31:07 Yeah.

31:08 If you're like sorting a list, for example, then doing comparisons between two different types, you'd want to make that as efficient as possible.

31:16 And that would inherently make sorting algorithms or quicker, for sure.

31:20 Yeah.

31:20 So there's multiple stages here. Right there's the uncompiled code or letting Python run the code, then there's compiling it with those hooks to understand what types come in for the specialization.

31:31 And then there's the generating the optimized version.

31:35 So if it's run once at best, you'll get the unoptimized compiled version.

31:40 An unoptimized compiled code is probably not that much better, right?

31:44 Yeah.

31:44 There are some things that happen that it can do to optimize.

31:48 For example, there's a list of all the built in, and it knows what return types the built-in have.

31:53 So for sure, like it knows if you run list as a built in function, then it will return a list.

32:01 And I have even put in a check if somebody's overridden their list built in, which is possible and how to test that as well. Which is interesting.

32:11 But yeah, it does make a whole bunch of assumptions like that, which is generic and works in most code.

32:17 And for example, if you're accessing the fourth item in a list, so you've got a list called names, and in square brackets, you put names, square brackets.

32:29 The number three, then three is a constant so it can't change it's compiled into the function.

32:34 If the code knows for sure that names is a list, then instead of doing calling us the API to see what it is and get the index, et cetera.

32:45 The Jit compiler can go, oh, it already know this is a list. I know the index you want is the fourth number.

32:51 So instead of calling with the stuff that just calculate the memory address of the fourth item in the list and then put in a little check to make sure that there are four items in that list and then just compile that into the function and it's immediately significantly quicker.

33:08 Go to where the data is stored in the list.

33:10 Go over by the size of four pointers.

33:13 So eight times four, something like that. Just read it right.

33:16 There something like that.

33:17 Yeah. Exactly.

33:17 Okay.

33:19 Assuming for sure that that's what it is, right?

33:23 Yeah. Exactly.

33:24 These are some of the security things, right.

33:26 Go over some part in memory and read it and then do something like that. Sounds like buffer overflow when done wrong.

33:32 So I can see why you'd be nervous.

33:34 Yeah, exactly.

33:35 But that's how compilers work.

33:38 You're dealing with memory addresses, essentially, and low level instructions.

33:42 Yeah.

33:43 Sometimes we thankfully don't have to do a ton of in Python, but like in CPython, you're in C, right? That's a C thing.

33:49 Yeah, definitely.

33:49 And when you're working with Tuples, you do that as well. So you work out the address of the Nth element and then just use that address and increment the reference counter.

34:00 Yeah. Also, don't forget that memory management break.

34:07 This portion of Talk Python to Me is sponsored by Linode.

34:10 Cut.

34:11 Your cloud builds in half with Linode's Linux virtual machines, develop, deploy, and scale your modern applications faster and easier.

34:18 Whether you're developing a personal project or managing larger workloads, you deserve simple, affordable, and accessible cloud computing solutions.

34:26 Get started on Linode today with $100 in free credit for listeners of Talk Python, you can find all the details over at Talk Python.FM/linode, Linode has data centers around the world with the same simple and consistent pricing, regardless of location, choose the data center that's nearest to you.

34:45 You also receive 24/7, 365 human support with no tears or handoffs.

34:51 Regardless of your plan size.

34:52 Imagine that real human support for everyone.

34:55 You can choose shared or dedicated Compute instances, or you can use your $100 in credit on S3 compatible Object Storage managed Kubernetes clusters, and more.

35:05 If it runs on Linux.

35:06 It runs on Linode.

35:08 Visit Talkpython.FM and click the Create Free Account button to get started.

35:13 You can also find the link right in your Podcast Player show Notes.

35:16 Thank you to Linode for supporting Talk Python.

35:21 Where are you with this? You said it's going to go to 1.0, which sounds like I could install this and I could run it and it would do its magic.

35:28 Right? It's going to 1.0 works only on Python3.10.

35:32 That's one big thing.

35:34 Upgraded it from 3.9 to 3.10 when three point ten was released, actually, and I won't be backporting.

35:40 It.

35:40 It's just so much has changed in Python in the APIs, you can PiP install it on Python three point ten.

35:47 You need to have .Net6 installed when that is released, or you can still install release Candidate two, which is already out.

35:55 And yes, you can enable it.

35:58 It sounds like it's pretty close to done, right. I don't know when it's actually, but they've got their Net conference in.

36:03 What is that?

36:03 Six days. Yeah, surely.

36:05 Yeah. They say it launches then by the time this episode out, it's very likely very close to just out.

36:12 So.

36:12 Okay, that's a pretty can I brewinstall .net. Do you know I've not tried?

36:16 I don't know.

36:17 That's a good question.

36:18 There is an installer for Mac.

36:19 Pyjion also works on Arm 64, which is worth noting, and it's a very complicated detail of J Compilers.

36:27 Arm support was no small feat, but it's something that people just expected to be there.

36:32 Yeah, sure.

36:33 Well, there's obviously the M one, right.

36:36 Everyone with an M one would like, who wants to do this? Would really like it to work on Arm, but there's also Raspberry Pi's and other places that Python shows up, helicopters on Mars. I don't know if that's Arm or not. Probably.

36:48 Yeah.

36:48 So I've tested Linux, Arm 64, which will be the Raspberry Pi and other and also Mac Arm 64. I have not tested Windows Arm 64 because there is no Python for Arm 64 on Windows.

37:03 Interesting.

37:04 Okay.

37:04 Steve Dower has released a preview package of the libraries, but not a standalone executable, but it may come out in the future.

37:14 We're going off on a bit of a tangent.

37:16 No, but the whole story with Windows on Arm, I would love to see it better handled, but it's like you can't even buy it, right?

37:24 Is it supported?

37:25 It's provided to OEMs, but kind of sort of I don't know.

37:28 It's in a weird state.

37:30 It's not normal Windows.

37:31 It's not super supported.

37:33 Yes, I think who knows?

37:35 I do work for Microsoft, so I'm definitely not going to get my yes.

37:38 I'm not asking for your this is more of me just making a proclamation.

37:43 I have Windows 11 running on my Mac Mini M one, and it is running the Arm, but to get it, I had to go join the Insiders program and then install it and it's permanently got this watermark, but it's a tainted version, but you can try to use it and it works fine, but it's kind of sluggish. And whatnot so anyway, hopefully that comes along better.

38:04 So it sounds like it's supported on the various things.

38:07 I have Net Six installed and Python 310 installed.

38:11 Neither of those have to be messed with. Right. You've got the Pep for Python and you've got just the Jit Vanilla installations.

38:19 Yeah, that's beautiful.

38:20 And so give us a little walkthrough on how we might use this.

38:23 What do I have to do to change my code to get these compilation steps wherever your code starts running and you need to import Pyjion and then call the enable function on Pyjion.

38:33 Okay.

38:33 There's also a config function which configures different settings in terms of how Pigeon runs like that threshold, for example, like how many times compile the hot code threshold optimization level, which is a level between zero and two, which is like how aggressive the Optimizer is.

38:51 And you can also enable or disable the profiler.

38:54 Yes.

38:54 I remember in C way back when I was doing C for projects, there were these different optimization levels and it was like, well, you can go to one or two, and it'll probably still work. If you go too far, it will just crash. It's like, what's going on?

39:10 I don't understand, but okay, we'll just dial it back until it's stable and that's as fast as we can make it go.

39:16 Yes, this is the same.

39:18 This is my recommendation.

39:19 Go as hard as you can without it catching fire.

39:22 Take a step back.

39:25 Maybe start at zero and it defaults to one.

39:28 So yeah, don't turn out to Eleven, but the profile is on by default, which I may disable in the future, because the profile probably causes the most issues where you've got a function which ran with integers 1000 times, and then all of a sudden somebody gave it some floating point numbers.

39:48 It won't crash.

39:49 It will either fall back to default path, or it will raise an exception to say that it got some values which I didn't expect.

39:56 And if you do see that, it will tell you in the error message, and it will suggest that you turn the profile off and then rerun the code.

40:03 You know, for me, I feel like that would suggest to me that maybe I should go check my code.

40:07 Yeah, not always, but often if I got something like I'm trying to take these things and add them and get what I think is a result.

40:14 I'm trying to do math, not string concatenation when I get a string.

40:18 Chances are that's actually a mistake, not something that I wanted to take account for. It could be, but probably not. Yeah.

40:25 She's a very useful example of this. Yesterday on Twitter, somebody shared adding two floating point numbers in Python A and B.

40:32 A plus B is not the same as B plus A.

40:35 They actually give different results, which is crazy, but yeah, when you work with floating point numbers and integers and you don't mean to, but you end up with different types, you will get some weird results anyway.

40:47 Interesting.

40:47 Yeah.

40:49 In terms of whether this will work, I've compiled some pretty big libraries and they've worked fine.

40:54 Pandas Flask Django chucked a lot of the CPython test suite this as well.

41:01 So yeah, it will run, I guess, to about 50,000 tests or something, I think quite happily.

41:06 Yeah.

41:07 A lot of CPython's tests testing specific internal things in CPython, so some of them do fail, but it's not anything that Pyjion has done.

41:16 And Py test works as well.

41:17 So yeah, there's a lot of big libraries.

41:21 Numpy works fine.

41:23 There's test for NumPy, test for Pandas, test for Flask Django. All the stuff that I expect people to try is in there.

41:29 If you're working with a lot of C extension modules, they also do work Cyphon extensions.

41:34 So in terms of compatibility, that was one of the main things I wanted to focus on was instead of going super aggressive with the optimizations, I just want to make sure this works with existing code because there are lots of other projects, which right.

41:47 We have PyPy already PY PY, which is also a Jit compiler, and it works in not with the Net backing legit, but some of the hot functions getting compiled versus just running Python.

42:00 That kind of stuff is pretty similar now, but they made the big trade off. Like we're going to just go all in on compiling and we're going to not necessarily integrate with the C APIs in the same way, which means breaking with some of these things, like NumPy or Pandas that people sometimes care about. Yeah.

42:14 And they also have to play catch up on the language features.

42:17 So if there's new features in 3.9 3.10 in the language like new operators or whatever, then PyPy has to then go and implement that which is hard Pyjion you load into CPython. So in terms of language, it would be exactly the same.

42:33 Right. Often a lot of those language features are just syntactic sugar over existing things. Right.

42:39 Yeah.

42:39 Exactly.

42:40 And then if there's anything which is not compatible, like I mentioned async and await, then it will default back to CPython, and that transition is seamless and you won't notice it will just run the code regardless.

42:51 Awesome.

42:52 It looks like it works for the most part. I haven't totally tried it, but it sounds like it works quite extensively the way you use it as you pip install Pyjion and then just import Pyjion. Pyjion enable is option one, and then that's it. Right. There's nothing else that you have to do.

43:06 Nothing else you have to do. You just run the Python code and it all just automatically spot stuff that it should compile and compile it for you.

43:14 Fantastic. And then another option I see on the page here is I can say Pyjion space some Python file and not necessarily modify the file, but tell it to execute.

43:23 What does it do? Import Pyjion Pyjion.enable Eval.

43:31 Yeah. Basically, it's a very small script.

43:34 So. Yeah, Pyjion is a standalone command that you can run instead of running Python, you run Pyjion against a script or a module, and all the arguments should work as normal.

43:45 Awesome.

43:45 You also have the dash M for built in stuff, right?

43:48 Yes.

43:49 If you want to run a script, then you'd run Pyjion and then the name of the script. If you want to run a module like Pytest, for example, then you would do Pyjion dash M Pytest and it would run Pytest with the Jit enabled.

44:01 Yeah. Fantastic or Flask or something like that, right?

44:04 Yeah. Exactly.

44:05 I guess the M would work with external libraries, right. As long as Python can see them.

44:09 Yeah.

44:10 And I've shipped a Whiskey extension as well so that you can use it in Whisky apps.

44:16 I think that that's an interesting use case, actually.

44:20 So when I run my regular Python code, it just loads up and runs.

44:25 But when I run Flask or FastAPI or Django or Pyramid or whatever, there's all sorts of layers of indirection or layers of not directly running it right in production. You would say, hey, I want Micro Whiskey or G Unicorn to run this. Like for FastAPI would be I want G Unicorn to run this, but with UV acorn workers and run five of them. And like, boom, boom, boom. Now you've described this chain of events, right?

44:50 Yes.

44:51 But it sounds like there's what middleware to make this work.

44:55 Still.

44:55 Yeah. It's Whiskey middleware that is for Pyjion, which will do the enabling and disabling fantastic required.

45:03 So that sounds like it works for any Whiskey app, Jingo Pyramid, et cetera.

45:07 What about ASGI apps due to the lack of Asynchronous support, then?

45:12 No, it doesn't really make much sense, right. Because the big thing that it does is not the thing that's supported, right.

45:18 Yeah.

45:19 If it's no sync function, then it will just give it back to CPython.

45:22 I'm sure there's a lot of synchronous things happening in those various places. Right.

45:28 Maybe the View method itself is async, but it might call a whole bunch of give me the headers and the cookies synchronously. Who knows?

45:35 Yeah.

45:35 Exactly.

45:36 It also depends on the nature of the program as to where the pyjion is actually going to make a difference to their performance.

45:42 So kind of where I'm up to at the moment is different benchmarks and running Pyjion against some standard benchmarks.

45:52 I showed that N- body execution time at my PyCon talk, and that was 33% faster.

45:59 It's now 65% faster. So I've doubled that.

46:02 Oh, nice gain.

46:04 So however, most people aren't calculating the position of planets.

46:10 But the few who are they going to be super thrilled.

46:13 Yeah. For the few people who are doing it in Python, the system doing in Python.

46:17 Yeah.

46:17 Then they'll be delighted.

46:20 Code, which is doing a lot of math and is in pure Python, would be faster.

46:25 Up to 20% faster, not 20%, 20 times faster. I've got some micro benchmarks that do, like simple calculus and stuff like that. And they're 20 times faster with floating point numbers.

46:39 And for us, it's small integers, because an integer in Python Int in Python is an unbounded thing.

46:48 It's bounded by your memory, right?

46:50 Yeah. It's actually a list of digits, so it can have an almost infinitely large number inside it.

46:56 Whereas CPUs work with 32 bit or 64 bit numbers and the other languages instead of keep growing.

47:02 They just go like we broke. So instead of going one more, it goes like negative 2 billion.

47:09 Yeah. You get funny overflows and stuff.

47:11 So one of the challenges I've had with Pyjion is trying to optimize integers, but trying to understand where it potentially could be a very big number.

47:21 And the number is like five, what's, five times five?

47:26 You don't need to allocate, like, half a Meg of memory to do five times five.

47:31 Yeah.

47:32 So that's one of the challenges.

47:33 If you're working with integers and you're working with floating board numbers and you do a lot of math and Pyjion will make a dramatic difference.

47:39 Fantastic.

47:40 There's also a feature called The Graph, which will create graphics files for the functions that it's compiled.

47:49 And you can see this on the website.

47:51 If you go to live.trypyjion.com.

47:53 You've got this interactive live website, right? Yeah.

47:56 So I kind of made a sort of Live demo site where you can type Python code in and click compile, and then it will show you, I don't change it. Michael, you broke it.

48:07 It's going to be fine.

48:09 And then do you compile?

48:10 It's going to be fine. I got faith in you.

48:12 Now I can delete it.

48:14 Okay.

48:14 That's the assembly that it has compiled that Python code into.

48:18 Okay.

48:18 And the fun thing actually added was there's a comment in assembly which says which bytecode this is for which is fun. If you scroll down on the page, you see this graph.

48:28 I got to do my and then keep going down.

48:31 Okay.

48:32 There we go.

48:34 I don't have a big enough screen there.

48:35 Here we go.

48:37 Okay.

48:38 This instruction graph gets generated.

48:41 If you enable graphing, it's on the documentation. But when you enable graphing, it will show you all the Python byte codes, and then what values are being sent between those by codes.

48:52 This is basically the Alpha profiler you can see here that you've got A and B B with a float.

48:59 A is an integer, and then it's doing a multiplication operation, and it knows that if you multiply a float by an integer, then the output is a float.

49:07 So it carries that value through the graph. PyPy Actually does things quite similar to this.

49:12 And then once the profiler has run, it will look at the graph and then make some decisions about which values don't need to be Python objects.

49:22 So for example, A times B plus these intermediate binaries here and stuff.

49:28 Right. This in place.

49:29 Add all those.

49:30 Yeah.

49:31 Exactly.

49:31 All of those values do not need to be Python objects. So what it will do in the next pass is recompiled that. And then it will what's called unbox them. So it basically just carries them in CPU registers as integers and floats.

49:46 And then instead of running this C function to add two numbers together, it will just emit the assembly instruction to add two numbers on the register.

49:55 Yeah, that's fantastic. That's what you're talking about, where you don't have to drop back into allocating Python numbers, if you know for sure that no one's going to look at it. It's just an intermediate value.

50:06 And this is where it gets tricky with integers, right? Because A and B might be small, but A to the power B might be larger.

50:13 Exactly.

50:13 And then it goes one step beyond as well. If you have code that uses fast values, it's tricky, because when you do Eval in the website, it will never used fast locals. But if you do have a function that has fast locals, if it detects that locals only ever used in places where it can be unboxed, then it won't actually store the variable as a Python object.

50:38 Either it will store it as a native stack value.

50:41 So that's something.

50:43 If you have a variable called A and you assign it to the number two, then it will actually just reserve an area in memory just to store that stock value.

50:53 And it will be an offset.

50:54 Which is way more efficient.

50:56 Enormous, like thousands of times more efficient.

50:59 And when you refer to that variable in your function, it will basically just access that point in memory to get the actual value.

51:07 Awesome.

51:09 One thing that stands out to me here is I wrote name equals Anthony plus Shaw as two strings, and you're like, I don't do it.

51:17 And yet what I see in the graph here is that it loads of the string Anthony Shaw.

51:22 So did Python look at that and then decide that those are two constants that will just make it one constant, or was that net or what happened there?

51:32 Yeah. That's constant folding is feature of Python yeah.

51:35 That's what I thought.

51:36 If you do two strings and a plus, it will actually compile those into one string, you'd never see it.

51:43 Oh, interesting, because they're static, right.

51:45 Like it knows both of them. Yeah.

51:47 Okay.

51:47 Very interesting.

51:49 So sounds like for numerical stuff, this is quite a bit faster.

51:52 Did you do any tests on the web frameworks?

51:55 It's kind of appealing to say, what if I could get fast to kind of run natively?

51:59 Yes, I have.

51:59 Because when you think about I'm writing this code to make this website go, most of what you're doing is you're doing, like a little tiny bit of code on top of a big framework. That's doing most of the heavy lifting. Right.

52:12 So if you could make Flask do that magic faster or Django or whatever.

52:17 Yeah.

52:18 So the areas where Python is faster as numerical work, similar to PyPy PyPy is a lot faster. With numerical work, it can make clear and simple assumptions.

52:28 And optimize based on the CPU.

52:30 That's brilliant areas where it's not faster or sometimes even slower is code, which just uses a lot of classes and very small functions, partly just because of the way the purpose designed it will jit compile functions.

52:48 And if your functions are just working with custom classes and you're passing things around, then trying to decide what type things are and then how it can optimize types.

52:57 If everything is a custom object, custom Python class, there's very little it can actually do to optimize.

53:04 Yeah.

53:05 When you were talking about the specializations earlier, it's one thing to say. Well, I'm passing a customer and an order object, but then they have fields themselves, each of which have potential types.

53:15 Right.

53:15 Like it's this object graph, this closure of all the object graphs, and you got to look at all those types you might touch, right?

53:22 Yeah. And then you've also got a check that field exists.

53:25 It's been set. It's not none.

53:27 Like if you DJango by the time you're done testing them all.

53:30 You might as well just run it.

53:31 By the time you've done all of that, you've basically just written what CPython would have done anyway.

53:36 But the difference is that if you JIT compile it, you've got to emit all those instructions and the compiled function.

53:42 The Jit compiled function ends up being bigger, because if it's compiled in C, it just has one function that does that shared by all libraries, whereas in the gym, you have to make it so that it's their standalone functions.

53:56 So that's one downside is that if you're working with stuff, which is similar to Django and Flask, I guess like lots of classes, lots of variables which are all custom types.

54:05 Probably not going to see a performance improvement.

54:07 Or potentially it could even be slower.

54:10 Will it be transitive?

54:11 If I write code that runs in Flask, let's just pick one of them and I run Flask with Pyjion. Will that then make all my code that Flask is executing also run in Pyjion. Yeah. So maybe if I was doing, like, highly computational stuff in my Flask app, having to do that might be worthwhile.

54:29 Yeah, definitely.

54:30 And in that case, you can just enable Pyjion before those functions, or you can set the threshold to be higher.

54:36 That's what makes more sense.

54:37 I think the other area is strings.

54:40 So if you're doing a lot of work with strings, I haven't done any work on optimizing strings, and I'm not particularly sure what you would optimize either, because they're so complicated.

54:52 You're dealing with all the Unicode, different encodings, different back links.

54:57 I don't even know how you would improve upon CPython string implementation.

55:02 To be honest, there's a lot of nuances to strings in all the languages about it, especially in Python, because you don't have to think about it, the fact that you don't know how complicated unit code is and word alignment and null characters and so on and so on.

55:19 Yeah. If you want a glimpse of how complicated is, look at the Unicode object source file in CPython.

55:25 Is it big?

55:26 It's an absolute monster.

55:29 It's probably more complicated than see about, actually.

55:32 Oh, my goodness.

55:33 Yeah.

55:33 All of that for Emojis.

55:36 Okay. I understand there's other languages.

55:38 No.

55:39 Interesting.

55:39 So one thing I wanted to ask you about here I was hunting around on the screen for it. Is it's cool that you can compile these things down and run them as native machine instructions instead of piping them through CFL C as single line operations?

55:52 That's great.

55:53 But when I think of the stuff that compilers can do and check compilers, it's a lot about the optimizations. It's like I saw this function and this function, actually, we're going to in line that one. And this one. We're going to rewrite this as some other thing that's going to be more efficient because we're seeing that. And so you do have some optimizations, right. Like that's part of the magic.

56:15 Yeah.

56:15 I've kind of documented them as best I can on the Pyjion documentation. There's a section called Built in Optimizations, and I've given them all numbers.

56:23 And if it uses that optimization, it will flag the function to say, oh, I'd use this optimization on this function.

56:30 And then in the documentation, I'll explain what's the background.

56:33 What was the idea?

56:35 What difference does it make?

56:36 You want to to give, like, some examples from some of these, like the is one, for example.

56:40 Yeah.

56:40 So if you're using the Is operator in Python, so if something is false, not something.

56:49 It's none or something like that.

56:50 Yeah, it is none.

56:51 Then it won't run the Py Is function.

56:55 It won't run the C API to do an Is comparison. It will just look at the address of both objects and see if they're the same.

57:00 Right.

57:01 Because it's actually asking, are these objects the same not. Are they equivalent?

57:06 Right.

57:06 So the same in CPython is the pointer addresses are equal. Yeah. Exactly.

57:11 It will just compile that down to a simple point of comparison.

57:14 That's actually one of the first ones I wrote. And it was good to learn also when it's doing comparisons for small numbers.

57:22 So Python inter it kind of immortalizes numbers between.

57:27 I can remember the range negative five and 256.

57:29 Yeah, that's it, because they use so much that maybe 255. But basically it's around that many. It immortalizes them. And then that keeps them as constant objects so that they're just reused.

57:40 So if you create a new integer with the number one, it will just reuse the same number one that he used before.

57:45 Right.

57:45 Because they're not created on the stack.

57:47 They're like complicated things on the heap.

57:50 Exactly.

57:50 So if something equals equals 25, Pyjion will go. Oh, well, I know 25 is an intern numbers, so I'm actually just going to do a point of comparison instead of a value comparison if the left hand side is a number, so that there's a bit like little things like this, which makes small differences, but when you add up, they add up.

58:09 Right.

58:09 Yeah.

58:10 So I felt like I don't know where you are now from where you were when you spoke at PyCon.

58:14 But I feel like this is an area that people could contribute to. This is an area for growth that doesn't require a lot of changes.

58:22 It could be super focused. But if you see this situation here's, one more way to make it faster.

58:27 Absolutely.

58:28 So. Yeah.

58:29 This is one area where I was hoping that the research that I was doing whilst writing Pyjion could contribute to other projects.

58:36 And I've been talking to Carl Friedrick bosh as well, who works on Pypy, and we're going to do some Py programming at some point to see kind of like how different projects work and stuff like that.

58:50 But hopefully these optimizations can be learnt from and then used when CPI gets to implementing its jit.

58:59 Surely the Net compiler has all sorts of optimizations in it.

59:04 Are you already taking advantage of those to some degree?

59:06 Yeah.

59:07 Just by nature of letting it run, basically.

59:10 Yeah.

59:10 Some of those Optimizer it does a lot of them already.

59:13 And I've been working with the compiler team on the Net project as well.

59:18 Pyjion is one of the only projects to use the compiler directly.

59:23 Even that can be used in that way.

59:27 I think it's designed to be run directly, but it's not like advertisement off the shelf yet.

59:33 So yeah, there aren't many projects that are using it in that way, and I do work with the compiler team specific test cases and stuff as well.

59:40 You have the advantage of being on the inside of Microsoft, even if you're half a world away, you still can get direct access remotely, which is like being down the street these days.

59:51 Yes.

59:51 I've got everything it's all been by GitHub, though, so I think as far as they're concerned, I'm just another name on GitHub.

59:59 They might not even know.

01:00:00 Right.

01:00:01 I probably don't even know that I work.

01:00:05 Let's see what else.

01:00:07 I think we pretty much have covered it, given the time that we got. I mean, another area that's interesting is how does it compare to the other things that have been done before it or going along a parallel? And you do have a whole section to read me on the GitHub page.

01:00:22 How does this compare to X-Y-Z, pypy piston number, iron Python and so on?

01:00:29 Do you want to maybe just have a quick statement about that? Yeah.

01:00:31 I think it's really hard to compare them, so probably the most obvious ones to compare it with would be Siphon number.

01:00:38 And Pypy.

01:00:40 Pipe is a Python interpreter that has a jit, so it interprets compiles and runs Python code written in Python pipeline has been around it's like a fork of Python that was rewritten to behave differently rather than Pep 523. Right.

01:00:55 Yeah. Exactly.

01:00:58 Pypi is written in Python and probably significantly faster in many cases.

01:01:06 It's a very mature project, but obviously that there's limitations around C APIs, and some things don't work in Pypy jit specific for NumPy.

01:01:19 If you're using NumPy, then if you use number, can JIT compile, NumPy, data arrays and stuff like that, that's actually very specific use case.

01:01:30 Right. That's great. But if you're not doing NumPy than in any other use case.

01:01:35 It would make very little, if any difference at all.

01:01:37 And then Syphon is not compiler, and it's a way of annotating Python code with concrete types, and then it compiles them into C. Concrete, compiles them into C extensions.

01:01:51 Yeah. It's a little bit like what you're talking about, trying to understand what are these types? And then can we create a specialization?

01:01:56 But it's the developer who just says no, these are integers.

01:01:59 This is a list for sure.

01:02:01 Go with it. Yeah.

01:02:02 That's the point where you actually specify the length of the integer as well. So in Siphon, you would say this is a 64 bit integer so that it knows that it can be converted into a 64 bit integer. In C.

01:02:15 Pyjion is probably closest to Siphon.

01:02:17 Obviously, with Siphon, you have to compile it to a C extension ahead of time, whereas the Pyjion you just import it into Python and just turn it on and it just runs and compiles live, and it will.

01:02:28 Yes. You don't have to have different compiled versions of your app or your library.

01:02:32 Not necessarily because of this anyway, for different platforms. Right.

01:02:35 And you don't have to annotate the code in this special syntax either.

01:02:39 Yes.

01:02:40 Well, let's close this out with a question on that.

01:02:43 So in Python, we've been able to say optionally that this thing is an integer X or it's an optional integer, or it's a customer or whatever the heck it is.

01:02:54 We've had these type annotations, and traditionally they've meant nothing, right?

01:02:59 Except for if you run tools against them. On the other hand, we've had things come along like Pydantic like FastAPI that look at that and go, you know what? I'm going to do something with that because I got a string because this is the web, and that's all I usually get.

01:03:10 I'm going to make that into an it because you said it's an end.

01:03:13 Is there a scenario where type annotations play into this to enhance it?

01:03:18 Somehow I'm against that idea.

01:03:20 Potentially, that's how Cython works, and with type annotations as well.

01:03:26 Having a type checker is great until the types are wrong, and it's not a fault of your own.

01:03:33 And having worked on, like, strongly type languages like C#, C# is brilliant, except when you need to do things like reflection.

01:03:42 So let's say you're working with Json, which like Jason Data or YAML, for example.

01:03:48 And working with Json and YAML in C# is incredibly hard because you're like, I've had to do parsing YAML files in Java.

01:04:00 It's incredibly difficult because you're like, oh, well, this could be a list of strings, or it could be a dictionary where the key is a string, or sometimes it's a number which is like YAML is like that sometimes.

01:04:14 And JSONs like that is completely free. So when you're working with dynamic content, using strongly type languages is extremely difficult.

01:04:21 It just gets in the way and just makes your life harder. And the compilers just complains and just won't let you pass because it's saying, oh, well, that's not compliant with my view on the world with Python.

01:04:32 I think it's cool to say, okay, if I just tag this as an integer, then it should be able to optimize it for integers. And I think that's what Mark was suggesting.

01:04:40 If it stops there, that's fine.

01:04:41 I think if it goes beyond that, then that's where things get very complicated and it just becomes a thing that's in the way, being noisy and slows you down.

01:04:50 That's where I have a strong disagreement with it is that we use Python because it's fast to develop in, and it's faster developing, because if you know what you're trying to get it to do, then the compiler doesn't really give you any grief if it's sync perfectly, it's okay. I'll just try and run it.

01:05:05 Yeah.

01:05:06 And I think the thing that's most similar to this is TypeScript, and my experiences with TypeScript have been often met with frustration.

01:05:15 I think it's really cool that you can write the language specification is really neat, but I want to use this library and that thing and pull them together.

01:05:21 And if that thing doesn't have the type specified just right, I can't get JavaScript to pop out the other side because the transpiler gives you a compiler error like this doesn't work right. Well, no it's going to work, right.

01:05:33 Just let this little part go through.

01:05:35 I don't want to read.

01:05:36 I know there are ways to say this is just any and stuff, but still, I feel like my experience was bumping up against that stuff, even though it's kind of the same as Python type annotations in a lot of ways.

01:05:47 Yeah.

01:05:47 And under the hood, it's still dynamic.

01:05:49 Yeah.

01:05:50 Exactly.

01:05:50 At runtime, it would still be the same thing, but you won't get to the run time because you can't get the stuff compiled.

01:05:56 Yeah.

01:05:56 Or it does really weird things at runtime that at compile time.

01:05:59 It made assumptions that wouldn't happen. All right.

01:06:01 Let me throw one quick thing out there before you pass final judgment on this type thing, because you're right.

01:06:08 I could write a thing that says my function takes an Int and a string, and I could run mypy against my code. And sure enough, it only takes ends and strings in these situations. But if it's a library, all bets are off.

01:06:20 There's nothing that says people who use your library are going to run mypy and listen to that or do anything right. They could write notepad if they had no self respect. But that was like you could write whatever you want, right. And then you could just feed it on and then you'd have these problems that said, over on the Pydantic world, there is a validator decorator called at validate arguments that will make sure at runtime they really are stringing an end.

01:06:47 Yes.

01:06:48 To make it feeling better.

01:06:50 Probably not because the transit closure of objects is too complicated to check and describe the value arguments is I think it's a convenience function.

01:06:57 It's there because you often have to validate data user provided data that's coming in from an interface, like a web interface or via phone or an API of some sort.

01:07:08 So people are submitting user provided data.

01:07:11 Then you want to have to validate the types. And that is just a lot of boilerplate code, and it's annoying to write. So as a convenience function, let's write this thing.

01:07:20 Right. Probably because we're going to immediately try to convert it. And the next thing right? Yeah.

01:07:24 Exactly.

01:07:26 I think it's quite different to using it internally to make changes in terms of how the code is compiled for numbers. Like I said, numbers and strings and base types, I think, is cool.

01:07:39 But for objects like which fields do you assume exist?

01:07:43 Yes.

01:07:43 Exactly.

01:07:44 And then do they all have to match the types or just the two that you touch?

01:07:47 Yeah.

01:07:48 You've got an attribute there, like sometimes it has a getter and a setter.

01:07:51 Sometimes it's just an attribute descriptor that inside the descriptor is the thing you actually want.

01:07:58 Is that the right type? Yeah. You're right.

01:07:59 It's insane.

01:08:01 Once you kind of open. I think that once you open custom types, then it just kind of mushrooms into this.

01:08:07 Sure and that's why you see things like Cython and stuff having like little fragments of here's some limited code that we're going to use often fundamental types.

01:08:17 All right, Anthony, this is really awesome.

01:08:18 I can tell you've done a massive amount of work on it.

01:08:21 That's more stuff I want to talk to you about, but I think we're at a time like client into net stuff like that, but we'll save that for some other time.

01:08:30 Yes, you have a little more progress now.

01:08:35 There's always the two questions at the end of the show.

01:08:37 So if you're going to write some code, what editor are using these days vs code right on. Yeah.

01:08:43 And notable PyPi library.

01:08:46 It could be Pyjion.

01:08:47 Anything else like you come across like, wow, this is awesome.

01:08:50 Yeah, I think I mentioned it before.

01:08:52 Tortoise ORM I'm a big fan of at the moment.

01:08:53 Yeah. Tortoise, ORM.

01:08:55 It's a nice async ORM and Beanie as well. I really enjoy playing with Beanie. Is an ODM and async ODM on top of Mongo.

01:09:04 Yes, beanie is very cool. I'm actually having Roman Wright on the show talking about Beanie not too far out as well.

01:09:10 And Beanie is cool because it's basically Pydantic plus async and await on top of MongoDB, which is cool.

01:09:15 Can I have three?

01:09:17 You can have three released like a small package called Hathi, which is SQL attack tool for Postgres my SQL and SQL Server.

01:09:27 I'm looking at your GitHub profile here. You've got one of these fancy profiles with a new read me that shows up in all of your stars. Look at this.

01:09:33 Yes, I made a custom graphic for and everything.

01:09:36 Yeah, you did. That's fantastic.

01:09:37 Okay.

01:09:38 What is this one called Hathi?

01:09:40 H-A-T-H-I have too many repositories.

01:09:43 Got it.

01:09:44 It is a dictionary attack tool for Postgres MySQL and My SQL. Microsoft SQL Server designed for internal testing.

01:09:53 Of course.

01:09:54 Don't be sad.

01:09:54 Yeah.

01:09:55 So don't break the law.

01:09:57 And yeah, I've been using it to test like internal stuff, but test environments and look at bad passwords.

01:10:05 See if an admin has a login like password, password one or admin or super user as a login or whatever.

01:10:12 Yeah, it was also a test of,async and await, networking and how fast I could make it.

01:10:17 I can do up to 120 login attempts a second.

01:10:21 So yeah, on my machine. But maybe this is a four core match, but if you had a few more CPUs, you could probably get a bit more than that.

01:10:30 But yeah, it goes through a few thousand passwords and there's a password listing as well of about 10,000 common database passwords. Yeah. Nice.

01:10:38 So yeah, just make sure your password is not on the list.

01:10:41 And if it is, you can raise a pull request to remove it.

01:10:44 Yeah.

01:10:46 You don't want to have your database open like it doesn't get much worse.

01:10:50 Then I just have read write access to your database.

01:10:53 But that's what this would test for, right? Yeah.

01:10:56 So you can scan a cluster, a machine and see if it can predict what the username password is. Yeah. Cool.

01:11:02 All right. Well, those are three great recommendations.

01:11:04 It's awesome to have you back on the show and Congratulations on the work here.

01:11:09 Final call to Action people. Maybe to try Pyjion. You want them to contribute?

01:11:13 Got other ideas.

01:11:14 What do you say? Yeah.

01:11:15 Go to trypyjion.com

01:11:16

01:11:17 Yes, on Try Pyjion.com.

01:11:18 You'll see at the top, you can try that. Live links to documentation, links to download.

01:11:24 Yes. I'd love more contributions or discussion. I think around what optimizations you could have and people using it and checking it out.

01:11:31 And if you have any issues, you just raise them on GitHub and I'll check them out. All right.

01:11:34 Fantastic.

01:11:35 Well done. I can tell this is a ton of work, so you've come really a long ways and Congrats on 1.0 when it comes out in a few days.

01:11:42 Thanks, Michael.

01:11:43 Yeah.

01:11:44 See you.

01:11:45 This has been another episode of Talk Python to me.

01:11:49 Thank you to our sponsors.

01:11:50 Be sure to check out what they're offering. It really helps support the show.

01:11:53 Choose Shortcut formerly Clubhouse.IO for tracking all of your project's work because you shouldn't have to project manage your project management, visit Talkpython.FM/Shortcut.

01:12:04 Simplify your infrastructure and cut your cloud bills in half with Linode's Linux virtual machines.

01:12:09 Develop, deploy and scale your modern applications faster and easier.

01:12:13 Visit Talkpython.fm/linode and click the Create Free Account button to get started.

01:12:18 Do you need a great automatic speech to Text API?

01:12:21 Get human level accuracy in just a few lines of code?

01:12:23 Visit 'Talkpython.FM/AssemblyAI'. Want you level up your Python, we have one of the largest catalogs of Python video courses over at Talk Python.

01:12:32 Our content ranges from true beginners to deeply advanced topics like memory and async and best of all.

01:12:38 There's not a subscription in site.

01:12:40 Check it out for yourself at 'Training.Talkpython FM' be sure to subscribe to the show, open your favorite podcast app and search for Python.

01:12:48 We should be right at the top.

01:12:49 You can also find the itunes feed at /itunes, the Google Play feed at /play and the Direct RSS feed at

01:12:56 /Rss on Talk Python FM.

01:12:59 We're live streaming most of our recordings these days.

01:13:02 If you want to be part of the show and have your comments featured on the air, be sure to subscribe to our YouTube channel at Talkpython.FM/

01:13:09 Youtube.

01:13:10 This is your host, Michael Kennedy.

01:13:12 Thanks so much for listening.

01:13:13 I really appreciate it.

01:13:14 Now get out there and write some Python code.