Imagine a future where you're building a rich client-side web app.
You start by creating some backend services, maybe in Flask or Node,
create an HTML page, you throw in a few divs and ULs,
and then you type script source equals main.py language equals Python.
That future might just be possible for the right type of applications
with Ryan Kelly's PyPy.js project.
This is Talk Python to Me with guest Ryan Kelly.
Show number 32 recorded Wednesday, September 30th, 2015.
I have many interests, sometimes conflates my creativity.
Welcome to Talk Python to Me, a weekly podcast on Python,
the language, the libraries, the ecosystem, and the personalities.
This is your host, Michael Kennedy.
Follow me on Twitter where I'm @mkennedy.
Keep up with the show and listen to past episodes at talkpython.fm
and follow the show on Twitter via at Talk Python.
This episode is brought to you by Hired and Codeship.
Thank them for supporting the show on Twitter via at Hired underscore HQ
and at Codeship.
Hey, everyone.
Thanks so much for listening in.
No news today, so let me introduce Ryan and we'll get right to the interview.
Ryan is a software developer based in Melbourne, Australia.
Most of his days are spent coding in Python and JavaScript,
commercially as an engineer at Mozilla,
as well as for a variety of open source projects.
He also maintains a strong interest in logic programming,
mainly as a result of his doctoral thesis.
Ryan, welcome to the show.
Thanks for having me.
I'm super excited that you're here today.
We're going to talk about Python,
but in a place that you typically don't find it,
in the browser, right?
Yep.
Yeah, there's a lot of cool projects out there,
and I'm a huge fan of the one that you've been working on.
Before we get to that, though,
let's talk about how you got into programming.
What's your story?
Sure.
So I think it was quite a while ago now,
and I actually started programming in JavaScript in high school way back in the day
for a class, right?
I was never the kid that sort of always grew up with computers.
I very much just got into it for a school assignment
and pretty much fell in love straight away.
You know, I just fell in love with the ability to have a thought
and kind of capture that in some concrete form
and have it executed and have effects on the world.
It's pretty amazing.
And actually, that kind of subject changed my entire career path.
I'd always planned to go into the sciences,
and I had sort of a half a semester of computer programming in high school
and changed my whole career trajectory into engineering and computer science.
I think a lot of us who are in programming had that experience
where maybe we didn't think that we were programmers per se, right?
But then you get some experience where then you're like,
wow, this is really amazing.
This is really way more rewarding than a lot of other stuff.
I had that experience in math.
I studied lots of math, and I'm like, but we don't build anything here.
Programming is so much more concrete but creative.
I love it.
Yeah, it's that beautiful combination of kind of creativity in the mix, right?
The discovering and creating is pretty cool.
Yeah, sure.
So how did you get to Python?
I was handed Python as a requirement for an internship I was doing in university.
I went to work for a company that was doing engineering simulations.
So I think like a lot of people, I sort of come to Python because it has really strong tools for doing data analysis and visualizations.
So that was in a kind of typical intern project kind of way.
It's kind of like, here is a relatively well-scoped project, and you're using this language, and off you go.
But I just found that it really fit my head in a way that a lot of other languages that I've been dabbling around with didn't.
It was just kind of super easy to get up and running and dive in, and I found that the more time I spent with it,
and the more I came to understand the language and find all of these more and more interesting corners of it,
it just kind of grew from there.
And sort of since from that point, it's always been my kind of go-to language for side projects or projects where I'm not kind of locked into a language for other reasons.
Yeah, I think Python has this rare combination of being very capable and yet very simple.
There's a lot of capable languages, and there's a lot of simple languages, but they don't often intersect.
So that's great.
I've worked in some interesting languages like Prologue, and there's a language called Mozart, which is like a logic programming language.
And they're incredibly powerful, but yeah, they don't have that just clarity that Python has.
So before we get to talk about your project, you work at Mozilla, right?
That's right.
I'm a huge fan of Mozilla.
Firefox is definitely my browser.
Cool. Good to hear.
Yeah, I try to spread the word.
What do you do there?
I used to work with the cloud services team.
We're not an independent team anymore.
So when I first started out, I was working on the web servers behind Firefox Sync.
So doing sort of data storage and RESTful JSON APIs.
They're all Python-based servers.
So it's pretty standard kind of Python backend web gig.
Through a series of kind of mergers and reorgs and so forth, I actually wound up on a project that's operating in Node.js.
So my time these days is spent on Firefox accounts, which is the login system for Sync and for all of the new kind of cloud-based services that Mozilla is doing.
And it's a 100% JavaScript shop, that team.
So it's an interesting change of pace.
But it does leave me kind of itching to get more involved in Python stuff on the side.
Yeah, that's really interesting.
I think that's a good lead into your topic because it's like Python and JavaScript, those two worlds colliding, right?
Right.
So maybe at a really high level, we could just talk about your project, PyPyJS, and then we could talk a little more specifically about it.
Sure.
So at a super high level, I like to describe it as an experiment in building a fast and compliant Python environment for the web.
So concretely, it means like I want a Python interpreter that will run on top of JavaScript.
So anywhere you would go that you would want to do something in JavaScript, you should be able to take Python to that platform as a first-class citizen.
And obviously, a big place where JavaScript shows up is in the browser.
And the browser is just dominated by JavaScript.
There's really not a lot of other games in town if you want to do scripting in the browser.
But I'm also kind of interested in how it might fit on the server side in Node.js or other JavaScript frameworks as well.
But basically, being able to take other languages such as Python and put them anywhere where JavaScript is currently claiming as its home turf.
Yeah, that's great.
I mean, there's definitely been people talking how JavaScript is becoming the new assembly language of the web.
And we'll get into that.
So the way that I heard about your project was you gave a really excellent presentation at PyCon 2015.
Oh, thank you.
Yeah, yeah, definitely.
What was the title of it?
I forgot.
PyPy.js.
What, how, why?
I think from memory.
Yeah, that's perfect.
Yeah, I think you're right.
And it's on the YouTube channel for PyCon US 2015.
So I'll be sure to link to that in the show notes.
One of the ways you opened your talk was you talked about how Mozilla loves the web.
And not just the technology of it, but sort of the whole idea and the culture of it.
Can you maybe talk about that a bit?
Sure.
So we have this saying at Mozilla.
And this was sort of one of the things that really inspired me to get started on this project.
We like to say like the web is the platform.
Well, this is kind of the world we want to live in is where the web is the platform.
And so the web, not necessarily from a technology standpoint, but from a kind of computing platform standpoint, is pretty unique and pretty amazing.
It's a platform that doesn't have any kind of all-powerful gatekeepers.
It's a platform where anyone can kind of show up and start participating.
You can publish stuff on the web without having to be locked into a particular hardware platform or a particular set of tools.
You can show up and start consuming stuff from the web without any of the similar sets of restrictions or requirements.
And so at Mozilla, we kind of see the web as a really powerful tool for empowerment and participation in the global kind of information economy.
Of course, the web, you know, from a social standpoint, that's all very well.
But when you get right down to the technology of the web, it doesn't necessarily tick all of those boxes, right?
In a sense, the technology stack of the web is a little bit kind of restrictive.
It's all kind of JavaScript and HTML in the browser.
There's kind of a lot of nice sociological aspects of the web that are really unique and really valuable.
And there are some technical aspects of the web that are not always to everyone's liking.
So that's an interesting platform in that respect.
It really is.
You know, you could have sold me on this idea with that statement in 2005, no problem.
But I think it's even more relevant now because we all have these app platforms and these proprietary APIs, you know, iOS, Android, OS X.
There's just more and more walled gardens and proprietary stuff.
Walled gardens and app stores.
Yes, absolutely.
And there's, you know, good financial reasons for that, right?
Absolutely.
And not just financial reasons.
Like curated content has a lot of user value and security value and stuff like that.
So the web has not completely undeserved reputation as a little bit of a security badlands because you don't know whose code you're running half the time.
You just could have got to trust that the browser is going to do the right thing for you.
That's right.
And that's, I think, partly why the JavaScript component of the web is so crucial because if it involves Flash or ActiveX or some other, you know, high privileged C-based plugin, it's not the web.
Not really.
Absolutely.
And you can't really trust it, right?
Absolutely.
And so your mission to bring things like Python to the web have to involve JavaScript somehow, right?
Right.
And I think there's been a long history and pretty much since the web has been a thing of trying to bring more kind of native platform functionality to the web.
You know, you mentioned Flash.
They've had, you know, ActiveX or Java applets, you know, way back in the day.
And the web has a really long history of trying to compete with native platforms by bringing native platform technologies kind of into the web by expanding the scope of the technology stack of the web.
As, you know, one after another have been kind of failures.
They fail to stick for any significant length of time.
And a lot of the time it's because of aspects like that security, right?
You're just kind of opening up too many holes, too many opportunities for things to go wrong.
Whereas if you take kind of JavaScript as your base level, then you can build on this sandbox that's had 20-odd years of, you know, figuring out how to get that right and build that model correctly.
So if you take that as your foundation, you kind of get a lot of the good security stuff and the lessons learned from the web for free.
Yeah, I totally agree.
And it's also worth noting the birthplace of JavaScript is Mozilla, right?
Well, it wasn't Mozilla back then.
But yeah, sort of the heritage of JavaScript sort of comes from the same place as Mozilla.
Yeah, absolutely.
So when I think about JavaScript frameworks, there's the kind of more lightweight ones where it's, I'm writing, I'm taking some JavaScript API and I'm writing some more JavaScript code to write maybe a single page application.
So I'm thinking of like Angular, you know, those types of things, jQuery, so on.
And then at the other end, there's an idea of bringing more stuff into the JavaScript world using the concept of something called asm.js.
Can you speak to that a little bit?
So asm.js is a funny beast.
It's sort of this tiny little subset of JavaScript, which if you look at it in the right light and you kind of suspend disbelief for long enough,
inside of JavaScript are the semantics of a 32-bit CPU, essentially.
So JavaScript has, for what is a high-level language, JavaScript has a surprising number of kind of really low-level features.
Like it's got 32-bit bitwise integer operators and things like this.
Asm.js basically came out of an attempt by a very smart fellow named Alonza Kai to compile C code to the web.
And through a series of kind of experimenting with different ways of doing that,
came to realize that there was this subset of JavaScript that you could target that implemented the semantics of the machine more or less directly,
that you could then take C code and transliterate into this subset called asm.js.
And because of the kind of browser wars and the JavaScript performance wars that had been going on up to that time,
it also happened that that stuff would run pretty fast.
So he likes to describe it as discovering a kind of low-level 32-bit assembly language hidden inside JavaScript,
and I think that's pretty apt.
Very cool.
Have you seen the video or the PyCon presentation, Birth and Death of JavaScript by Gary Bernhardt?
I was present in the audience for that talk, and I loved it.
That was an amazing talk.
Those of you guys out there listening who haven't seen this, I'll put it in the links.
And I've referenced it before on the show.
But he did a really good job of showing off Asm.js and basically bringing anything that can be written in C to the web.
And one of those was like a pretty high-end video game like Quake or something like this, right?
They're doing sort of more – within half the speed of Native, I think they usually get these things running.
It's pretty amazing.
Yeah, very, very amazing.
And if you say high-end game like Quake, but I sort of sense that presentation has gone past,
I think they actually have been launching with even more kind of –
the very latest and greatest gaming frameworks are starting to ship support for generating Asm.js JavaScript natively.
That's really amazing.
Okay, so there's this idea of we can take C code and we can compile it not to machine instructions,
but we can compile it to a special subset of JavaScript that then can be translated by the browsers to execute very fast.
Correct.
So how does this relate to your project, PyPy.js?
The PyPy.js is sort of a coming together of what to me are two amazing feats of engineering, right?
And one of them was Asm.js and Emscripten and this whole idea that you can treat the web more or less like a compile target,
as if it were a native CPU.
The other half of that, of course, is the PyPy Python interpreter,
which I think you had an episode on not too long ago.
That's right.
That was show 21.
And PyPy, the interpreter, does some really amazing stuff.
It's incredible.
So one of the things that I think actually was talked about a little bit in that episode,
but one of the things I really like about PyPy is they weren't just focused on writing a really fast Python interpreter.
They were focused on creating a platform for experimenting with interpreters.
So it seemed to me the perfect kind of opportunity to learn more about both of these projects.
I'll learn more about PyPy.
I'll learn more about Emscripten and Asm.js.
Because PyPy is structured in such a way to make it really easy to dive in and kind of change bits in and out and play around with it,
I had this idea that I could get in there and like, well, I'll just find the just-in-time compiler backend inside PyPy,
and we'll just change it to emit Asm.js instead of, you know, x86 or ARM assembly,
which in a lot of projects would be kind of a terrifying proposition.
But because of the amount of kind of just good abstractions and so forth that are inside PyPy,
it actually was quite approachable as a problem.
So the coming together basically looks like the PyPy interpreter toolchain run through the Emscripten compiler toolchain
so that you get a version of it that is compiled to JavaScript.
And then at runtime, it uses the PyPy just-in-time compiler infrastructure
to take your Python code that you're running and spit out Asm.js at runtime,
in the same way that a native PyPy interpreter would spit out kind of specialized native code for your application.
So very much treating Asm.js as if it were just another dialect of some assembly language for a machine.
That's so amazing.
How long did it take you to get a proof of concept that was actually doing something?
I think my very first proof of concept without the just-in-time compiler part came together pretty quickly.
I lost maybe two weeks trying to figure out that Emscripten was actually assigning file descriptors
at different numbers than they usually are in Unix.
So like standard in is usually file descriptor zero, standard out is usually file descriptor one.
And it was giving them different numbers because it had never mattered before.
But PyPy really didn't like it.
So there are a few kind of just like toolchain compatibility problems like that.
But sort of once I got that figured out, right, is sort of just compile it as if you were running it through a cross compiler
and you get it up and running pretty quickly.
The JIT backend, I probably worked on, on and off for the better part of a year.
I conceived this scheme at the end of PyCon Australia probably three years ago.
And then I presented the kind of final first version of it at the following PyCon Australia the next year.
In your presentation, you said your goal was really to bring support to the web for Python.
And in your mind, that had to sort of meet three criteria, compatibility, performance, and webbishness.
Can you talk about that?
I wish I'd found a better word for that.
I think that's the official word, yeah.
Yeah.
So this is not the first time people have tried to do Python-y stuff on the web.
I think I even said in the talk it probably won't be the last.
But it's one of those classic cases that you get in software a lot of a triangle where the three goals are kind of more or less opposed to each other.
So you've had languages that compile to JavaScript.
That's a thing that we know how to do.
And we know how to do that relatively fast.
As long as your language semantically maps to JavaScript pretty well.
Python doesn't.
So in Python, you have big ints by default.
You have a variety of checks built into your arithmetic operators and a few things like that.
So the semantics of Python are actually quite different to the semantics of JavaScript, even though they look very similar.
On the one hand, it's easy to make something go fast in JavaScript.
If you're not too far from the semantics of JavaScript.
On the other hand, we know how to compile languages from one to the other, right?
It's sort of a Turing completeness argument.
So if we wanted to get a really compliant Python interpreter, you know, one that implements all of the edge cases of CPython up and running on a different platform, you know, it's not implausible that we can get that up and running.
So it's interesting to see how fast we can do it.
So there's kind of two relatively opposed goals there, right?
Being fast and being – I like to say being Python, right?
But doing all of the things that Python does, not kind of looking a bit like Python, but having JavaScript's number model and things like that.
Right.
Exactly.
There's all these – you know, when you talk to people about JavaScript, they're like, oh, there's all these JavaScript gotchas you have to be on the lookout for.
I don't typically hear people go, look out for all the Python gotchas because they're lurking everywhere, right?
Right.
Python's usually got your back.
This episode is brought to you by Hired.
Hired is a two-sided, curated marketplace that connects the world's knowledge workers to the best opportunities.
Each offer you receive has salary and equity presented right up front, and you can view the offers to accept or reject them before you even talk to the company.
Typically, candidates receive five or more offers in just the first week, and there are no obligations ever.
Sounds pretty awesome, doesn't it?
Well, did I mention there's a signing bonus?
Everyone who accepts a job from Hired gets a $2,000 signing bonus, and as Talk Python listeners, it gets way sweeter.
Use the link Hired.com slash Talk Python to me, and Hired will double the signing bonus to $4,000.
Opportunity's knocking.
Visit Hired.com slash Talk Python to me and answer the call.
It's usually got your back.
That's right.
And so you can't just, for example, translate floats in Python directly over necessarily to just run as JavaScript or numbers, especially integers, like you say, over to JavaScript numbers.
So there's all that.
That's the compatibility bit, right?
Right.
So, yeah, that's the compatibility aspect.
And things like parts of the standard library as well.
You know, do you have all of a full implementation of the sys module, including get frame and kind of the debugger and all of these things?
And it's interesting.
It's sort of like, well, I don't really use those things that often.
You know, I don't really depend on integer overflows escalating to big ints by default that often.
Or, you know, I don't use sys get frame that often.
But it's interesting.
As soon as you start trying to run, like, whole programs on Python, like, you'll find that, oh, you use some library somewhere that uses some library that happens to use one of those things, you know, in some corner of its API.
So even though they seem like edge cases, they're often not at a whole program level.
Right.
They're not necessarily edge cases for small, simple things.
But if you're going to really write major apps there, then all of a sudden those become things you have to worry about, right?
Right.
And something I'm very interested in is less about kind of writing Greenfield's Python code for the web.
Because, you know, personally, I am not that unhappy writing JavaScript.
I'm interested in taking Python code that you've written and putting it on the web without having to do a whole lot of work.
And the compatibility aspect really is really important for things like that.
Right.
Like script source equals some file dot py.
Something like this, right?
Right.
Well, so, something like that.
Yeah.
But it would be, you know, taking something that you've already written and being able to put it on the web and make use of all of the fun distribution aspects of the web without having to do a whole bunch of busy work to make it run in JavaScript.
Right.
Absolutely.
So it sounds to me like pypy.js completely nails the compatibility part because it's effectively PyPy.
Is that right?
Correct.
So the fun thing about basing this on PyPy is we essentially get the compatibility part for free because they put an amazing amount of work into being a really, really compatible with the Python language reference, cPython.
And then you get to the performance.
And performance has multiple aspects.
All right.
One is I've got my code loaded up.
And now I can run this and it's doing XYZ iterations per second with such and such amount of latency.
The other part of it is I visit the web page and then it responds.
Right.
Yes.
Maybe we could talk about those a bit.
Absolutely.
PyPy being interpreted with a just-in-time compiler generally targets kind of steady state performance of your application.
So, you know, you load some code and you run it for a while and the JIT has a chance to kick in and generate a specialized code and you get very nice kind of steady state performance for the code that you're running.
And I think, again, the combination of PyPy and Asm.js means we can kind of get a lot of that more or less, I don't want to say for free, but, you know, easily in a JavaScript environment.
What we really don't get and what is actually really important on the web, as you say, is that kind of initial startup experience.
Like the time between hitting a web page and being able to interact with it is even more critical on the web than it is in sort of places where you'd normally find Python.
And PyPy.js, realistically, right now does a pretty terrible job of it.
It's only so fast you can start interacting with the page and you've got to load sort of megabytes worth of JavaScript to even start running your code.
Right.
And so right now, if you look, I think, you know, you have a really great page, arewepythonyet.com, that talks about a lot of cool stuff.
We can get to that.
But on there you have the download size, both, you know, off of disk and sort of maybe on the wire, the GZIP representation on the wire.
And on disk, it's like it was a little higher, but you guys have gotten down to 15 megabytes, right?
Yep.
And what is that, like five on the wire?
Yeah, I guess down to comfortably on a single digits on the wire.
I think five is probably about right.
Which for a native application is not terrible.
Like that's kind of in the ballpark.
But for stuff that you have to load up in order to interact with a web page, it's not great.
Yeah.
Typically, we talk about minification of our JavaScript and it's smaller, usually smaller than five megs.
Yes.
Usually not measured in megabytes, let's be quite frank about it.
That's right.
That's huge.
What is that, 100K?
No, no, that's 20 megs.
But, you know, there are potential solutions, right?
I mean, there are CDNs, extreme caching.
Right.
Could alleviate this somewhat, right?
The core runtime parts could somehow be sort of shared across sites.
Indeed.
I think there's actually still a lot of low-hanging fruit in that size, right?
So that size includes, for example, an entire Unicode character database, which is built into the interpreter in PyPy.
It includes all of the standard library modules that are built into the interpreter, whether you're going to use them or not, and things like that.
So there is definitely some scope for making that smaller.
And I've made more minimal builds of it that are several megabytes smaller.
But I don't think it's ever going to get down to, you know, tens of kilobytes in size.
This is pretty unrealistic.
If you go and download CPython from python.org, compressed, that's like 22, 23 megabytes, right?
And so, you know, if you're going to ship the whole runtime, the implementation, the PyPy implementation, that's kind of a lower bound in some way.
Exactly.
But that's not always a problem.
You know, if you look at this as a way to write all of your kind of interactive website scripting in Python instead of JavaScript,
like you were saying, kind of script lang equals text Python,
that's a tremendous amount of overhead to pay before you can do any of the interactivity on your site.
But if you're doing something like shipping a kind of more appy experience that you've built in Python,
kind of bootstrapping into, in the same way that you might if you're loading up a game on the web
or like some sort of more heavyweight application, it's less of a big deal.
So, and, you know, the games that are compiled for the web within script and have this problem as well, right?
You want to get into the game and start playing it.
You've got to download many more megabytes of like image assets and compiled code and all this sort of stuff.
But it's okay for that use case.
And I think that's sort of the point where we're at with the project now where the people that are sort of really showing an interest in starting to use it
are cases where that's going to be less and less of a concern,
where you're okay to kind of wait for a few seconds while you're bootstrapping into a fairly elaborate experience.
Right.
And like I said, once you get that loaded up the first time, you can cache it.
So if you're able to wait five seconds or whatever it takes to download that, then, you know, maybe it's possible.
So games are obviously one concept here.
Maybe if you're doing some, like I said, single page app thing where you kind of load it up and then you just stay there for a long time.
Right.
So I had some preliminary kind of interactions with the Jupyter and IPython people trying to get this plugged in.
And in a sense, it actually went really well.
They've got a really excellent infrastructure for plugging in different language backends and different, you know,
they can do Ruby or whatever plugged into their infrastructure.
So that, in a sense, kind of works.
Like, hey, you can plug in, plug Pypy.js and run this completely client side.
Of course, then, what that's missing is all of the fun libraries that you might want to use with that.
So if you want to do scientific Python on the web, scientific kind of programming on the web in Python,
you're going to want things like NumPy and sort of all of the analysis libraries that typically go along with that.
So I'm still interested in kind of pushing ahead and seeing where we can go with that.
But I think it becomes more of a library problem at that point, making sure you can actually ship that,
all of the details of that experience that people would expect and not just the Python interpreter itself.
The Python interpreter itself is actually probably the easiest of the bits, right?
Because then you've got to get NumPy and SciPy and Scikit-learn, and a lot of those have C extensions,
which then you've got to get into Asm.js again.
Right.
There's a lot of nuances that people building those libraries have never thought of, right?
Exactly.
And so in theory, that's all possible, right?
It's all the same principle of we've got code for a native platform where we can retarget it for the web.
But it's never quite that straightforward in practice.
Do you want to talk about arewepythonyet.com a bit?
Sure.
Yeah.
So what was the idea?
You said that was inspired by some sort of performance metrics at Mozilla historically.
The domain name in particular is a little kind of inside joke at Mozilla, right?
When the browser performance wars were really heating up, folks that were working on JavaScript performance at Mozilla needed a place to put their public JavaScript benchmarks.
And so they created this site called arewepastyet.com, which you could go to and just kind of look at all of the different JavaScript benchmarks that were running in the different engines.
And it kind of became a little bit of a thing at Mozilla.
So when people were talking about, oh, Firefox is a real memory hog and we need to do something about the memory usage of Firefox, you know, they started up a metrics tracking site called areweslimyet.com.
We were starting to really measure our community contributions and figure out, you know, how many contributors we're getting from different parts of the world and how big the community is growing.
Like they set up a metrics tracking website called arewemillionyet.com.
And sort of this ongoing habit of just trying to picking things that we really want to make a big difference to and measuring them and putting it up somewhere for everyone to see.
The other folks that are really keen on, you know, doing public performance measurements are the PyPy team.
So the benchmarks that are on arewepythonyet.com are all taken from the PyPy Speed Center, which I haven't managed to get all of their benchmarks running, but I've got a good subset of them.
But yeah, sort of both places, I guess, have a really strong history of being really upfront and public about all of their performance metrics tracking.
I think it's a wonderful philosophy to try and follow in the footsteps of.
Yeah, that's great.
And you go there and you can really quickly see where the performance is really good and where it might lag behind.
And one of the things I thought was really interesting is there are plenty of places where PyPy.js is literally faster than CPython.
Yep.
All of that magic, of course, is due to the PyPy JIT.
And if you change the comparison to compare it to a native PyPy, of course, it's going to be uniformly slower.
But yeah, I was pretty amazed.
That's sort of one of the challenges I set myself when I was starting out on this little experiment was to say, look, you know, is the combination of Asm.js, you know, being almost as fast as a native platform and PyPy being kind of consistently faster than CPython.
Can that in combination mean we can actually run Python benchmarks in the web faster than your standard Python interpreter?
And it's pretty astonishing to find out that that is, in fact, the case.
This episode is brought to you by CodeShip.
CodeShip has launched organizations, create teams, set permissions for specific team members, and improve collaboration in your continuous delivery workflow.
Maintain centralized control over your organization's projects and teams with CodeShip's new organizations plan.
And as Talk Python listeners, you can save 20% off any premium plan for the next three months.
Just use the code TALKPYTHON, all caps, no spaces.
Check them out at CodeShip.com and tell them thanks for supporting the show on Twitter where they're at CodeShip.
Yeah, I was, when I pulled this up, I was like, all right, let's see how many times slower it is.
Is it like something we could live with because it's going to run in the browser?
And I was like, wow, okay, that's really cool.
It's also fun to click off the D8 interpreter in those comparisons and get just the SpiderMonkey engine.
It'd get a little bit more times faster than CPython as well.
That's awesome.
And of course, D8 is Chrome and SpiderMonkey is Firefox, right?
V8 JavaScript engine, yes.
Same one that runs Node.js, right?
So very interesting.
So the performance story is really, really good on the execution side.
But then you also have the stuff about sort of the startup and the download size and so on.
Right.
So there's no point, you know, trying to hide any of that stuff.
There's a whole tab there about startup overhead.
And it does indeed measure the download size in megabytes.
And you can, you know, it's even got a little feature there.
You can actually click to just test how long it'd take to load up on your machine.
What's interesting here actually is the download size is one thing.
Like that's kind of understood and easy to grasp.
But there's also a graph of the time it takes.
Like once you've downloaded that, how long does it take for the JavaScript engine to actually kind of initialize this interpreter?
And that actually is on the order of a second or two just by itself, right?
Kind of taking all of that JavaScript code and parsing it and loading it up into the browser is actually, you know, makes the browser work really hard.
Yeah, it's a large quantity of JavaScript that you can understand, right?
So, you know, you work at Mozilla.
Maybe you can pull some strings.
Is it, would it be possible to take projects like this, like Asm.js and PyPy.js and have them sort of certain versions shipped with browsers?
So like when I load up Firefox and I hit a page and it says, give me PyPy.js, it goes, we already have that downloaded.
Let's go.
I'd love to think I could pull those strings.
I don't know if I really have that power.
But I mean, the thing about the web, and this has been a general principle of the web for a long time, right, is no one browser vendor really has the power to do that by themselves.
So, you know, we could kind of put this in the browser and say, yeah, like, you know, we want to treat Python as a first class language and we're going to special case it in the browser and get a really good experience for Firefox users.
But the individual market share of us or any other browser who tried to do that is probably not sufficient to, for people writing websites to be able to depend on it, right?
Yeah, absolutely.
And the most important use case of that would be on mobile.
And then that's an even harder sell.
Absolutely.
What's really interesting and sort of segueing out of that a little bit, what you really need in order for something like this to kind of take the next step and to get over some of these kind of fundamental problems of code size or load time or whatever is, you know, if you can't find a way to hack around that in JavaScript, which for some of these things we really can't.
You kind of need all of the browser vendors to agree on a way forward and sort of everyone's got to ship something better together.
And one of the amazing things that's happened coming out of Asm.js and experiments like the Games initiative that have kind of proven that that's a really powerful thing is you're actually seeing all of the browser vendors sort of getting together and talking about, well, like, what's next after Asm.js?
Like, how can we make this better?
How can we really commit to this as the web rather than any particular browser platform?
That's really, I mean, that's the way the web works, right?
That's the way the web is successful anyway.
Like you said, people have tried to jam their way through, you know, like, IE6 type stuff that happened back in those days, but that's not really what sticks.
You've got to have a good story for getting from where the web is to where you want the web to be.
And Asm.js kind of really came in and provided that.
It's like, all right, we want to treat the web as a compilation target.
And here's a pretty okay way of doing it.
All right.
You know, we can get to that point.
So now you've got impetus to actually make that better.
Yeah, a foundational idea to start from that everybody can start to come from.
Right, exactly.
You know, people had traditionally talked about JavaScript as the assembly language of the web, sort of conceptually, but Asm.js is kind of making that concrete in a very real way.
The name of the kind of project that's spinning up out of that discussion is called WebAssembly.
And it's very much targeted at being like, here's an actual assembly language for the web, right?
It's kind of like, take that idea that Asm.js proved is plausible, you know, doing this assembly language on top of the semantics of JavaScript, but outside of the limitations of necessarily, you know, 100% backwards compatibility with JavaScript.
So it's gone from kind of like, yeah, this should be the assembly language of the web and kind of like, this is what it looks like, to suddenly taking that idea really seriously.
Yeah, the future is going to be super interesting when that sort of takes hold.
Absolutely.
And I think, you know, a lot of these problems, like the load time of the JavaScript and the download size, like they'll get better for free when that technology starts to kick in.
They won't go away, but they'll get better.
Right.
And a lot of the stuff that you sort of shipped from your site that would have been JS files or whatever are now binary, right?
In that spec?
Right.
Right, exactly.
So Python 2 or Python 3?
Currently Python 2.
Mostly by default.
But that describes me as well as this project.
I'm one of the laggards.
I'm sort of slowly coming to the realization that I'm falling behind and that I still use Python 2 by default everywhere.
That said, one of the neat things about this, right, is the PyPy team are working really hard on their Python 3 compatibility.
And so it's a pretty straightforward prospect to kind of ride along the coattails of that work and ship Python 3 in the browser just as easily as you can ship Python 2 on top of this technology stack.
Yeah.
That's great because you're just taking literally the PyPy interpreter as it is and sending it through this tool chain.
Whatever support PyPy has for Python 3 more or less is the support you have for it, right?
Right, exactly.
It sort of just comes out the other end.
And that goes for all of the other languages that people are experimenting with on top of the PyPy tool chain as well.
Yeah.
I read an interesting article just this week talking about Python 2 versus Python 3.
And, you know, there's a lot of practical reasons that a lot of people are still doing Python 2.
And this article was called Planning an Early Death for Python 2.
And it had some really interesting ideas.
I'll be sure to link to it.
But a lot about the way, like, people culturally think and talk about Python.
They were attempting to change that there.
And I thought it was an interesting idea.
Anyway, I'll link to that for everyone.
Yeah, cool.
For what it's worth, I've really noticed a change in the community in the last year or two where it's kind of Python 3 is – it seems to have kind of broken through in a sense.
Like, people might look at you a little bit odd if you're not taking Python 3 seriously.
Or at least it's something I've noticed anyway.
Yeah, I've definitely noticed the same thing.
So are there notable deployments or places that we could go find that PyPyJS is being used yet?
Or is it too early for that?
What's the story around there?
I think it's broadly too early for that.
I mean, you can go to the PyPyJS website and muck around with it.
It's sort of an interesting project for me.
I was sort of in the position of doing a project that I don't really want to build anything with.
Like, the project itself is the point of it for me.
So I've been talking to a couple of different people who are interested in trying it out as part of an actual product.
In places where you might use something like IPython or sort of one of the other in-browser Python environments currently.
But I don't think any of those are live yet.
They've certainly – I've got people showing up and chipping into the project and trying it out and experimenting with using it in their stuff.
But I don't think anyone's quite put it into production just yet.
Right.
And if you want to check out the code, it's at github.com/PyPyJS.
So pretty easy to find, right?
Do you see maybe a use case in some of the more, like, offline app-ish HTML5 apps?
So I'm thinking, like, Firefox OS, like, PhoneGap-type platforms?
Absolutely.
So that's one of the places where the startup overhead of this whole infrastructure will be less of a problem.
Because you can list, you know, the files to be downloaded as, like, your app manifest.
And you kind of get better control over the caching and things like that.
So it probably makes more sense in that sort of contained experience of an app than it does as, you know, a bit of scripting on the open web somewhere.
Sure.
Yeah, that definitely seems like it.
What about Chrome OS?
To be honest, I haven't thought about it.
I'm not sure you're allowed to promote them, but, you know, those do have sort of prepackaged offline manifest type stuff, right?
So it seems potentially would be good.
And particularly as the web kind of comes to terms with that packaging of applications and more standards sort of start to emerge around it, then, you know, ideally, you don't have to make that distinction like, oh, PhoneGap or Firefox OS or Chrome OS.
It's kind of like, oh, this is how we do packaged apps for the web.
And Python has a place in that ecosystem.
That would be a really great outcome.
That would be absolutely a great outcome.
There are other sort of Python in a browser type frameworks or attempts or projects.
And you listed something like at least nine in your talk.
You know, two that are pretty popular come to mind.
One is Sculpt.js.
I had Brad Miller on one of the earlier shows to talk about that.
And another was Brython.
And those seem more like, you know, script lang equals Python sort of type of frameworks.
But they make tradeoffs to make that possible.
Can you compare those a bit?
Exactly.
I'm not super familiar with Sculpt.
But I had a look at Brython in some detail.
And I'm actually quite a fan of the stuff they're doing.
I think they've selected an excellent set of tradeoffs.
It's a different set of tradeoffs to the ones that PyPy.js is interested in.
And the big tradeoff that they make in Brython to get good performance is the number model.
So if you do like a for loop, like, you know, for X in range one to 100, right, that will get turned more or less directly into a JavaScript loop over JavaScript numbers.
And for that use case, it's absolutely fine.
Like, JavaScript numbers will behave correctly.
But if you ever happen to get yourself, get your code into a situation where, you know, you're counting up a little too high or you're dividing by a number that you're accidentally dividing by zero or something like that, then you're not quite getting the semantics of Python's number model on top of Brython.
For the script kind of text equals Python use case, we're actually, like, writing new code.
You can write that code for Brython, and it's a really excellent set of tradeoffs.
Because you know to watch out for those things.
But if you were to take a, just, you know, get some Python code off the shelf and try and run it through, then, you know, you've got more chance of it's going to trip you up.
So, of course, what you get in return for making that tradeoff is you get a much better kind of startup experience, much lighter weight integration into the browser, a lot less JavaScript to load.
And in some cases, sort of a tighter integration with the browser and with the native DOM and stuff like that.
Right, because they can lean more heavily on JavaScript itself.
Correct.
So, from Brython's perspective, it's targeting JavaScript.
So, it can interact with JavaScript objects and this sort of thing.
From PyPyJS's perspective, it's kind of targeting this weird C-like platform that happens to run on JavaScript, but you can't really see.
It's difficult for it to see the native JavaScript objects and so forth in a sensible way.
You did show a way that you could interact with libraries such as jQuery and the DOM and so on.
Absolutely.
From PyPy.js, right?
Yes.
That works.
That actually works surprisingly well for what it is.
And the way to think about that is a lot more like C-types or CFFI.
Like, it's more like you're doing a C-binding to the objects in your JavaScript world than interacting with them directly.
And on top of that, you can build a nice little API that exposes them as Python objects.
But when you go to use that bridge between PyPyJS and the native JavaScript environment, you're going through basically indirecting through a C-API.
That makes it something of a challenge to do fast and to do things like cross-language reference cycles, for example.
You know, if you've got a Python object that refers to a JavaScript object that refers back to a Python object, it can get a bit weird.
But for a lot of stuff, it works surprisingly well.
Oh, yeah.
Because, you know, maybe you don't need super complicated interaction.
But, like, I'd like to grab something out of this text box and then display it over here in this, you know, select or whatever.
Yeah.
For things that are, you know, strings and things that are immutable or things that don't really have references back into Python, it's fine.
No dramas at all.
One thing that I was thinking about as I was thinking about your project was, you know, you're talking about taking Asm.js and focusing on that.
And compiling down to this subset.
And there are other projects that both describe Python sort of more fully and JavaScript more fully.
So, for example, TypeScript on the JavaScript side, which is like an optionally typed JavaScript type thing that AngularJS has adopted, right?
And then the type hints that are coming in Python 3.5.
Do any of those types of things help here or does it just not really matter?
I think the way this particular kind of collection of technologies operates is at far too low a level for those to make a difference.
I think it would be really interesting to try and do higher level correspondence between those two things.
But I think it would be a very different size and shape project.
Yeah, it's not something you could shoehorn in, is it?
Right.
No, not at all.
Yeah, for all of the fun of kind of having it eventually run on JavaScript, actually, the experience of building it feels a lot more like building something in C.
And the JavaScript kind of just appears as a platform underneath it.
Because you're using the same tool set that they use for C, like LLVM compilers and things like that, right?
Right, so classic compiler tool chain that just happens to spit out JavaScript at the other end.
Yeah, it's like saying, well, when I'm doing C++, how does stuff happening in machine language and assembly, you don't even think about it, right?
Right.
Same thing here, more or less?
Right.
Exactly.
Well, Ryan, we're kind of coming up near the end of the show.
Do you have maybe some call to actions for people out there?
I'm really interested in connecting with people who want to put this to concrete use.
I think what the project really needs now is kind of be driven by concrete use cases.
So it's past the point of like, oh, experiment with these fun technologies and see what's possible, right?
See how fast we can make it or things like that.
There's still ground to cover, but that's kind of the real experimental phase of that is more or less done.
And in order to go from experiment to actual product and for me to like go into the website and remove the word experiment from the description of the project,
it really needs to make that jump into something that people actually would be happy to deploy and practice.
I need to talk to a couple of people who are doing like Python shells in the web for a variety of reasons who are interested in trying it out.
But I'd love to hear if people had a Python application that they thought would be really interesting to port to the web in the same way that you might like take a game that you've got written in Python and package it up with Py2.exe and ship it to your Windows users, right?
If you've got something that you think, that'd be really great if I could just take this Python kind of application and port it to the web.
I'd love to just kind of talk through the possibilities.
I think that's, as much as anything, that's what the project needs right now.
People to come and try and put it to use and see what works and what doesn't and we'll fix the bits that don't.
A great call to action and that is what makes projects real, right?
When people use them and they have to hit all the little edge cases and sort of smooth it out.
So it's a real product, right?
Exactly right.
You sort of can't get away from the fact that you've got to download 10 mega JavaScript.
You can wave your hands and pretend that's not an issue when you're doing an experiment but not when you're doing a product.
Right, absolutely.
So another thing that I thought you had as kind of a final call to action in your talk was the concept of filing bugs against the web.
Do you want to maybe talk to that really quick because that's a cool Mozilla idea, I think.
I guess the driving force for this project for me is really that philosophy.
It's the idea that the web should be a first class computing platform.
And a lot of people kind of in the habit of thinking like, oh, the web can't do this.
Or I really want to do this on the web but it's not going to work because X, Y, and Z.
And it wasn't until I actually started working at Mozilla that it kind of occurred to me that actually, you know, the web is a really fluid medium.
There are a lot of people who really care deeply about the web being a really powerful platform.
And sort of invested in the web.
Not necessarily winning but competing with native platforms.
And it's a change in perspective, right?
So if you can realize, I actually like the fact that the web can't do this, I should consider that a bug in the web.
That's, there's one thing to say that another thing to put it into practice, right?
If you come into bugzilla.mizilla.org and you're like, bug, I can't run Python on the web.
Like we may not close it as invalid straight away.
But it's probably, you know, it would be Intel projects like this.
A big chunk of work.
I don't know.
I encourage people to expect more from the web.
It's actually getting more and more powerful as an application platform.
And if it's something that you feel like you should be able to do but can't quite make it work on the web, like find someone at Mozilla or in the Google Chrome team or like people who will take that really seriously.
There are a lot of people working in the web who want it to be just every bit as powerful as a native platform.
So don't just assume that it can't.
Yeah, I totally agree.
That's, you know, that whole way of thinking about the web is why I brought it up because I think it's really cool.
All right, two more questions before I let you go, Ryan.
If you're going to write some Python code, what editor do you open up?
I'm very much a Vim user.
Specifically Vim in an 80 by 24 terminal window.
Sort of, you've got to work pretty hard to encourage me to make it any wider than that.
I don't know if that's actually a productive way to get code written or if I'm just so used to it that I can't cope with anything else.
But Vim all the way.
Perfect.
And then there's, you know, 60 plus thousand excellent packages out on PyPI.
What are some that you think people should know about that maybe they don't?
Interesting.
So a friend of mine from the local Australian Python community, Russell Keith McGee, who's a kind of does a lot of stuff in Django.
He has an interesting new project out called Toga, which is a kind of GUI library, right?
He's working on native applications.
But it's a GUI library with kind of all of the last however many years of experience writing Python under its belt.
So you can do fun things like pip install Toga, and it works on whatever platform you happen to be on.
And you can get up and running with your GUI library.
It's got things like support for iOS and Android UIs and stuff like that.
So I think it's pretty neat.
I haven't had a lot of excuse to play around with it, but I've kind of checked it out out of interest.
So if you're working in native GUI applications in Python, I think it's really worth checking out.
All right.
Excellent.
I hadn't heard about the one, so I'll check it out myself.
Ryan, this is really excellent.
Congratulations on this project.
I think the fact that you brought Python to the web in a way that executes faster than CPython in a lot of ways is really an accomplishment.
And maybe we'll solve this download speed thing, and it'll really take off.
That'd be great.
We'll see how we go.
It's been a lot of fun in any case, so that counts for a lot.
Yeah, it sure does.
Thanks for being on the show.
Talk to you later.
Thanks so much, Michael.
Bye.
This has been another episode of Talk Python to Me.
Today's guest was Ryan Kelly, and this episode has been sponsored by Hired and CodeShip.
Thank you guys for supporting the show.
Hired wants to help you find your next big thing.
Visit Hired.com slash Talk Python to Me to get five or more offers with salary and equity presented right up front,
and a special listener signing bonus of $4,000.
CodeShip wants you to always keep shipping.
Check them out at CodeShip.com and thank them on Twitter via at CodeShip.
Don't forget the discount code for listeners.
It's easy.
Talk Python.
All caps.
No spaces.
You can find the links from today's show at talkpython.fm/episodes slash show slash 32.
Be sure to subscribe to the show.
Open your favorite podcatcher and search for Python.
We should be right at the top.
You can also find the iTunes and direct RSS feeds in the footer of the website.
Our theme music is Developers, Developers, Developers by Corey Smith, who goes by Smix.
You can hear the entire song on talkpython.fm.
This is your host, Michael Kennedy.
Thank you for listening.
Smix, take us out of here.
I'm first developers.
First of all, first of all, first of all, first of all.