Can you write me a Python app for all the major platforms out there?
Oh wait, I also want them to be native GUI applications.
And I want them to work on mobile, so Android, iOS, tvOS, and watchOS, as well as the major desktops.
Also, I need them to appear indistinguishable from the native apps in terms of the way you deploy them.
So a .app on macOS and a .exe on Windows and so on.
What technology would you use for this if you were going to build it in Python?
This week, I'll introduce you to a wide set of small, focused, and powerful tools that make all of this and more possible.
We're speaking with Russell Keith McGee, the founder of the Beware Project.
This is Talk Python to Me, episode 76, recorded October 4th, 2016.
Welcome to Talk Python to Me, a weekly podcast on Python, the language, the libraries, the ecosystem, and the personalities.
This is your host, Michael Kennedy.
Follow me on Twitter, where I'm @mkennedy.
Keep up with the show and listen to past episodes at talkpython.fm.
And follow the show on Twitter via at Talk Python.
This episode is brought to you by Rollbar and Hired.
Thank them both for supporting the show.
Check them out at Rollbar and at Hired underscore HQ on Twitter and tell them thank you.
Hi, everyone.
I want to take a moment and give you a quick update about my online courses.
I just opened Access for my Python for Entrepreneurs course for pre-order.
If you're thinking about taking this course, you should act soon.
Right now, while we're still finishing the content, you can buy it for just $59.
That's $30 off the retail price.
And you'll get early access to the content while it's being developed.
Currently, there's about 3.5 hours of content, and there's more coming every day.
Check it out at talkpython.fm/launch.
The early reviews have been great, so I'm sure you'll love it too.
Now, let's hear about the Beware Project with Russell.
Russell, welcome to the show.
Thanks for having me.
It's my honor.
We're going to talk about so many things in the Beware Project.
Several of my guests have actually recommended the Beware Project as something that's really
standing out in the Python space, and I kind of have to agree with them.
You guys are doing some cool stuff, and so I'm looking forward to talking about it.
Oh, fantastic.
It's been a fantastic ride the last couple of years, and it's over the last sort of 12 months
who sort of just started to get a lot more momentum and starting to – partially because
we're actually getting to the point where we've got something we can really show that's
interesting, not just sort of theoretical.
So, yeah, really enthused to talk about it.
Yeah, great.
Now, before we get to it, let's start with you.
What's your story?
How did you get into Python and programming?
The programming story for me started a long, long time ago.
My father is a mechanical engineer who kind of saw these computer things at the point when
home computing was just starting to become a thing, and he got one of the first Commodore
64s that was available for sale in Australia.
Not with any specific plan of what he was going to do with it, just could see that these computers
were going to be a thing and kind of maybe we should play with them a little bit, which
is something he's always done with technology.
He bought a hideously expensive 3D printer when they were still hideously expensive, just
so he could play with it.
So, yeah, I got introduced to the Commodore 64 when I was 8 or 10, somewhere around that
sort of territory, and I've kind of been just tinkering with them ever since.
It's kind of gone from, hey, I can write this really, really dumb, my sister is ugly, my sister
is ugly, my sister is ugly, go to 10 loop, through to doing a PhD in computer science over
time.
So, my undergrad degree actually wasn't in computer science, it's in physics, but I picked
up all the computing degrees and computing courses I could along the way, and they kind
of encouraged me to come over to the computer science department for my postgrad studies.
So, I did my honours in computer science and a PhD in computer science, and then progressed
from there into working in industry.
My PhD supervisor and I had a very, very frank conversation at one point about halfway through
my PhD, even when we decided mutually that my future did not lie in academia.
And so, yes, from that point, she was happy to try and get me out of academia and into industry,
and I started working professionally as a computer scientist since then.
I was in a PhD program, and I know you feel like you're on the path to just keep going
into academics.
But I feel personally very lucky that I had a broader exposure around that time to see what
was possible in industry, because for every opportunity in academics, I feel like there's
100 opportunities that are just so amazing out there, you know, outside of the confines of
a university and a grant.
I think it's great.
Yeah.
I mean, for me, it was, I don't know, a slightly more practical thing is that I actually like
doing things, not just kind of the theoretical aspects of it.
And I was getting, during my PhD, I was getting very, very tied up in playing around with interesting
pieces of technology with a view to actually doing something, but wasn't actually furthering
my research at all.
So, and it's kind of a precursor to a lot of the stuff that I'm now doing in Bware in some
regards.
Excellent.
Okay.
So where along the path did you come across Python?
I first used Python back in the 1.5 days.
So a long time ago, I was talking like early, late 90s, at which point it was mostly a system
config language, mostly because Red Hat was using it in their setup tools and what have you.
At that point, it was kind of, well, this is an odd language with odd syntax.
I don't want to use odd syntax, so let's not use it.
What's up with this white space thing, huh?
Yeah, well, and that's the thing.
It was, it's the thing that everybody gets caught on.
But as I sort of grew as a software developer, that interesting property became more and more
the thing that I actually value in Python.
So it's one of these things where once you've read enough incredibly badly formatted Java
code, you realize how valuable it is having a language that literally cannot run unless
you get the white space indentation, right?
Yeah.
And so, yeah, it's sort of, over time, it grew from being just the systems language I
kind of have to use to something that I vaguely tinkered around with.
Over that whole period of the sort of mid-90s through to about 2005, when I got involved with
Django, I've had this series of little background projects that I've tinkered around with.
And whenever I wanted to learn a new language or a new framework or a new whatever, it's sort
of the first project is, let's render that in this new language.
And at some point, I got to the point where, yeah, well, hang on, I can do this in Python,
can't I?
Let's build this in Python.
And yeah, this is actually working really.
I like the way this is coming out.
I then got kind of distracted by, hang on, maybe this web thing is actually a thing too.
Oh, I can use this Python for the web.
Oh, look, here's this Django thing, which is brand new.
And 10 years later, I'm still working on the Django project.
So you actually became and have been for a while a core developer on the Django project.
That's correct.
Yeah.
So I was introduced, like I discovered Django because I was poking around.
I had this vague idea that the web was a thing that maybe I should know more about, and maybe
I could use it to solve some of these technical problems that I had.
And I found Django just before they did like the 0.91 release or around that sort of timeframe.
So very, very early days, November of 2005, effectively.
And it was sort of two things really happened.
One, because it was a really early stage project, the sort of the barrier to getting involved was a lot lower than it is today
because there's not as much sort of momentum and inertia around what's going on.
But also because it was written in Python, I come from a physics background.
I have done theoretical computer science.
But like all theory, it's utterly useless for actually doing things.
I never actually formally studied networking or anything like that.
So when I tried to work out how to build a web page with PHP, the PHP website didn't actually help me at all.
I didn't understand what it was trying to do.
But the Django tutorial got me from I don't know how the web really works to, oh, yeah, okay, I can build a website.
And this makes sense now relatively quickly.
And when I hit the limits of what Django could do, because it was written in Python, I could dive into it and I could modify it relatively easily.
I understood what was going on pretty quickly.
So within a month or two of picking up Django for the first time, I was submitting patches that would – if you've got a many-to-many relationship between two tables,
back in the day, you could only go from the model that defined it to the model that it was pointing to.
I needed to be able to go the other way.
So I added that modification into the query language.
And when you start making changes like that, the core team, Adrian and Jacob, offered me the commit bit.
And sort of the rest is history from there.
Yeah, that's really great.
Do you feel like getting started and contributing to Django was easier 10 years ago because it was a newer, less mature, less sort of solidified project?
Absolutely.
I mean, when things are – like 10 years ago, Django didn't have a backwards compatibility guarantee.
There was certainly a preference to not just move things around because, but changing backwards compatible things was at least an option.
There wasn't as much – the community was smaller, so it was easier to get consensus with a bunch of people.
There wasn't the consideration of a million – if we deploy this change, Instagram is going to break because we all of a sudden have changed this API.
So, yeah, 10 years ago, it was absolutely easier to get involved.
There's no doubt about that at all.
Yeah, that's really interesting.
I think that's a big tradeoff, right?
I mean, the value of Django is probably higher now.
But it's just – it's all sorts of projects that have been around for so long.
So, let's talk about a project that's not been around for quite that long, and that's your project, Beware.
Yes.
So, Beware is interesting because it's kind of many things that work together sort of synergetically.
And you said that your project follows the Unix philosophy for tools.
Yes.
Give us a quick overview of what Beware is and then maybe touch on that.
Sure.
Okay.
So, the place where I started with that sort of Unix philosophy is a talk that I gave at PyCon Australia.
I think it was four years ago, but I have to check the exact date.
But essentially, the theory that I was positing at that point was that the Unix philosophy as a process is quite strong.
Now, the Unix philosophy is the idea that you don't build one tool that does everything.
You build a large number of small tools, each of which has a well-defined interface.
So, that – and then the power comes from the small tools being able to plug together really easily or having a natural way of plugging together.
Now, the unfortunate corollary of the Unix philosophy has become that in order for something to be Unix, it must be text-based.
So, yes, you can run LS, and LS you can pipe into sort, and sort you can pipe into unique.
And these are really, really powerful things to do.
But at the end of the day, it's all text.
And there are many, many areas where text is not the natural interface.
And the example that I gave at the time was testing.
If you run a unit test suite or a pytest suite or whatever, what you get coming up on the screen as the user interface is dot, dot, dot, dot, dot, dot, F, dot, dot, dot, dot, dot, dot, dot, dot, dot.
Which is completely correct and completely accurate, but not helpful from a user interface perspective.
What you want to know is which dot is running?
Why did it turn up an F?
When something writes to standard output at the wrong time, which test did that?
How long has my test suite got left to run?
Now, these are all questions that you're entirely in a position to answer or at least provide good guesses for in the case of something like runtime.
But when your only user interface is writing something to console, you have – you are deliberate or you have – you have just hobbled yourself in terms of what it is you can actually display.
So what I was originally positing with the Beware project was that what we can do is build tools that are still Unix related.
Now, they do one thing.
They do one thing well.
But one of those one things can be let's display a GUI that enables me to see all these useful things.
So here is – where we started was with Cricut.
Cricut is a user interface for running test suites.
It shows a tree, which is your test suite.
You press go and it works through your test suite, highlighting the lines of the tree that are currently running as individual tests.
As the test suite is running, you can click on other pieces of the tree and see what tests have run and how long they took to run and what output they generated and whether they passed and failed.
And because you know, well, this is how long the test suite is, this is how many tests there are.
Each test takes an average of X milliseconds or seconds.
Therefore, we can estimate this entire suite is going to take 20 seconds to run.
That's really nice.
I think the visual component of it is just – I am with you.
I don't mind working the command line.
But we have such powerful and beautiful OSs these days.
Like, I feel like we're using one-tenth of what we can in terms of interactability, right?
Yeah.
I've bought a $4,000 laptop with a retina display that's able to render things at a resolution that my eye can't discern.
And I'm using it to show 80 by 25.
It just doesn't make any sense.
Yeah, it totally doesn't make any sense.
But I think that taking this idea of the Unix philosophy applied to like a modern GUI-based set of tooling, that's really powerful.
And that's one of the things that I find really appealing about your project.
Yeah, and that's sort of – like I said, we started – I started initially with Cricket as a proof of concept because, you know, unit testing is a thing that I do a lot of and I enjoy.
And the unit test suite is difficult to get into when all you've got is dot, dot, dot, dot, dot.
So here is a user interface that lets you use, interrogate, manipulate a test suite as it runs.
The other – like I also have – there's another tool called Duvet, which is a coverage tool.
It takes the code results partially from a test suite but also from anywhere else that it's running and shows you graphically which lines of code have been operated rather than giving you, you know, line 17 to 25 were not covered.
Well, what's line 17 to 25?
Oh, it's that for loop.
Yeah, exactly.
Yeah, 94% coverage is not the same as scrolling through your code and seeing, oh, there's a red spot.
Hmm, why is that red?
Yeah, exactly.
But in order to get that capability, you don't have to swallow the whole of an IDE.
You just have to have a mechanism for displaying that isn't 80 by 25 console.
And so one of the things when I was poking around your projects that seemed to be a theme was they seem to be supported on all the major OSs, at least the desktop ones for the most part.
Yeah, so that's the intention anyway.
So originally I use a Mac as my daily developer box.
I do also do work on Linux mostly as a server capacity.
I know I have many, many friends who are using Linux as a desktop.
And, you know, if I write a tool that is useful, I don't want to lock them out.
You know, I don't – my passion for Mac is not because I particularly like Macs.
It's because I like the hardware.
I think it is a usable operating system.
It is a usable platform where I don't have to mess around.
If someone wants to use a different platform, they should be able to.
And my choice of tools shouldn't be affecting their productivity.
I've been from the beginning very actively trying to do this as a cross-platform thing.
Plus also from my perspective, you know, one of these days I'm anticipating that I probably will end up leaving Mac and going somewhere else.
And I want all my tools to keep working, please.
Yeah.
That would definitely lock you in if you spent five years building an amazing set of tools.
But you had to stay on your – whatever OS you happen to have been on at the time, right?
Yeah.
You couldn't possibly leave.
And the thing is, you know, at the end of the day, so much of what these platforms are doing is not platform-specific.
You know, it's show me a menu, show me a button, show me a pull-down.
You know, these are generic concepts.
Originally, it started – like, Cricket is still and was originally built using TK Inter, mostly because it is available cross-platform.
And you can install it saying pip install, you know, pip install whatever, and it just works.
And a lot of where Beware has been sort of hunting for the last sort of three and a half years has been TK Inter.
The major limitation I hit is that TK doesn't have a web widget.
You can't display a web page in TK.
And over time, TK's theming on OS X has gotten worse and worse, but that's kind of secondary to just you can't display a web page.
And trying to write a new widget in TK is just pathologically bad.
So it turned out for me the easy solution was to write a new widget toolkit, ironically.
Yeah, I think there's a lot of cross-platform tool or tool chains, but not too many of them seem to really focus on the truly native component.
And that makes a big difference to me.
I remember running some Java apps and just looking at, like, the file open dialog.
Oh, this just so does not belong here.
What is this thing?
It kind of resembles a file open dialog, but I know it's not.
Like, the hotkeys don't work.
All the various things that you would expect, right?
Yeah, exactly.
And so there are cross-platform widget toolkits that work to various degrees of work.
There are things like Swing in Java, which sometimes looks like native and sometimes doesn't quite.
And then you've got things like GTK, which, yeah, sure, works on Mac,
as long as you're running an X server.
And then you get the loving user interface of an X server running on your Mac.
And then there are things like QT and WX, which are relatively mature and been used for a while and have Python bindings.
But the Python bindings are kind of a secondary thought.
In the case of QT, there's a whole licensing thing that's going on that's just a complete mess.
And they're not as simple as pip install widget set because there's compiled components and libraries.
And you've got to install this 150 meg library.
And you've got to know which one you've got to have.
And it just becomes such a nightmare that the basic thing of, I just want this damn thing to run, gets in the way of getting stuff done.
Absolutely.
The UIs in your apps look really nice.
And they're cross-platform.
What did you do?
I accidentally stumbled upon a couple of things that I didn't realize were actually possible.
So I was looking a long time ago at Pygame and realized that Pygame actually manages to do this native thing without actually having a compiled component.
And the way it does it is because Objective-C, whether you know it or not, is actually a raw C API with some pretty syntax around the outside of it.
And Python gives you this thing called C types, which lets you call any C API without having to compile something.
So with a little bit of Python class magic and Meta class and descriptors and what have you, you can write a class in Python that is a pure Python class that looks and feels and behaves in the Objective-C environment like an Objective-C class.
And then when you go, you look at the next option, you look at GTK.
GTK provides native Python bindings that are actually native Python bindings.
So there you've got two major platforms where you can write native applications using the native APIs with pure Python without having a binary component involved.
And then so from there, it was kind of a, well, okay, that means I've got Coco and GTK, which are the two that I'm sort of covering my bets on.
Can I do this on Windows?
Well, yeah, because there's a Win32 and that's a C API.
Okay, so that works.
What about, well, but why is anyone going to use this new platform?
You know, it's nice that it doesn't have this extra compiled component, but that's not sort of compelling.
What's going to be compelling?
And this is, again, sort of three years ago.
It was like, well, what if I could use these widget sets on mobile as well?
A lot of what I've been doing for the last couple of years is trying to get Python to run on mobile, both on Android and on iOS, so that you can actually then say, well, okay, yeah, I can run this, this similar API or identical API on macOS.
I can run it on GTK on Linux.
I can run it on iOS and I can run it on an Android.
And that's sort of, you know, that's what's been keeping me busy for the last couple of years at this point.
Yeah.
And that's quite the achievement, especially the mobile part.
But I think even, you kind of went through it quick, but I feel like native on the three major desktops is actually a bigger deal than, you know, maybe it sounds at first.
I think if you're going to build an app and if it can look and behave literally like a macOS app or a Windows app or a Linux app, that is a really big deal.
And if you can use Python for that, that's even better.
Now, one of the projects, and I'm sorry, I don't remember the name of the project, but one of the projects you're working on is bundling that up so you can distribute.
You can actually ship sort of a bundled up version of Python with everything it needs to run your app, right?
Correct.
Yeah, that's briefcase.
So, yeah, the idea of briefcase, yeah, is to say, well, okay, a macOS application is basically a directory with some metadata in very specific places.
And historically, giving someone a tool that is written in Python is kind of a, well, you need to install Python, but you need to install it from here and you need to give it your system password.
And then you've got to hope that someone hasn't buggered up the site packages directory and, and, and, and, and, and.
Here's the dependencies.
Install those too.
Yeah, and then you sacrifice the chicken and wave the blood this way and hopefully it works.
Unless you're on Windows, then you better find out where VCVAR is about.
Oh, exactly.
And, you know, because every Windows user knows how to set their path variable.
But at the end of the day, we're now at the point where a 100 megabyte application is absolutely nothing.
You know, okay, 10 years ago, that was a major drama, but now that's just getting up in the morning.
So why not ship the version of Python you actually need with the dependencies you actually need?
And when you run it, it runs it locally.
And so, okay, with macOS, well, yeah, I can do that.
I can template.
I can use cookie cutter, which is a fantastic project from Audrey Roy Greenfield and Danny Greenfield Roy, to like template out what this application needs to look like.
And then your Python script goes here and your dependencies go there and we can pip install into that directory, zip it up, ship it to someone, and it just runs because it's pure Python.
You're giving them the Python interpreter it has to use with the dependencies that they need and it's running natively.
And then from there, you can do similar sorts of things.
I haven't done as much work with Linux because installing apps on Linux doesn't quite work as well.
I've done some exploratory work with Windows and it should absolutely be possible on Windows, possibly to the point of like actually generating an MSI installer that looks like it's a completely native installer.
It just happens to spit out something that starts a Python interpreter, but you never know that.
I think the irony is we in the Python community will be successful on these desktop systems and mobile systems if people don't know that they're running a Python application.
Oh, absolutely.
I completely agree.
So as soon as you know or you can tell what tool someone used to build something, I think you've lost the game.
Like you need to be able to run a piece of software and just be able to give it to anybody and it will just start, it will just run, it will behave the way they expect it to and they don't get any surprises.
Anything less than that and yeah, okay, yeah, it may be very useful in various areas or demographics or whatever, but you're not going to get the mass market adoption if you've got to hold your breath and lift your leg in a funny way on a Tuesday when the moon's full in order for it to work.
Yep, it might work for developers, it might work inside your company, but it's not going to be something you can broadly distribute if that's the story, right?
Yeah, exactly.
But even beyond that, I mean, looking at the developers themselves, this is part of the great lamentation about why I got involved with Beware in the first place.
You shouldn't be in a situation where you have to be this tall to get onto this development project.
If you can't just pick up a tool and use it and pick up a tool and develop it, then you've just set a barrier to entry to anybody helping to make your project better.
So the dream of open source is that anyone can get involved, anyone can make their project better, anyone can contribute a patch.
But if I can't even get a development environment going without 10 years of a computer science experience, then what the hell chance has the random person who has no computer science background got of fixing the tool?
They don't.
It's the being able to contribute open source dream is at best a dream.
Yeah, yeah.
And then you end up with people running to things like JavaScript because, hey, we can all start a browser or something like that, right?
Yeah, exactly.
This portion of TalkPyPundemy has been brought to you by Rollbar.
One of the frustrating things about being a developer is dealing with errors, relying on users to report errors, digging through log files, trying to debug issues, or a million alerts just flooding your inbox and ruining your day.
With Rollbar's full stack error monitoring, you'll get the context, insights, and control that you need to find and fix bugs faster.
It's easy to install.
You can start tracking production errors and deployments in eight minutes or even less.
Rollbar works with all the major languages and frameworks, including the Python ones such as Django, Flask, Pyramid, as well as Ruby, JavaScript, Node, iOS, and Android.
You can integrate Rollbar into your existing workflow, send error alerts to Slack or HipChat, or even automatically create issues in Jira, Pivotal Tracker, and a whole bunch more.
Rollbar has put together a special offer for Talk Python to Me listeners.
Visit Rollbar.com slash Talk Python to Me, sign up, and get the bootstrap plan free for 90 days.
That's 300,000 errors tracked all for free.
But hey, just between you and me, I really hope you don't encounter that many errors.
Loved by developers at awesome companies like Heroku, Twilio, Kayak, Instacart, Zendesk, Twitch, and more.
Give Rollbar a try today.
Go to Rollbar.com slash Talk Python to Me.
Yeah, so you said, well, your website, which I'm presuming you had a say in what it said,
says that the end goal of the Beware project is to be able to do for mobile and desktop facing software the same thing that Django has done for web software.
I think that touches on the point you just made that like Django, the Django community, Django girls, all these different groups seem to have found a way to make it really easy to onboard into dynamic server-based web development,
which is not really a simple thing to do, to be honest.
There's a lot of moving parts and things you've got to know.
Yeah, and I suppose that's the Django girls analogy is really the one that's really is my yardstick for when this is successful,
because what the Ola Sotaska, Ola Sendecker have done and the rest of the team with Django girls as it's grown has been absolutely amazing,
because it proves that you don't need to have a computer science degree to be able to do things on the web and to do really meaningful things on the web too.
You can take people who are intelligent and engaged with no computer science background and in a day's training course,
get them to the point where they have a database-backed website that is under their control, publicly visible somewhere on the internet.
Now, there's that, yeah, at some level, that's an absolutely huge achievement, you know, because this is incredibly empowering for those people.
But it also, having APIs, part of the reason it's possible is because we've got an API, we've got Django,
we've got a language like Python that is readable and gives reasonably good error messages and enforces reasonably good practices.
But we haven't done it by taking something and dumbing it down.
We've taken the live tool and just pointed them in the direction of things they have to do.
That's powerful.
That's super powerful.
I think that's also one of the powers of Python is there are simple languages out there, Visual Basic, for example,
that is super easy to get started with, but they don't go very far.
And then there's really powerful languages like C++, Java, C#, whatever.
But there's, you know, that first step is like a really big jump.
And Python, I think, is just really well designed in that it actually spans both of those.
And so we're kind of lucky to be in this space as well.
Yeah, no, I agree.
I mean, it's Python is a language that is easy enough to teach to beginners and simple enough and conceivable enough that you can –
hello world is one line of code.
You don't have to teach a whole lot of computer science concepts to get going.
But that doesn't mean the computer science concepts aren't there.
You can do asynchronous decorators and, you know, all this kind of really, really hardcore computer science stuff without –
but you can opt into it as you need to over time.
And you can get basic stuff done.
And Django, you know, something like Django, you can provide an API that lets you get stuff done with most of the important –
sort of the difficult questions answered for you, but with an option in the future that if it ever starts to become a problem,
you can dig in and replace that decision with something that's going to scale better or perform better or whatever the problem happens to be.
Yeah, absolutely.
And we have a lot of good examples like, hey, you can build Dropbox or Instagram or whatever.
Yeah, exactly.
I mean like, okay, well, whatever you're trying to do is probably not that complicated.
So perfect.
This will work for you.
You can scale from Django Girls to Dropbox without – or to Instagram without – you know, okay, yes, you've got to level up along the way.
And there's – you know, the Django that's being used in Instagram is not the same Django that's being used in Django Girls in some regards because so many parts have been swapped out.
There was a quote from Cal Henderson at the very, very first DjangoCon where he sort of points out that the frameworks – the companies that are at the top of that performance tree,
the top 1% of websites have very, very unique requirements because they've got very specific traffic patterns and very specific queries that need to be issued.
And there are some things they have to do a lot of that other people don't have to do a lot of.
The good news is that 99% of websites are not in the top 1% of websites.
So for most of those websites, you can just take a common entry point, give them something generic, and as long as they've got the power to move over time,
then it doesn't matter that you're hiding the detail from them initially.
Yeah.
Yeah, that's really great.
One of the things that I thought would be fun to do for the audience would be to look at all the different projects that you have going on
because I looked at it here and I was just blown away at how many different things, you know, following the Unix philosophy that you pointed out,
cool little tools that fit together that you guys have here.
So maybe we could do like a quick survey of those.
Sure.
Okay.
And it's interesting that this is actually, no, reactions are probably the wrong way to putting it.
It's sort of, it's lessons learned from being involved in Django.
The Unix philosophy here is not just about a single tool that does something well and therefore makes it easy to use and powerful to plug into something else.
It's also a contribution aid because if you've got a tool that you know only does one thing and only does one thing well,
the code base is inherently smaller.
Now, overall, the amount of lines of code that are ultimately going to end up in everything that's Bware is probably going to be as big, if not bigger than Django, I imagine.
But if you want to just contribute to the coverage tool, you can look at just the code for the coverage tool.
You don't have to carry the entire baggage of all of Bware to be able to understand that one piece.
Yeah, that's a great point.
It makes it easier to consume piece and which makes it easier to get involved.
But yes, so a survey of what we have.
Sure.
Well, also, I think that also is helpful from a project management and scope perspective.
Like somebody proposes a feature for Cricket, the test runner.
Well, like it really had better be about running tests, right?
And not like an editor feature or something, right?
Like if this was some giant project that did all these things, well, maybe it makes sense to add this funky drag and drop editor thing that somebody's proposing or whatever their idea was, right?
Yeah, yeah, exactly.
And, you know, you can say, well, okay, yes, this should absolutely be a tool.
It's just not Cricket because Cricket is a test runner.
Yeah.
Excellent.
Perfect.
So just for people listening, you can find all this stuff at pybee.org.
So P-Y-B-E-E dot O-R-G.
And so let's talk about some of the tools.
Like you had built some tools for actually performing checks when people do pull requests.
And you call that B4 with two E's.
Nice.
Yes.
We're big on puns as well.
So there's almost always an inside story or a background gag.
The person who suggested B4 to me suggested at the same time a different name that I really am kicking myself I didn't pick, which was Apis Praiore, which, yeah, sort of is the Latin for B.
But, you know, Apriori, Apis Praiore.
But, yeah, so B4 is – we haven't actually got that one live yet.
It's basically we're looking for a way to actually run that and integrate.
But it's essentially – all it does is listens for a pull request trigger, runs in the same way that you don't – you know, pysetup.py –
pysetup.py test runs your test suite.
Pythonsetup.py B4 will go through and run linters.
It'll go through and run spell checkers.
It'll go through and run any of the pre-commit checks that you might have, which is also not something that's particularly uncommon.
But the thing that's different is that it integrates with GitHub so that when you – if you look at, like, your GitHub pull request, if anyone's integrated with something like Travis, it will run your pull requests – sorry, run your pull requests through Travis, run the test suite, and then if it passes the tests, give it a little green tick.
B4 will also put a little green tick to say, and the linting is fine, and the spell checking is fine, and this, and this, and this.
So just as a sanity checker, before anyone even reviews a pull request, we can go through 15 different checks if we need to about what this code needs to look like and provide the automated feedback to say, oh, yeah, no, hang on.
You've got an indentation problem here on line 17 and a spell check – a spelling problem in your documentation over here and so on and so on.
That's actually taking a suggestion from the Rust community that they've sort of discovered – I don't know if it was accidentally or intentional or if there's something behind it – but that if you've got feedback that needs to be given, it's – that is sort of mechanical in nature.
It is better for that feedback to come from a robot because people understand that robots don't have emotions and aren't being – they're not picking on this because you're a human.
They're picking on this because it's not fitting a rule, and so they will work with it.
It also means it can be automated, which is one more thing that you don't have to do as a developer who is reviewing this code.
If I look at a B4 approved patch, I know that it passes all the tests, and it has been linted, and it doesn't have any spelling mistakes, and so on and so on.
Yeah, it's great, and so you save your energy for reviewing things like algorithms and the checks that should be there and whatnot, right?
Yeah.
Cool.
So the next one that's on the list is Duvet, which is your coverage visualization tool.
We talked about that.
And then there's Briefcase.
That's the one that lets you package your Python app into a standalone application, and I just can't get over the possibility of how cool that is.
I know there's been like Py2exe and CX Freeze and Py2app, and they all seem to like almost work, and they just drive me crazy.
Yeah, and that was kind of my – like I'm aware these projects exist, and a lot of them work really well for Windows but don't work on any other platform.
So in some cases, I will admit I haven't actually done a whole lot of packaging work here on Windows, and I haven't done any on Linux at this point.
But the approach that I'm sort of taking there with the way Briefcase is working is basically, well, here is a template for what an app needs to look like if only I knew the one script I need to run.
Put my one script here, and the rest goes.
It's also a helper sort of thing for iOS and Android because in order to get an iOS project running, you can't just drop a Python script somewhere.
You actually need to have this great big iOS Xcode project, and most of that can be automated.
Yeah, and it actually creates an Xcode project file, right?
Yes, exactly, and the intention is in the not-too-distant future, hopefully, that it will create the Xcode project, compile the Xcode project, push the Xcode project to the device, and start it.
Yeah, that's fantastic.
Yeah.
Like I said, I'm very thrilled about this, and this is going to require lots of attention because that looks cool.
The other one that we talked about that's next in the list is Cricket, so that's your GUI test runner.
That was your first foyer in this whole beware plan, right?
Correct, yeah.
Yeah, that's where I started as a proof of concept that, yes, you could build a cross-platform GUI tool that was useful for development purposes that you could pip install into your development environment and just use.
I mean, the thing that's interesting there, that sort of pip install thing, if you've got – like test runners aren't unusual in IDEs, but in order to get them to work, like you need to know where PyCharm puts its path setting.
You need to know where the environment variables go for the test environment to start up.
So like in Django's context, it's, you know, you need to set Django settings.
You need to know where the Django settings file is so that the application can start.
Working out where to put that in your IDE is a complex thing.
Whereas with – one of the interesting side effects of Cricket is because you pip install it into your environment, when it starts, it forks from inside your running Python environment.
So you inherit all the environment that it needs to run in.
So as long as you can run a test suite, as long as your environment is set up the way that, you know, your command line environment should be set up, Cricket just works.
You don't have to – there is no configuration dialogue for Cricket.
Yeah, that's awesome.
I really like it.
And, you know, the playful icons and the fun names really make it, like, enticing to go explore these.
So the next one, speaking of that, is called Bug Jar.
Yeah, and I'm actually mildly surprised that that name hadn't been taken anywhere because it seems like an absolute gimme for a debugger.
That's actually, like, if you dig back into the archives behind – before Cricket, Bug Jar is actually where I really started because what I really wanted – PDB is an incredibly powerful tool.
PDB is – the API, the way it all works is fantastic.
The PDB interface, to my mind, is toxic.
And I know that there is IPDB and there's PDB++ and there's a bunch of other tools out there, but they're all console-driven.
And I know there is WinPDB, but I've never been able to get it to actually, you know, work.
What I really, really wanted was the debugger that was like the one that I had in 1988 when I first got my copy of Ball and C, but for Python.
And I got a little distracted along the way.
Eventually, I'm going to get back there, I swear.
That's a great idea, and I really appreciate a visual interactive debugger myself as well.
I know you can do it with – in the command line, but it's not the same when you're working on a large project, in my opinion.
Yeah, I mean, the visual scope and context you get by having an arrow pointing at a line of code with a window on the side that says the value of X is 3 versus a command line interface where you have to type 15 commands to get the same amount of information.
It just – it seems like a no-brainer to me.
Yeah, absolutely.
So the last one is kind of a quick documentation one called Galley.
Yeah, the name on that may actually need to change because we've lost the namespace on PyPI.
That's the one that finally pushed me over the edge with TK because TK doesn't have a web widget.
I write documentation.
I like – I don't like writing documentation, but I appreciate the documentation is important and I need to write it.
But Sphinx, as much as markdown and restructured text try to look like markup, they don't.
So you end up writing a bunch of code, compiling it.
Oh, I missed – I didn't put – there's one less underscore that I needed on that particular thing or I forgot the colon after this bit.
So the idea behind Galley was to provide you a text window on one side where you edit your text and then actually render it so you can see what it looks like as you modify it.
But in order to do that, you need to have a web browser to actually display the content of the code or the content of what has been generated and you can't do that in TK.
Actually, the other interesting thing about Galley is that it actually takes me a little bit closer to the thing that got me involved in Django in the first place, which is an editing tool.
There's a whole other podcast in the tool that in my dream of dreams I'm eventually one day going to write.
That's sort of an editor that is context-sensitive or syntax-sensitive.
But, yeah, that's a talk for another time.
All right.
Awesome.
Sounds great.
All right.
So that wraps it up for the tools.
Let's talk about some of the libraries.
And these libraries are – each one of them is really ambitious.
And it's just one more library.
So that's pretty cool.
So the first one is Coliseum.
This has to do with the layout and putting stuff basically on a screen.
Is this for the web or is this for your Python desktop app?
No.
So this is actually targeting – like the reason I started that project is specifically because of Toga.
When I started Toga, the original plan was to use something called Cassowary, which is if you've ever programmed for iOS,
it's the constraints algorithm that underpins Cassowary.
So that underpins iOS.
So effectively, it's a linear algebra approach to describing how widgets appear on a page.
If you've ever written any GUI code for a native application, you effectively have to say,
I've got a button and a button, and there's a split pane, and the split pane is this, and so on and so on.
And there's usually some box model that's involved, which here is a stack of vertical boxes,
or here is a grid of some description.
The Cassowary is trying to do an algorithmic approach, which says, I'm going to specify how my desktop lays out as a series of linear programming constraints.
So the left side of this button must be 10 pixels from the left side of the screen,
but the right side of the button must be no more than 10 pixels from the right side of the screen, and so on.
And then you run Cassowary to solve the linear programming constraints, and then your widgets appear on the screen.
Now, that's incredibly powerful, but it's also an absolute pain in the butt if when it goes wrong,
because all of a sudden you're trying to solve-
Yeah, it's super hard to debug that iOS stuff, and you're just like, what is going wrong with this?
Why does this not work?
When it works, it's great.
When it doesn't, you are screwed.
So prompted mostly by React, they sort of, or React Native rather, they sort of pointed out that you could lay out,
like the CSS box model, yes, it's intended for the web, but there's actually nothing web-specific about it per se.
It's basically just a set of algorithms for how you put boxes of content and relate boxes of content in a tree-like structure.
So what I said originally ported just the version of Flexbox that React has developed,
and the intention is to connect that into a full CSS2 plus Flexbox plus any of the other interesting pieces of the CSS spec
that you can use for any boxes.
Now, obviously, that immediately translates to buttons, widgets on a page,
but you could also do things like, okay, I need a HTML to PDF translator.
Well, okay, I can take the DOM tree that I've got and use that to turn out a set of drawing instructions.
So the intention is that ultimately Colosseum is going to be powerful, or it will be powerful enough
that you can throw an arbitrary piece of CSS and some tree of content and be able to render it.
The interesting side effect of that, of course, is that when you actually want to take Toga and deploy it to the web,
which is something I've spoken about at DjangoCon Europe this year, it means that you've actually got CSS.
So you've got the same layout mechanism on the web that you're using in your native GUI toolkits.
So you don't actually have to have a separate layout scheme working out there.
Yeah, that's cool.
I really like it.
That's quite ambitious.
I definitely think CSS is a fabulous way to layout things.
I've used a lot of Windows, rich GUI type of layouts, and I've always gone, gee, if I just had CSS, this thing would be maintainable, you know?
I will admit I do have a bit of a love-hate relationship with CSS more generally, but it's sort of – in terms of a general – it is something that is well-known.
It is something that is well-understood.
It is something that is well-documented.
It is something that is open.
And that sort of combination of stuff, but the only – it makes it really powerful, but the only thing we don't have is an open implementation that isn't tied to a browser.
Yeah.
So, yeah, at some point I'd need to sit down and spend a whole lot of time just working on Coliseum.
At the moment it's fairly limited in what it can do.
It's useful enough but not completely CSS or the complete CSS.
Sure, sure.
Yeah.
Cool.
And, you know, I think part of that frustration, at least projected my own frustrations I've had with CSS and things like that and the design is,
more about when the implementation varies.
It's like, why does IE have a line here, a border, and why does Firefox have a space and Chrome has neither of these, like – that's not really a frustration with CSS.
That's just, like, with the browsers.
Yeah.
Yeah, for sure.
IE platform for top Python developer jobs.
Create your profile and instantly get access to 3,500 companies who will work to compete with you.
Take it from one of Hired's users who recently got a job and said,
I had my first offer on Thursday after going live on Monday and I ended up getting eight offers in total.
I've worked with recruiters in the past, but they've always been pretty hit and miss.
I tried LinkedIn, but I found Hired to be the best.
I really like knowing the salary up front.
Privacy was also a huge seller for me.
Sounds awesome, doesn't it?
Well, wait until you hear about the sign-in bonus.
Everyone who accepts a job from Hired gets $1,000 sign-in bonus.
And as Talk Python listeners, it gets way sweeter.
Use the link Hired.com slash Talk Python to me and Hired will double the sign-in bonus to $2,000.
Opportunity's knocking.
Visit Hired.com slash Talk Python to me and answer the door.
Yeah, so you mentioned Toga a couple times and Toga is a Python native, OS native GUI toolkit.
Yes.
Tell us about that.
Okay.
So this is essentially the bit that comes before the actual application.
So the philosophy we've kind of taken is that native is good.
Native is what you're looking for.
You want something that looks native, feels native.
It's not pretending to be agnostic.
It's not pretending to be the platform that it's not.
It gives you – if you want a button, it's a button the way that a button looks on that platform.
And it sort of does that by being Python first, Python native, which means that you're not looking at an API that is very obviously C wrapped in Python.
It enables you to do things like – the classic example is whenever you've got a long-running task in most widget toolkits, they are C or some variant thereof.
And so you come up with some small quantum of work that can be evaluated and you push that quantum of work into some form of timer in a separate thread that gets evaluated at some regular interval.
In Toga, you use the same – just a function that's a callback except you make it a generator.
You make it yield regularly.
And when it yields, what it does is it yields to the event loop and then it resumes inside where it was previously.
So you can exploit the fact that Toga gives you – or so Python gives you this mechanism to resume where you were inside a method, which isn't a capability you have inside Java and C and C++ and so on, to produce an API that is rich and flexible.
You know, when you want to specify a list of something, you just give it a list of something.
When you want it in a string, you just give it a string.
And it works out how to make that render in a native fashion, in a native appropriate fashion.
Yeah, that's really amazing.
And if you want to try it out, anyone listening can just pip it, soltoga, T-O-G-A, and then toga-demo, right?
And then boom, you have a running GUI right there.
It looks totally native to me.
Yeah.
And it is completely native because it is – you know, when Toga demo runs, it is firing up on Mac.
It's an NS window with an NS split pane with an NS list or NS table and so on and so on.
And when you click on the button that's a dialog, it's an NS alert box or whatever it happens to be.
And then if you do exactly the same on the GTK, it's a GTK window with a GTK split pane with a GTK button in it and so on and so on.
So if you look behind this, it sounds like it's incredibly complicated, but the actual code behind Toga is relatively simple once you've got the bridging to the native layer done.
On GTK, you get that for free because there is a native Python API.
And Rubicon, which is the library that lets us get to Objective-C, is complex but almost completely transparent because you just sort of name the classes and name the methods in a slightly funny way and you can access them.
So the native Toga implementation of a button on macOS, for example, is about 30 lines of code and it's 30 fairly straightforward lines of code at that.
So adding new widgets is relatively simple.
And at the end of the day, if you do actually want to get into a widget that is very macOS specific, you can just drop into that level.
Okay.
Yeah, that's really great.
Toga looks really amazing.
And like I said, I think building out these truly OS native things in Python, mix that with a briefcase.
Very lovely, right?
So the next one, I'm sure I'm going to mispronounce.
So can you hit it for me?
Okay.
So that's Oroboros.
Oroboros.
Okay.
Yeah.
So Oroboros, for those who don't know the Greek mythology or the old Latin mythology, it's the image of a snake eating its own tail.
So the idea here is that if we're going to – part of being – of Toga is trying to be cross-platform.
So we've got it on macOS.
We've got it on GTK.
We've got it on – we've got some parts of it on Windows.
But they're all just sort of standard Python running in standard Python-y kind of ways.
We've got Python running on iOS, and that's actually just CPython compiled for iOS because iOS is, as much as they try to hide it, C under the hood and Clang under the hood.
But on Android, they really, really, really want you to be using Java.
So I've kind of had to do a bit of an interesting shim to get there, which I'm sure we'll come to.
And for deploying to the web, if you want Python on the web, well, you've got to have it written in JavaScript.
In order to do that, you need an implementation of the Python standard library written in – written so it works on that platform.
Now, a lot of the Python standard library is written in Python, but there are some parts that aren't.
So things like the underlying core of date time, mk time, and time structure T and things like that are native concepts.
So you've got to have a native implementation of those concepts that the rest of the standard library can then build on.
And so Oroboros is attempting to build as much of the standard library in Python as we can possibly build, cribbing from CPython when we can, and then filling out the rest with the interface, the API that you need to implement in order to have date time working inside a web browser or working on an Android phone.
That's really amazing.
Have you combined that with PyPy?
I think we have actually taken some pieces with PyPy, and I think there probably is some crossover there in terms of what we're doing and what they're doing in that PyPy benefits from having a pure Python version of any module because they can then optimize the entire VM rather than just optimizing parts of it.
So at some point, I don't know if we'll merge with PyPy, PyPy merges with Oroboros or whatever, but there is a piece there that I think that can be shared at the very least.
Yeah, absolutely.
That definitely seems like that would be great because if you have it in pure Python and you feed it to PyPy, it should go nice and fast.
Theoretically.
That's the intention, yeah.
And that is ultimately what PyPy is doing a lot of the time.
It is just they have re-implemented parts of what is written in C in Python so that it will run fast because they can pass the full bytecode optimizer over everything it's doing rather than just parts of what it's doing.
Right, absolutely.
So that brings us pretty naturally to the next major category, which you call bridges.
Yes.
I love the subtitle, boldly going where new Python has gone before.
Sorry, the question from before might be like, well, all right, great.
It's a pure Python standard library.
Like, what can you do that?
But then you have Batavia, which would work really well with that, right?
Yeah, so the idea here is that Batavia specifically started as a bit of a joke at PyCon Australia Sprints last year.
I don't particularly like JavaScript.
I accept it that it exists and it is the lingua franca that is on the web and it's not going anywhere.
And I don't want to language shame anybody for having knowing JavaScript.
It is an important language to learn, to know, and there are some very valuable things in there.
But I would be a very happy man if I never had to write another line of JavaScript in my life.
So part of that is that if I'm going to write web software and I want responsive front-end web software, I need to have Python running in the browser.
Or alternatively, I need to rewrite everything I'm doing on the server in JavaScript, which given that I don't want to do that, I need to get Python into the browser somehow.
So what Batavia does is take a look at the fact that when you compile CPython, now Python as a whole is a, you know, if you wanted to have a complete Python compiler and interpreter and everything, yeah, that's a big thing to build.
And there are people who are doing that.
You know, Brython and Sculpt both do that.
PyPy.js takes a slightly different approach.
I had Ryan from PyPy.js on and there's a bunch of cool stuff that's happening to him.
But a lot of those projects are really large or they make big trade-offs in terms of support.
Yeah, so like, you know, PyPy.js, I don't know exactly where it's at in terms of size now.
But like the first time I looked at it, it was like 20 megabytes as a compressed download.
And that's fine if you've got that already cached.
But if you want to build your next Instagram, downloading a 20 meg bootstrap is just not a candidate.
So Batavia treats it at a slightly different angle.
So in saying instead of we don't actually need the full Python interpreter or the full Python compiling tool chain with a REPL and everything else,
what we need is the ability to run Python code.
CPython compiles to bytecode.
Bytecode is a simple virtual machine with 150 odd instructions, most of which are push, pop, add value, subtract value, this kind of thing.
Can we implement that virtual machine in JavaScript and then just give it Python bytecode?
And the answer is yes.
And it turns out to be a relatively small amount of bytecode as well.
You can get Batavia, like a fully functioning Batavia instance running in something like 10 kilobyte of compressed JavaScript.
And I'm sure there's all sorts of opportunities to optimize there as well.
That's amazing.
And that's totally acceptable for downloading 10, 15K in JavaScript.
Yeah, yeah, yeah.
So that's less than your logo image in most cases.
You know, it's easily an achievable goal.
And all you're really losing is really performance at that point because it is CPython that is running.
It's just running in the browser using JavaScript to emulate the pieces that can't be done natively.
Now, the other thing I have to shout out here is part of the reason that I know this or I thought this joke actually might be plausible is a blog article slash book chapter written by Ned Batchelder and Alison Captur called Architecture Open Source Volume 4, which looks at sort of a bunch of complex problems and says, okay, let's solve this problem in 500 lines or 500 lines of Python.
And Alison's chapter in that book.
Yeah, that's a cool book.
Yeah, it's a very cool book.
Yeah, it's an amazing book.
And one of the chapters in there is here is a Python interpreter written in 500 lines of Python.
And that blew my mind when I read it.
It's amazing stuff.
But it also means, hang on, 500 lines of Python, that means I should be able to do it in about 500 lines of JavaScript as well.
And that's about dead on.
It's not that far from it.
You need a little bit more bootstrapping to say like JavaScript's only got numbers.
You need a concept of integers and floats and they need to behave the right way.
There's a little bit of bootstrapping you need to do around the outside.
But it's achievable is the point.
And you can do it in sub 100k of code or some 100k of deliverable artifacts, which means that when you – if you've got like a – say you've got a Django server that has a form and the form has a validation function at some point.
That validation function is Python code.
It can be compiled as a code object.
That code object has bytecode.
That bytecode can be turned into a string.
The string can be sent to the browser.
The browser can then run that JavaScript – or say run that bytecode in the JavaScript version of the virtual machine.
And then you've got not just similar or transpiled or whatever, exactly the same code running on the client and the server.
And you've written it all in Python.
Yeah, that's great.
So it takes the load off your server because most of the stuff would happen client-side.
And it means you write it once.
Yeah, it's not just the load.
I mean the load is obviously a benefit.
But it's also – if you want a rich user interface, you want to say you want to validate credit card numbers as they're being typed in.
You can't do that server-side because this is something I get acutely aware of because I'm in Australia.
You can't change the laws of physics.
Ping time from me to the east coast of the U.S. is 300 milliseconds.
You cannot change that unless you are willing to accept a 300 millisecond or longer delay in every user interface action.
It has to be running locally.
And if you can run it in Python, then all the better.
Yeah, very cool.
Those two go together.
Great.
That's awesome.
So that's Python in JavaScript.
Yes.
What if I want to run Python bytecode on, say, the JPM?
Well, I'm glad you asked.
So, yeah, the next project on the list is VOC, VOC.
That is a – takes a very similar approach to what Batavia has done.
But instead of saying let's target JavaScript, let's rewrite a CPython interpreter in JavaScript, it says, okay, CPython, when it runs, is a stack-based virtual machine.
Java, when it runs, is a stack-based virtual machine.
Now, okay, to be sure, they are different stack-based virtual machines.
They have different requirements.
They have different needs.
They have different operational characteristics.
But they're close enough that if you squint, you can turn one into the other.
And so that's what VOC does.
VOC says, okay, let's take this Python code, CPython code, and compile it into directly the equivalent Java class files.
Not turning it into Java code that gets compiled.
Turning it into the class file directly.
So the output isn't a PYC file.
The output is a .class file.
That then runs as if it was Java because it is Java.
So, and the sort of interesting little side effects of things like that is that when, you know, you write your Python code and it throws an exception, for example, the exception can point at the line of Python code that generated it.
Now, this sort of general approach isn't that unusual.
If you look at something like Scala, Scala is just a compiler that produces Java class files.
It happens to be a different functional language.
So all we're doing here is effectively a compiler that takes Python input and spits out Java class files.
Now, that's critically different to something like Jython, which is a full implementation of a Java virtual machine and Java compiler and all the rest.
So when you run your Python code through Jython, it is Python code being interpreted in Java at runtime.
Right.
Which is also not to be, not to dis Jython.
Jython is a great project as well, but it's a different approach.
Right.
It's almost like instead of using C to implement the virtual machine, they're using Java versus actually compiling, compiling the steps.
Yeah.
And ultimately, the reason that I've kind of started VOC is that you say it's written in Java rather than written in C star, except for the parts that aren't.
Because there are parts of the compiler tool chain, Xerces and what have you, that are written using the Java native interface.
And so you can't actually, at least not at the moment anyway, you can't compile Jython for Android because there are parts that will not run on Android.
So I needed a different approach to get things working on Android.
And so, you know, VOC was born.
Cool.
And VOC works on Android.
And yeah, the intention was originally to target it for Android.
But by a side effect, because Java is actually a cross-platform runtime environment, I've, you know, I knocked out a demo swing application.
You can write a native swing application that gets native swing APIs in Python and it will run through Java.
And you can't tell that it's not Java.
That's really cool.
It turns out a Java API, a Java user interface that looks awful because it's not native.
But, you know, it is using Swing's native components.
You could entirely plausibly write a, you can't do it yet, but conceptually, you could take Django, compile it to Java class files, run it in a J2EE container, and satisfy your, you know, your enterprise deployment requirement that everything's written in Java.
Because it is.
It's just written in Python in Java.
Yeah.
They just care how it executes, right?
Interesting.
Yeah, exactly.
So I'm always, you know, I've done a lot of, like, scientific parallel computation type stuff previously.
And I'm always on the lookout for things that let me dodge the GIL.
Does this let you dodge the gill?
Well, effectively, yeah, because, you know, it is on, when it's compiling, it's compiling for the Java virtual machine.
So it is producing thing, you know, the variables are all locked the same way they would be in a piece of Java code.
So it is a gill-less version of Python.
With all the language, well, not all yet, because I'm still working on bits of it.
But, you know, you can do anything you can do in Python, you can do in Vox.
You know, you can, for example, you can do yield statements in Vox.
There is no equivalent of a yield statement in Java.
But I can, you can generate the Java bytecode that does the functionality of yield, because it's essentially just assembly language at that point.
It's just Java assembly language.
Okay, very interesting.
All right, final one on your bridge.
Is Rubicon.
Yeah, so Rubicon was, there was originally a Rubicon Java as well, which I was looking at using for Android, but turned out to not be quite plausible for various reasons.
But it's mostly there for the Objective-C portion.
So I touched on this earlier, that Objective-C, for all of the funny syntax, is ultimately just a series of C APIs.
And C types, which is part of the Python standard library, lets you call any native C function as long as you know it's prototype.
So you can invoke any Objective-C API directly from Python without having to compile anything.
And then Rubicon then sort of wraps that up in a bunch of meta classes and whatnot, using the Python 3 type annotation syntax to say, well, this argument is an integer and this one is a Boolean and it returns an object.
Map that to the underlying call in Objective-C.
You end up with, you know, you can instantiate an NS button the same way that you would in Python normally.
The only difference is that the message passing syntax replace colons with underscores and the method names get really long.
But that's kind of, you know, you don't have to pre-compile, pre-declare, pre-anything about what is in your Objective-C environment.
You've just got to call the things you need.
Wow, that's fantastic.
So you brought up Python 3.
What's the story with Python 3 across Beware in general?
Okay.
So I have taken the wonderfully controversial stance of saying Python 2 is dead.
I am developing a set of tools.
I started developing a set of tools four years ago.
Four years ago, I was developing stuff in Python 2 primarily.
I am now at the point where I'm developing new runtimes.
I'm developing new languages or new runtime environments, new language interpreters, new libraries for cross-platform.
I have made the decision that I am Python 3 and for the most part, I don't care about Python 2.
Now, if someone else wants to come in and care about Python 2, power to them.
I won't turn down the patch.
But I am aggressively targeting Python 3 and hoping that that will be enough of an incentive.
Like by the time that Toga is mature enough that you can seriously do stuff with it, that that will be enough of a reason for people to consider building new code in Python.
And part of that is also that it is also new projects.
And this is a new library.
No one is coming to this with existing code base.
So I can start with the new APIs.
And so I am.
Yeah, that's brilliant.
And on behalf of everyone, thank you for moving us all one step closer.
That's great.
Or how many projects do you have?
15?
Maybe move just 15 steps closer.
I don't know.
But you definitely move just closer.
So thanks.
All right.
So that kind of more or less rounds up beware.
There's a few more things on here.
One of them is Python Apple support, which let me just read the description because I think it really brings home some of the power of what you have here.
A meta package for building a version of Python that can be embedded into macOS, iOS, tvOS, or a watchOS project.
Of course, for Python, written in Python, right?
That's just so cool that all those OSes are involved in this, right?
Yeah.
So I mean this is kind of one of these accidents that happens because Apple is trying to make a consistent development environment for themselves.
We just happen to be able to, you know, horseshoe, crowbar into place.
As Apple tries to put or is pushing Swift, they have historically pushed Objective-C.
Both of those are built on top of Clang.
Clang is a GCC advanced in various ways.
So it is, under the hood, a C compiler.
The good news is that CPython is written in C.
And so with the application of a few patches, you can compile the default CPython sources for any of those platforms.
Now, the catch is that, and this is sort of a process of long negotiation with the CPython team,
is that it's not just a matter of a patch that says, hey, look, you can build for macOS.
MacOS these days only compiles for one platform.
It compiles for x86.
When you compile for iOS, you actually need to compile for, depending upon where exactly you're targeting,
possibly as many as five or six different platforms.
You need ARMv6, ARMv7, ARMv7S, i386, x86-64, ARM64.
And you need to have all of those compiled and turned into what's called a fat binary.
So it's like one library file that has all of those pieces.
And there's a whole bunch of tooling that you need to, like, you need to build all the libraries separately
and then mash them together with this thing, this tool called LiPo, to make them into one library.
Now, that doesn't fit well into the CPython build chain at the moment.
I've submitted a proposal for a patch that would enable you to manage that,
but I don't know if that's likely to ever get merged.
There's a whole bunch of sort of technical and political things that go on there.
So Python Apple support is essentially just an external project that says, okay,
take the CPython source code, download it for you, apply the patch that I know needs to be applied,
build it, build all the dependencies, build OpenSSL, and build libbz2 and all the other pieces that need to be built.
And spit out a Python.framework that works on iOS.
That's great.
And then the interesting thing is then, okay, well, doing that for iOS, well, you can do it for macOS as well,
because that's basically just, you know, you only have to compile it for x86.
tvOS is exactly the same thing, except you've only got to compile it for ARM64.
And watchOS is exactly the same thing, except you've got to compile it for ARM7K.
So it's the same basic infrastructure.
You've just got to change the key that you're calling into.
I see.
Just make it a little more Apple-friendly, bundle that all together.
Yeah.
Yeah, exactly.
Awesome.
And so one of your main goals was to build a Python toolchain that lets you write these UI apps in Android
and the various Apple OSes as well as Windows.
And so you have a bunch of templates as well to help people get started, right?
These are cookie-cutter templates, which I haven't talked a lot about cookie-cutter, but it's a really great project.
And I'm sure we'll have Audrey on the show sometime to talk about it.
Yeah.
Yeah, so tell us really quickly about your templates before we're out of time.
Sure.
So the templates are effectively saying, well, you know, okay, a Python project is, at the end of the day,
a entry.py that contains print hello world.
You know, it can be more than that, but that's all you have to have.
But in order for them to run on iOS, you have to have a whole lot more.
You have to have your Xcode project and your dependencies and your icons and your configurations and all this other stuff.
But most of that information is relatively constrained.
Like we know we need the name of the package, and the name of the package needs to appear here, here, here, here, and here.
So the templates are basically just saying, okay, well, I know what format it needs to come out as.
All I don't know is the name of the package I want to generate.
So let's just leave that as a blank, generate it from template, and then you've got something you can use.
And then Briefcase takes those templates and takes the support libraries and things like that, pre-compiled versions of the support libraries, and spits them out.
You run Briefcase over your project.
If you've got a project that has your hello world.py file, and it has a setup.py that says that's where the file is and these are the dependencies,
it takes all that information, uses that to generate the template so that you can then just say, run this Xcode project, and off goes your iOS project.
Or run this, generate the GTK application or generate the watchOS application.
Yeah.
That's excellent.
Yeah.
So I think your project is great.
I definitely want to encourage people to check it out.
We'll probably have to kind of leave it there for that one.
So one thing I did notice is that you have a couple of upcoming events that you're going to be at.
Were you just in Portland?
I was not in Portland for PyDX, no.
I was in Portland for PyCon this year, and I'll probably be back there next year at this point.
All right.
Yeah.
I was thinking, maybe I just missed you like three days ago in PyDX.
Anyway.
No, no.
Unfortunately not.
Yeah.
Okay.
But so, yeah, maybe tell people about some of the upcoming events that you have that you're going to be at in case they want to meet you or some of the team members or something like that.
Sure.
Absolutely.
So I myself will be – I'm keynoting Python Brazil on the 15th of October this year, and I'll be around for the sprints there as well.
So if you're in Florianopolis, please come and see me.
We're very keen to have new people involved.
About two weeks after that, I'll be in Amsterdam for Django Under the Hood.
So that's the Django deep dive conference that happens once a year in Amsterdam, and I'll be there for the full sprints of that as well.
Probably working on Batavia and some of the more webby parts of the framework at that point.
I'm also speaking at LinuxConf Australia 2016 in January next year down in Hobart.
If you haven't ever been to Australia, I can highly recommend Hobart as a venue to go to and LCA as a conference.
It's a great little community.
It's sort of the Australian equivalent.
It's like the peak open source conference.
Despite the name LinuxConf AU, it is a multi-language – there are people there talking about FreeBSD.
It's kind of an odd name that's kind of stuck around for historical reasons rather than being actually just about Linux.
So they're the ones immediately.
There are a couple of other conferences that I'm speaking at in the future, some of which I can't talk about.
But what's your directories?
I'm coming to a city near you.
Awesome.
Yeah, spreading the good word.
Excellent.
Indeed.
I also noticed that if people need help, they want to set this up and they've got a big team and they really want to get going, that you also offer training.
So people want to get a jump start, they can contact you, right?
Yes.
Then just maybe give a quick shout out to all the people on it if you want to just pull up the list.
Yeah.
So I haven't specifically got – it's not like you can come to me and say, here's my rate card for Python training or Choga training.
It's more an aspirational thing about being able to fund the development of Beware and Python development on a more sustainable basis.
I have been working on this as sort of a part-time volunteer capacity for the last couple of years.
I've had some amazing help from Katie McLaughlin and Philip James and Chris Swenson who have done amazing work as well.
But they definitely have volunteer capacity.
I have recently been – started to be paid on a part-time capacity from a company called Jambon Software to do bits of Beware development.
But I would like to ultimately turn this into a full-time gig for myself and a small team of crack Python developers.
And also to use that as an opportunity to reach out, to get new people involved, to reach out, do diversity, to do diversity and outreach, to bring interns in, pay them for their time and actually introduce them to open source development.
And once you get to give them an internship with a crack team of people at the top of their open source contribution game, let them loose on the world and see what else they can do.
So, yes, training is one axis where I'm definitely – I am notionally available for hire if someone wants to do that.
But I'm also open to all sorts of other options, including if you go to pyb.org, there are some options there for corporate sponsorship if you want to become a member of the project and drop some money in the tin regularly so that we can continue to do what we're doing.
Because ultimately, the reason I'm doing this is because I like Python.
I want to keep using Python.
But I don't think we can continue to sustain large-scale Python development the way that it sort of has been on a wish and a prayer for the last 10, 15 years.
Django development has not been anywhere near as fast or as motivated as it could have been if we only had a little bit more money or if the Django project had a little bit more money.
And given the amount of money that is available in the Django community, the number of people who are making millions of dollars with Django, I think it's telling that we haven't been able to find a way to pay the bills a little bit more effectively.
Yeah.
I just want to sort of second that thought and say there are so many companies that are many, many million dollars of revenue or billions of dollars that take open source and just go, okay, great.
This is free.
This saves us money.
There's not a lot of giving back, right?
Like if they just put a small portion, a very small portion back into these projects, it would make such a difference.
So, yeah.
Yeah.
I mean, it's like I've made the gag on Twitter a couple of times where, you know, I'm offering corporate sponsorships in Beware.
And, you know, $1,000 a month, it sounds like it's a lot of money to you and me as individual developers.
To a company, it's like not even – it's a rounding error on their soda budget.
But that's effectively a Desi Russell.
$1,000 is a Desi Russell.
You can get one-tenth of me for $1,000.
Yeah.
The problem is I need 10 of you to agree to buy a Teddy Desi Russell and then I'm working full time.
Now, there are more than 10 companies in the world who have a very vested interest in Python.
I just need to work out who they are.
Yeah, absolutely.
Before we move on, before we sort of round up the show, there's always two questions I ask at the end.
So, I'll send them to you now.
So, first of all, I think you may be able to pull a few from your own project.
And that's totally legit if you want to do that.
But there's over 80,000 packages in PyPI.
What one do you think maybe people don't know about, you want to recommend, you say, this thing's amazing, you should check it out?
If I may indulge, I actually have two.
Oh, you may.
They are both at the – they're not my project, so I will completely be above board there.
There's no nepotism at all going on here.
They're both in the sort of the developer space and they're tools that I have found incredibly useful, particularly since one of the downsides to Beware being lots of little projects is that you have – the process of releasing Toga is not trivial.
There's a lot of things that need to happen because there are a lot of pieces that need to be released.
And each one of them has a release process, each one of which needs to be tested and all the rest.
So the two tools – one is called – a tool called Check Manifest, Check-Manifest.
It is a really, really boring thing.
All it does is you pip install it into your project, you run Check Manifest, and it checks that your manifest.in file matches what is checked into your version control so that you don't accidentally release a package that doesn't have three files that have to be there for the source code to actually run and compile.
Okay.
Excellent.
It is mind-numbingly simple, but the number of times I have made that mistake, it is paid for itself – it's paid for its free cost on PyPI many, many times over.
Nice.
All right.
So that's number one.
Cool.
That's number one.
The second one is DevPI, which is a version of the Python package index that you can run locally.
So it is a fully – it follows the full PyPI protocol and all that sort of stuff.
And effectively, it just lets you run a local PyPI server that you can talk to.
So you can upload a package to PyPI and see if it works without actually loading it up to PyPI to see what you've broken.
So you can sit there and upload the half a dozen packages that are part of Beware, realize that, oh, crap, I've forgotten the dependency here between this one and this one.
And as a result, repackage Toga version 0.2.1 so that it gets the dependency right or misses the file that you forgot to run check manifest this time.
And test it out in a sandbox environment rather than pushing it to PyPI where once you have uploaded Toga 0.2.1, you can't upload it again.
You have to release Toga 0.2.2.
So, yeah, that one has also saved my bacon on more times than I care to count.
Yeah, that's a great recommendation.
All right.
And favorite editor?
So I am a sublime text person.
I like it, but I don't love it.
And this is one of these areas where one of these days, Toga is going to solve this problem for me.
I have an intellectual love of Emacs, but I can't bring myself to actually use it.
Partially because the Lisp syntax does my head in.
And secondly, because the people who developed Emacs seem to be scientifically opposed to user interfaces that make any damn sense whatsoever.
Every application on the planet has decided that command save saves on a Mac box, but not on Emacs.
Now, OK, yes, before the Emacs people come out of the woodwork.
Yes, I know you can configure it, but it does not come out of the box usable for development.
And that for me is if your tool ships in a form where it is unusable, then you have a problem.
My classic one that I absolutely hate about Emacs is that, yes, you can.
Emacs ships with a tutorial.
That's fantastic.
Let's do the tutorial.
If there's something I need to know, let's do the tutorial.
The first couple of pages of the tutorial teach you about the key combinations for page up and page down.
I have a button on my keyboard to do that.
It says page up and page down right on it, doesn't it?
It says page up and page down.
And they work really well.
So stop trying to teach me how to use my keyboard.
Actually teach me how to do the things in Emacs that are interesting.
And I know there are many things in Emacs that are interesting, but I can't get there because the onboarding process is toxic.
Oh, boy.
You also might have a home and an end button as well.
Yes, yes, exactly.
And one of these days, the end goal for me, I'll be a very happy person when I've been able to build a Toga.
Sublime Text is great because it's cross-platform.
It's bad because there's no source code.
It's not open sourced.
So I want to write an open sourced programmable editor that is the analog of Emacs but programmed in Python but open source and native user interface everywhere.
That also just happens to plug into all the other really useful Bware tools because I just need it to be an editor.
I don't need it to be a debugger.
I don't need it to be a coverage tool.
I need it to edit things really well and talk out to APIs of other tools when it needs to talk to other tools.
I think that's a great vision and probably a great place to leave it.
So it's been really great.
Is there any final call to action?
Are you guys accepting contributors or anything like that?
We are always actively seeking out contributors.
So this is one of the things I'm moderately proud of in the way that Bware is operated.
We've got an open offer to anyone of any level of experience to contribute, to be mentored through the process of contributing to Bware.
And that's not just sort of idle words either.
This is something that has actually been fleshed out in reality.
So at PyCon US this year, we ended up filling, I think at the peak, there was something like six or seven tables full of people who were contributing to Bware.
When anyone contributes to Bware, there's actually a little coin.
There's a challenge coin that we've had struck thanks to originally from MaxCDN for funding but now RevSys have funded the second pressing of these coins.
And essentially, as soon as you get a contribution into Bware, you get one of these coins.
And you can only get it if you contribute to Bware.
But we had people who literally learned to use GitHub two hours earlier who have now got commits in Bware.
That's great.
We have been able to mentor people with very, very little programming experience to be contributors of non-trivial features in Bware.
So if you go to the Bware website, there is a link in there for how to get involved with contributing.
One of the big things it shakes off is, yes, okay, this whole imposter syndrome thing, it's real.
And by the way, the members of the core team have imposter syndrome as well.
So, yeah, don't worry about the imposter syndrome.
We will walk you through this.
We want to help you become the best developer you can.
And we'll mentor you through that whole process.
So get in touch.
And we'll be more than happy to help you get involved.
All right.
That's great.
Russell, thanks so much for being on the show.
It's been fun to talk about your project.
Yeah, it's been an absolute pleasure.
Thank you very much for your time.
You bet.
Talk to you later.
This has been another episode of Talk Python to Me.
Today's guest has been Russell Keith McGee.
And this episode has been sponsored by Rollbar and Hired.
Thank you both for supporting the show.
Rollbar takes the pain out of errors.
They give you the context and insight you need to quickly locate and fix errors that might have gone unnoticed until your users complain, of course.
As Talk Python to Me listeners, track a ridiculous number of errors for free at rollbar.com slash Talk Python to Me.
Hired wants to help you find your next big thing.
Visit Hired.com slash Talk Python to Me to get five or more offers with salary and equity presented right up front and a special listener signing bonus of $2,000.
Are you or a colleague trying to learn Python?
Have you tried books and videos that just left you bored by covering topics point by point?
Well, check out my online course, Python Jumpstart, by building 10 apps at talkpython.fm/course to experience a more engaging way to learn Python.
And if you're looking for something a little more advanced, try my WritePythonic code course at talkpython.fm/Pythonic.
Be sure to subscribe to the show.
Open your favorite podcatcher and search for Python.
We should be right at the top.
You can also find the iTunes feed at /itunes, Google Play feed at /play, and direct RSS feed at /rss on talkpython.fm.
Our theme music is Developers, Developers, Developers by Corey Smith, who goes by Smix.
Corey just recently started selling his tracks on iTunes, so I recommend you check it out at talkpython.fm/music.
You can browse his tracks he has for sale on iTunes and listen to the full-length version of the theme song.
This is your host, Michael Kennedy.
Thanks so much for listening.
I really appreciate it.
Smix, let's get out of here.
Outro Music.