#127: Shipping software to users Transcript

00:00 Michael Kennedy: To make softare useful, honestly, to even make it real, you have to ship it. Building that web app? Then get ready to deploy it. Building a toolset for data scientists? How are you going to send them that application? Did you manage to get a cool GUI app going in Python using PySide? It's time that your users start downloading it. Python is an amazing ecosystem, but one of the frailer parts of it has been shipping or deploying software. Especially to end users. In this episode, you'll meet Glyph. Join the two of us as we discuss where we are and where we're going with shipping Python code. This is Talk Python To Me, Episode 127, recorded June 12, 2017. Welcome to Talk Python To Me, a weekly podcast on Python, the language, the library, the ecosystem, and the personalities. This is your host, Michael Kennedy. Follow me on Twitter where I'm @mkennedy. Keep up with the show and listen to past episodes at talkpython.fm and follow the show on Twitter via @talkpython. This episode has been brought to you by Rollbar and Datadog. Be sure to check out what they're offering during their segments. It helps support the show. Hey everyone, two things I want to share quickly before we jump into the conversation with Glyph. First, when we started talking with him about what he's up to day to day, at that time of the recording, he couldn't announce the company name or say too much about what he was doing, but they have since launched, and their name is Pilot. Pilot does bookkeeping for startups using a combination of expert human bookkeepers and software automation. It's a software service that does your books. Not an app for you to do your own books. You can find out more about them by visiting pilot.com. While you're there, you can imagine that Glyph has definitely shipped some Python code to it. Next, I have a new course to announce. I just released RESTful and HTTP APIs in Pyramid. This course will teach you the mechanics of building amazing APIs in Python, of course. But you'll also learn the theory of RESTful HTTP services in general. How to design them, when and how to use HTTP verbs and status codes, content negotiation, and even a little SQLAlchemy. Check the course out at talkpython.fm/rest, that's R-E-S-T. Now, let's talk about shipping some software. Glyph, welcome to Talk Python.

02:23 Glyph: It's great to be here.

02:24 Michael Kennedy: Yeah, it's a real honor to have you on the show. You've done so much cool stuff. I'm surprised our paths haven't crossed more but I'm glad they're crossing today.

02:32 Glyph: Same here. I got to say, I know you're always so complimentary of your guests, and it's an honor to be on the show. I got to say you podcasters do some real good for the Python community yourselves. I feel like I'm talking to a rockstar here, too.

02:46 Michael Kennedy: Thank you so much.

02:47 Glyph: It's just a great way to access the culture of the elite in the Python community, the luminaries. I know a lot of people listen to this show and really kind of get a sense of what the culture of the community is like from it. We all appreciate what you do for the community.

03:03 Michael Kennedy: Thank you so much. I definitely see that as like one of the major services that we provide just to get the back story. Everybody knows about Requests, they know about Twisted. They know about all these things, but they just know the API, they don't know the story and the people. As much as we can share that, that's awesome. Speaking of stories, how'd you get into programming?

03:25 Glyph: That's a very long one. A lot of people, you know, it's this sort of stereotypical story of like, well, I had an Altair when I was a kid and I learned Basic. I had a kind of oddball version of that which is that my father who's a fairly prominent software developer himself, he was an APL programmer when I was a kid.

03:45 Michael Kennedy: APL blows me away, by the way.

03:47 Glyph: Yeah, if you've ever actually seen like a full APL program.

03:50 Michael Kennedy: I've seen little snippets and it's incredible. Can you just describe what the syntax looks like to everyone 'cause it's really not standard.

03:57 Glyph: Imagine that you are an astronaut on the moon at an archeological dig, and you found a tablet that was a programming language from an alien civilization, that's what APL looks like. It's a combination of geometric symbols and Greek letters, and I think there might be some Hebrew letters in there. It is a very odd sort of programming language in that it comes off as evolutionary branch that did not have a lot of other languages on it and it's definitely not the main one, you know, this sort of ALGOL-derived C, C++ type programming languages that we're all used to these days.

04:34 Michael Kennedy: Yeah, it's really a great description, like an alien. When people ask me to describe it, I think of it as like if Egyptians use hieroglyphics to like program, this would be APL, basically.

04:47 Glyph: Yeah, very much so. Of course, when I started learning the program at eight years old, that was not what I was looking for.

04:54 Michael Kennedy: Did you try to learn APL at that age?

04:55 Glyph: Dad tried to teach me. That might not be the same thing. I sort of learned a little bit of APL just enough to start trying to write a text adventure. Zork style video game. I learned how to do the very first thing which was variable assignment, so that when you would type west, it would print out the description of the room to the west. That's kind of where I stayed in terms of programming sophistication for the next decade or so. Well, maybe not decade, maybe around eight years. I started off trying, I didn't want to learn programming, I just wanted to be able to do just enough to make these Zork style video games. Without learning to code, obviously, I couldn't actually get anywhere. So I went from there, which is APL 68K on the Amiga, to HyperCard on the Mac. HyperCard was, I don't know, probably some of your listeners are familiar, but maybe not all of them. HyperCard was a English-like scripting environment where you weren't writing a program like a textual thing, you would launch HyperCard and you'd start editing these documents which were a combination of like buttons and text fields as well as little scripts that were attached to each of them. And so I spent a long time very, very tediously kind of teaching myself to code by accident because I had chosen this environment because it was the one where you didn't have to know how to code. Finally, after years of resisting this, I discovered what a variable was. I discovered that you could like have a value that didn't have to be like put into a text field or stored as a button's label, like you could just actually have a abstract notion of a value. That sort of blew my mind. One of the reasons I'm telling this story is like I feel like that architect that we have of the kid who learned Basic and was programming since they were like five years old, kind of discourages other people from getting into coding. They feel like, oh, I couldn't do this. It takes forever.

06:58 Michael Kennedy: I'm too old to get started.

06:59 Glyph: Yeah, like you have to do this from, it's like language learning, you can only do it when you're really young. Despite having the story where I did learn to code pretty young, I feel like it's important to emphasize, I screwed it up as hard as I could and I really like squandered every opportunity to learn anything useful until I was at least 16, at which point, I actually started to learn to code. I learned some C++ and then I learned some Java, and started actually making games because it was pretty hard to do it without actually knowing how to do any of the coding part of coding. I went from there where I was doing a bunch of kind of hobby projects and game development. I got a job at a small software company in New Jersey and had a terrible, terrible experience writing their flagship application in Java. We were porting it from an old legacy database system to Java, and I thought...

07:58 Michael Kennedy: What went wrong with that experience?

08:00 Glyph: What did go wrong? It was a interesting lesson in depending on proprietary software you don't pay for. The major thing that went wrong was the Java Runtime Environment on the Mac at that time would leak about 3/4 of a megabyte of memory every time you opened a window. Since this was a close-sourced runtime and our entire application is based around these windows that would open and close, you would lose just megabytes of memory at a time. As a cocky 19-year old, I didn't know anything about doing metrics or setting performance targets or anything like that, so we got several months into the project and I of course had been reassuring everyone the whole time, this is going to go great, this is going to be fantastic. It's going to be so much better. We got close to what should have been the end date of the project, and it turned into frantic bug hunt trying to figure out where these problems were coming from in our code and it turned out they were coming from the Java Runtime.

09:00 Michael Kennedy: Oh yeah. You're thinking, what are we doing wrong, let's double check it one more time, and there's nothing you could do, right?

09:06 Glyph: Right. It's never the compiler until one day it actually is the compiler and then you're in a really bad situation. I spent a while trying to debug that. Really, I did like Java as a programing language, but I left that job shortly thereafter just because I felt like dying of shame after convincing everyone this is going to be great, and then it was a huge disaster. As a formative experience, that taught me that, maybe I should be looking for systems that have an open-source component that if there's a horrible, horrible bug, I can kind of get down into the runtime and fix it myself. That's one of the things that led me to discover Python. At that time between jobs, I was kind of working on this text adventure. Once again, this is a recurring theme in my programming.

09:51 Michael Kennedy: I bet you're a fan of MUD, these types of things.

09:53 Glyph: Yeah. I always definitely played one or two in my time.

09:57 Michael Kennedy: Nice, me too.

09:59 Glyph: I was developing this thing and I've written it all in Java and it was a giant massive multithreading like four or five threads per connection. Many of your listeners I hope would be too young to remember this or will have blanked it out as a traumatic memory, but this was a time when there was actually a marketing term, massively multithreaded. This was like a good thing that you would be proud of that your code would do. This Java thing definitely qualified. It was full of just unfixable race conditions and bugs. One of the things that the Python language had which I didn't have exposure to in Java or any of my previous C++ work was the Select module. The Select module, I discovered it first because one of the people who was helping me out with this was developing a kind of client-server protocol. This was a MUD with like a rich text front end so that it wasn't just a console. Graphics and background patterns and things like that.

11:03 Michael Kennedy: Right.

11:05 Glyph: He was rewriting this communication layer with a client and he was tired of coordinating the five or six threads that got involved in the networking layer, and so he just wrote it to use the Select module to detect which of the operations that we might need to do to the socket were available, read or write or error. I had never seen such thing before and I thought to myself, that's interesting. You could do like two or three things at a time but without threads, and that's more or less where Twisted came from once I started investigating that and it ended up being a much simpler, less error prone way to develop apps, and so I fell in love with Python on many levels simultaneously. One of them was that it made that whole style of programming possible. Another was that it's sort of was this thin foam padding around C that made it easier to distribute applications.

11:58 Michael Kennedy: And this thing's not as sharp, it doesn't cut me anymore. It doesn't hurt.

12:01 Glyph: Exactly, yes. Physically, it didn't hurt, actually. I was typing so many fewer lines of code in Python that I wasn't developing serious wrist pain or anything at that time but like after a day of coding, I would feel it if I were writing in C++ or Java. In Python, like it literally did not hurt me to code. That was a pretty big upgrade.

12:22 Michael Kennedy: That's really good, that's cool. Yeah. That's an unpected benefit. I think that using things like I/O ports and completion ports, this type of programming that's sort of asynchronous but not threaded, that's something that takes a while to run into. Maybe a little bit less these days but certainly it's really easy to jump in to threading straight away. Just like, I need parallelism, let's go create threads.

12:48 Glyph: It's kind of interesting how it depends where you come from. Different programming cultures, they give different exposures to this. I actually did a PyCon talk a couple of years ago called So Simple You Could Even Do It In JavaScript about event-driven programming. Because if you come from a background, not just a JavaScript but with any kind of graphical user interface, event-driven programming is very natural. It just seems like something you do.

13:12 Michael Kennedy: Even like VB6 or something like that, yeah.

13:14 Glyph: Right, just because it's really clear when you're saying like on click, do X, on mousedown, do Y. Like those types of things that make intuitive sense. The problem is when you're kind of in a backend environment, you're talking to a database or you're doing some kind of transaction processing and then you have to say on transaction complete and you think to yourself, wait a second, why can't I just block? This is kindof like a function call. That's when thread seems really appealing and if you've been developing things that way, then it just seems natural to keep developing them that way once you have concurrency. I do think that it's definitely kind of becoming more popular in that domain, but there are lots and lots of programmers that you might consider sort of novice or not really or like non-programmers who can code who actually take to it very easily, and so it's kind of an interesting piece of expertise that people pick up in odd ways.

14:09 Michael Kennedy: Yeah, I'm sure, and also the rise of JavaScript, the rise of NodeJS, they really put that front and center like it's unavoidable.

14:16 Glyph: Exactly, and it's that same thing where you just say on click equals, then some code, and it intuitively makes sense. You don't have to explain that that's an event handler and the cascading model in JavaScript. You can learn a whole lot about handling events before you hear the word event.

14:30 Michael Kennedy: Yeah, for sure. What do you do day to day for professional work?

14:36 Glyph: I work at a startup that is started by one of your previous guests I believe, Jessica McKellar is one of the founders. We do bookkeeping, actually. We're a bookkeeping service primarily I think right now for startups, but the idea behind the company is there's lots of apps for doing your own bookkeeping like QuickBooks. We do the work of bookkeeping with software though so we can automatically classify transactions. Our product is not software, it's your books.

15:11 Michael Kennedy: That's awesome. Yeah, that sounds really awesome.

15:13 Glyph: But of course now we have some software behind the scenes. Since podcasts are a durable cultural artifact and our name is temporary, I will avoid saying our name. Hopefully we'll have a permanent one soon.

15:24 Michael Kennedy: It sounds like a really noble goal. There are pretty decent bookkeeping bits of software like QuickBooks, I'm not sure I'm willing to judge it, FreshBooks, Invoiced, those type of things. But that doesn't solve the problem of like, I would rather do something technical and not force myself to become an accountant, right?

15:44 Glyph: Yeah, we are a bookkeeping for companies that don't want to worry about bookkeeping. We have a real focus on automation and trying to make things kind of seamlessly just work.

15:53 Michael Kennedy: Nice, it sounds really fun. You guys have like machine learning and stuff going on there?

15:57 Glyph: I guess I could say we have proprietary machine intelligence which might mean machine learning or might mean a pile of regexs. Who could say?

16:04 Michael Kennedy: Yeah, sure. Yeah, who could say. All right, awesome. Well, it sounds like a really fun project. We're going to talk about some, spend a little time on your PyCon talk. This is a PyCon 2016 talk but were you at the 2017 PyCon?

16:18 Glyph: I was. In fact, I spoke at the 2017 PyCon.

16:21 Michael Kennedy: That's right, that's right. We're going to also talk about your talk there. How was that experience, what do you think?

16:27 Glyph: PyCon is always the highlight of the year. There's just so much stuff that goes on, I can't possibly consume all of it. One of the really interesting things about PyCon is for a technical conference, it has a really large number of tracks. In fact, not only does it have a large numer of tracks, but there's kind of an intentional scheduling thing they do where they will put high profile talks up against each other. Cory Benfield, for example, gave a really good talk right at the same time as I did, so I didn't get to see his.

16:54 Michael Kennedy: Yeah, that was inside requests or something like that, right?

16:57 Glyph: I believe that was, I always get mixed up which year it is, but yes, I think that was this year. There's so much stuff going on beyond just the talks. There's the summits. There's just an incredible hallway track where there's people doing open spaces and informal conversations, and it's a real opportunity for open-source projects to get together and be strategic which is not something we get to do a lot in open-source but we get to sit down and see people from Twisted and Requests and Async I/O and Flask and Jinja and Django all in the same room, all kind of talking about the future of Python and how we're all going to add features to our various projects and things that we can work together on, standards we can collaborate on. There's so much stuff going on at PyCon no matter what your level of skill is. There's a lot of new folks every year at PyCon. I learn something really interesting about some of the software that I work on from someone who's either never used it or who's just started using it for something really unusual. So I love hearing from those types of people as well. Yeah, PyCon, always amazing.

18:09 Michael Kennedy: Yeah, I definitely look forward to it. I'm already looking forward to the next one and it's like 49 weeks away or something, right? I'll just start, get a calendar and start checking them off or something.

18:19 Glyph: I really like what they do with the program committee where there are also a lot of new speakers every year. There are people sharing new and different perspectives. Obviously, I give great talks. You should go see all of my talks, they're all archived. But it's not just me and Cory and Henek and Guido standing up every year and giving the same talk over and over again. If the old timers want to get up on stage and say something, it's a very competitive environment because there are people talking about monitoring black holes with Python.

18:55 Michael Kennedy: Automating robots and factories or all sorts of stuff, right?

18:59 Glyph: Yeah, there's just so much especially with the uptick in data science. There's so much science going on that has Python involved in it in some way.

19:07 Michael Kennedy: Yeah, that's interesting. It's probably going to only get more competitive in that there are more people and more disciplines its pulling from.

19:14 Glyph: Exactly, and it's great to see that reflected. I like to say that Python is always the second best language for everything. It's never quite the best thing you can reach for, but as soon as you need to do two things suddenly it's the best language because you can't do machine learning and be a web server at the same time in any other language nearly as well.

19:35 Michael Kennedy: Unless you want to try to write it in R, which I hear is theoretically possible, but I'm not sure it's a good idea.

19:40 Glyph: Yeah, well, it comes down to that same thing. I can guarantee you R is not the second best language for writing a web server.

19:48 Michael Kennedy: That's right, that's right. This portion of Talk Python To Me has been brought to you by Rollbar. One of the frustrating things about being a developer is dealing with errors, relying on users to report errors, digging to your log files trying to debug issues, or getting millions of letters just flooding your inbox and ruining your day. With Rollbar's full stack error monitoring, you get the context, insight, and control you need to find and fix bugs faster. Adding Rollbar to your Python app is as easy as pip install rollbar. You can start tracking production errors in deployments in eight minutes or less. Are you considering self hosting tools for security or compliance reasons? Then you should really check out Rollbar's Compliant SaaS option. Get advanced security features and meet compliance without the hassle of self-hosting, including HIPAA, ISO 27001, Privacy Shield, and more. They'd love to give you a demo. Give Rollbar a try today. Go to talkpython.fm/rollbar and check them out. I definitely want to touch on a few of your projects before we get into shipping software 'cause you got some popular projects and you've got to package them up and ship them. Maybe we could touch on like probably the best known one is Twisted and the stuff that spun out from Twisted, right?

21:03 Glyph: I think that that will probably hound me to my grave is the most popular thing I've ever done, yes.

21:10 Michael Kennedy: Yeah, you don't get a pick, right?

21:11 Glyph: Indeed, we do not.

21:14 Michael Kennedy: Maybe tell people what Twisted is for those people who don't know.

21:17 Glyph: Twisted is an event-driven programming system networking engine for Python. It's a thing that basically if things happen on the network or over the passage of time and you want to get a method called on your object as a result of that, Twisted can help you out. It's got web client and server. It's got an IMAP client and server. It's got a DNS client and server. It's got a whole bunch of protocols implemented and it's also got a lot of stuff that lets you kind of combine asynchronous control flows in various interesting ways. It's kind of almost the original framework for this. I hesitate to say that because there are a lot of precursors that Twisted drew from, but a lot of those precursors were much more special case and kind of purpose specific. Whereas Twisted is like the concept of Deferred originally came from Twisted and Deferred eventually moved kind of front end in the JavaScript and eventually Deferred became Promises, and Promises are obviously a very popular thing for JavaScript developers to use these days. Twisted is both new and dynamic. There's lots of interesting stuff going on in that community, lots of modern practices being adopted, but it's also very kind of venerable and pretty old as a non-legacy technology goes.

22:34 Michael Kennedy: Yeah, it was definitely one of the original ways to write this style of programming that you talked about in the beginning where it's kind of I/O completion ports, not just 100 threads running next to each other, right?

22:45 Glyph: Right, and one of the things we did, we actually presented a paper at USENIX at the very beginning of the Twisted project talking about the need for kind of a high level interface to things like I/O completion ports. 'Cause at that time, like FreeBSD was adding kqueue and Linux was starting to add epoll and Solaris had its own multiple action of mechanism. But of course...

23:06 Michael Kennedy: You guys are like, stop the madness, we can't have all of these.

23:09 Glyph: Actually, we were kind of almost saying the opposite. We're like, this is great. All of these like hyper optimized low level kernel things. And I guarantee you, no one will ever write an application to talk to one of these. You're going to get the three biggest servers out there do it and then no other app will ever be able to take advantage of your custom kernel special thing. We need a way for apps to write against a common API so that when the kernel goes off and does some crazy new thing, we can actually use it without having to rewrite all of our code from scratch. There is definitely an element of stop the madness from the perspective of stop putting all of these new .h files that I have to include, like they're not going to, they're not helping me.

23:48 Michael Kennedy: Yeah, that sounds really interesting. Twisted's still going strong, huh? People are still doing interesting new things with it?

23:56 Glyph: Oh yeah, absolutely. One of the interesting things about it which I did predict when Async I/O came out, there were a lot of people who don't like Twisted for one reason or another and who are sort of sharpening their knives when Async I/O came out. They were like, oh great. Now, we don't have to use Twisted anymore. It's like, it's true if you don't like Twisted, there's more options now. There was a sense among many people that it was going to kind of the be the doom of Twisted. Oh no, there's this event loop in the standard library. But one of the interesting things that came out of that was that event loop is just one small piece of what Twisted did and we did not want to have to be the kind of standard bearers for all event-driven programming in Python. We wanted to provide like a nice little suite of protocols that work together nicely and tools to help you with testing and generally be like a framework, but it would sort of be like, if Django let's say were the only way to speak HTTP from Python, and one day, somebody came out with a second web server where there is like a web server in the standard library...

24:58 Michael Kennedy: Yeah, watch out, there's Bottle. Now, everyone's going to just stop using Django.

25:02 Glyph: Exactly. It's like, no, Django does like one or two other things that you might want to do. And Twisted's the same way. We've actually seen a big uptick in activity and interest. A lot of other people starting to pick up Twisted. Because for a long time, there was a vacuum in the standard library. It really wasn't clear how Python was going to do concurrency, and now that the answer is like Async '08 event-driven I/O. That's what's coming, that's the future. It's like, oh okay, well, Twisted's been in the future for the last 15 years.

25:30 Michael Kennedy: That's cool. Are you thinking of or can you already access it in Python 3 with like async in await type programming model?

25:38 Glyph: Oh yeah. I believe we've been adding new features every release for the last couple of years. I'm not sure exactly where everything landed. But in 17.5, which is a release that just came out I believe yesterday, we have full support for converting Deferreds into Futures and creating Futures from Deferreds. So, not only can you use async await which you'd been able to do for a while with Twisted, you can even use Twisted in the middle of an Async I/O app. You can switch to using Twisted library code and then switch back anytime you want. It fully integrates at every layer. It's got...

26:13 Michael Kennedy: Wow, that's awesome.

26:15 Glyph: It's got coroutine support for interacting with Futures. It's got event loop support. It's got an asyncio reactor that you can use, so you can map Twisted's event loop on top of Async I/O. There's txtulip which is a third-party thing that's been able to do this for a really long time even before Tulip became Async I/O and got int the standard library. Twisted is always been a very integration focused project. I mean, we speak gross difficult protocols like IMAP so you don't have to and we do the low level plumbing so you don't have to, too.

26:49 Michael Kennedy: Yeah, that sounds really, really nice. Two quick comparison questions. First of all, what do you think of the some of the new frameworks coming out like UVLoop and things like Sanic and Japronto and those efforts there?

27:03 Glyph: I think it's great that everybody is learning this new paradigm and doing cool experiments with it. I do think that there is maybe a slightly odd perspective where people are looking at something like Sanic which is brand new and kind of named after a weird meme to be equally kind of feature complete and robust as something like Twisted. I think that in many cases, the comparison is almost unfair because like, it's not like Sanic comes with its own DNS server, right? There's a lot more stuff in the Twisted ecosystem and in the Twisted core.

27:42 Michael Kennedy: I think both of those projects, personally, I think they're pretty interesting and they look very promising, but I don't feel like they're close to ready yet.

27:51 Glyph: I mean, they're certainly ready for some things and I wouldn't want to discourage people from trying them out but I also think that it's important to kind of calibrate your expectations. I mean, there are many things that Twisted still doesn't do right after 15 years of trying to do this. And so I think that, there's also like Curio and Trio, like weird new event loop models that people are experimenting with.

28:12 Michael Kennedy: Yeah, that's my second question. Curio also sounds a little bit similar to Twisted in that it's trying to really be the loop and chop different protocols and network and stuff like that.

28:23 Glyph: I mean, I think that the idea behind Curio and Trio was kind of to invent a new style of programming, and Twisted has this kind of unofficial policy of no new ideas. We don't implement like novel or original. Everything that Twisted does at a fundamental level is something that my dad was doing on Stratus mainframes in C in like the '80s and '90s. We are not as focused on kind of being a research platform, which means that there's less interesting new agey stuff in Twisted that might spark an interesting research direction. But it also means that you can kind of let Twisted be boring and make your application be the interesting part.

29:07 Michael Kennedy: Yeah, well, sometimes boring is good if you're building on top of it.

29:10 Glyph: That is our hope.

29:12 Michael Kennedy: Do you want to just maybe touch a little bit on some of your other projects that you want to mention? Then we can talk about shipping software.

29:18 Glyph: Sure. A couple that I think are always need to be mentioned in the same breath as Twisted are Klein and Treq. Klein is like Flask but for Twisted. Treq is like Requests but for Twisted. They're a little bit different than Flask and Requests in certain ways, but if you want to build like a standard service-oriented app or like a little web app with Twisted in an event-driven way, those are pretty much almost required 'cause you're going to need to not just listen on HTTP like Twisted. The web server that comes with Twisted, you should think of like as a competitor to Apache or Nginx, not Django or Flask. It's a web server and it's designed to be, the API that it offers is designed to be kind of like a configuration language for composing resources together. Whereas like Klein is an app platform. You can write a little server with it.

30:07 Michael Kennedy: Right.

30:08 Glyph: Then there's also txacme which is based, I made a project called txsmi which is just sort of designed to make it easy to dump your SSL certificates into the files without caring what order their private key and the chain certificate and the nine other things you seem to need to dump into a SSL configuration. You can just put them all into a file in any order and call it like yourdomain.pam and it would automatically get uploaded. Then Tristan Seligmann built on top of that with txacme which is a Let's Encrypt implementation for Twisted so you can just like combine those two things and get a service that automatically listens on an SSL port, automatically provisions certificate and gives you nice async server and client web APIs.

30:50 Michael Kennedy: That sounds like a fun set of building blocks.

30:52 Glyph: Yeah, those are the ones you should definitely check out. We've also been trying to make an effort to move some of the data structures that Twisted uses that have nothing to do with event-driven programming, that are just kind of useful Python library functionality out. Mahmoud Hashemi recently extracted our URL object into a project called Hyperlink. Hyperlink got adopted partially as part of that inter-project communication and collaboration that I mentioned at PyCon into the Python HyperOrg. The name and the fact that the org is called Hyper is just kind of a coincidence, but people I'm sure are going to assume that it's called Hyperlink 'cause it was made by Hyper, but that's a really handy object if you need to like parse URLs like a web browser does, like dealing with Unicode in all the different places that Unicode can show up and parsing percent escape stuff into like human readable text, that's a handy library to have.

31:45 Michael Kennedy: That sounds nice. That's the kind of stuff you don't want to mess with with like weird escapes people can do to like get around security and all sorts of stuff, right?

31:53 Glyph: Oh yeah, you can find slashes in all kind of strange places. Hyperlink, because it kind of parses like a browser does, it lets you do all kinds of interesting stuff, like interesting transformations where you can say there's like a click method that would take the URL and give you a new URL as if you had clicked on the given string in an HREF link looking at the page represented by the first URL, which is the kind of thing where if you like doing web crawling or indexing web documents, it's the sort of thing you have to be able to do a lot. But there's a couple others. There's also incremental which is we have a version object in Twisted because we we're very concerned with backward compatibility and long term supports for various things that you need to be able to track versions, so we spun that out to a different library. There's also Eventually which is because we're concerned about compatibility, compatibility means you need to have deprications, and so Eventually is a library for depricating stuff and eventually removing it, which is where its name comes from.

32:53 Michael Kennedy: Yeah, I really like that project.

32:54 Glyph: We're hoping it becomes a little more popular because in such a high level dynamic language as Python, you need to be kind of disciplined about making sure that you communicate to your users like, hey, this is going away. It's going to stop working in so many versions and like this is perhaps seguing into our main topic here. It's important to be able to do that because you want people to be, if they're packaging applications including your stuff, if it's part of their requirements.txt, you want them to be able to upgrade regularly but you also want them to know how much technical depth they've got pending and like how much work they have to do before they can upgrade again.

33:27 Michael Kennedy: Yeah, and you don't have the compiler with a depricated flag right there, right? So you basically have this runtime thing that does something similar, which is cool.

33:37 Glyph: Exactly.

33:38 Michael Kennedy: Yeah, nice. Hey everyone, this is Michael. Let me tell you about Datadog. They're sponsoring this episode. Performance and bottlenecks don't exist just in your application code. Modern applications are systems built upon systems, and Datadog lets you view the system as a whole. Let's say you have a Python web app or any Flask, it's built upon MongoDB and hosted and scaled out in a set of Ubuntu servers or Nginx and uWSGI. Add Datadog and you can view and monitor and even get alert across all of these systems. Datadog has a great getting started tutorial that takes just a few moments. And if you'll complete it, they'll send you a sweet Datadog t-shirt for free. Don't hesitate, visit talkpython.fm/datadog and see what you've been missing. That's talkpython.fm/datadog. You gave a really nice talk and you structured it in the form of a play, more or less, at PyCon 2016 called Shipping Software to Users with Python. This is something I'm also super passionate about and I feel like we've seen the growth curves of Python really, really taking of in like surprisingly even in the last few years becoming much more popular than it was. But I think if we had a better deployment story, it would even be steeper, that curve.

34:52 Glyph: I totally agree. This is is one of the reasons that I wanted to write that talk in the first place was that I was and am really passionate about Python being available to people kind of where they are. One of the things that drove me to talk about deployment is it's cool that Python's usable as a web service thing. It's certainly been a huge driver of economic activity and scientific discovery in that kind of role, but there's also something just tremendously empowering about being able to program your own computer and make it do useful stuff for you. The problem with having a broken deployment story is it seems like a kind of arcane technical solution of system deployment or application packaging, but when you think about it, it's really just like some fifth grader makes a hilarious guess the number game and then they want to share it with their friend. Like, what do they do at that point?

35:49 Michael Kennedy: They go to their friend's house and they set up their machine to run it.

35:51 Glyph: Right, exactly. They take their laptop over there because that's all you can do these days. And so that often drives educators away from Python because if they do it in Java or they do it in, 'cause in Java it's a little bit weird, you often have like a strange runtime environment that all the students have to set up. But if it's in JavaScript, they just load the webpage and there it is, right?

36:13 Michael Kennedy: Yup. Swift is making inroads there as well with their Playgrounds and all that kind of stuff as well.

36:18 Glyph: Exactly. Swift is, as amazing as Playgrounds are, there's a lot to like about them, it's almost the thing that I was, it's sort of the nightmare scenario that I'm afraid of which is why I really think we should get our deployment story together in the Python community, which is that it kind of draws this line between technical people and non-technical people, and if you're like a real programmer, you ship to the app store and if you're just playing around, you use Playgrounds. That dichotomy creates the sense that people are not empowered to do like real programming or make real apps. I think there's two sides to that. Like, Apple has also done quite a bit to reduce the barrier to entry on shipping the software into the app store. It's a lot less expensive than it used to be. It's a lot less effort than it used to be. But it is still this hurdle that for example, kids are just never going to get over. Whereas when I was learning to program as a kid, the sense that grown up professionals, serious software company like Broderbund could make something and put it on floppy disks and then you could do the same thing was very powerful. It gave you a sense that you could do something real in a way that kids don't get to experience in a lot of different levels, right? Like if a kid wants to be a doctor when they grow up, they can't like kind of do some surgery just to try it out. But as programmers, we...

37:37 Michael Kennedy: That's right, we hope not.

37:39 Glyph: Exactly. But as programmers, we have this luxury of the safety of being able to potentially make something that's like real and useful and shareable, but if deployment doesn't work, you don't get any of that.

37:51 Michael Kennedy: Yeah, absolutely. And on the other end of the spectrum, if you go to the pro places and you say, let's write some, especially the one that really strikes me is, let's write a GUI app in Python that we're going to ship around. Like, its certainly is possible and we'll talk about some of the tools that you can do it with, but it's not as easy as some of the other, here's the thing I compiled, and you just take it and it just runs stories.

38:20 Glyph: Right, it's really not very streamlined, and it is getting better, but again I guess we can talk about how.

38:25 Michael Kennedy: Yeah, absolutely. You said that you had your talk sort of focused in different areas. Like you were focused on maybe the people who create libraries, the people who run stuff in the cloud like web developer types, and then this third one where we talk about either from a kid perspective like learning or from a pro perspective, shipping stuff to end users. Was that basically the three areas you were focusing on?

38:53 Glyph: More or less, yeah. I mean, I also talked a little bit about the people maintaining the infrastructure in Python, but that's kind of more of like telling them to consider those other three perspectives.

39:04 Michael Kennedy: All right. Okay. You went through some of the stuff that you can do for the server side, and that's pretty straightforward. We can make packages. We can use things like Twine and ship those up to PyPI. We can package them as wheels. We have requirements.txt that takes these. That kind of stuff, I don't know it's super clean, but it certainly works well once you get used to it. I've had a few people who are learning Python web development come to me and say, this is all really hard. How do I make this stuff work? I don't know what to do. Maybe it's not hard but it, and you kind of touched on this, it's like it's confusing or not clear what the steps are.

39:42 Glyph: One of the problems is, we actually have a fair amount of documentation at this point, except, there's so much of it now. We trip over our own feet trying to explain to users how to do this kind of stuff, and so you end up with something where it's like, it's actually pretty straightforward to assemble a Docker container, let's say. But you'll find so many half-written tutorials and there's so many concepts and so many separate tools that you have to chain together that most people do not end up on the fast path. They do not end up just doing the one thing that's going to be easy for them. I think that that's getting a little bit better in terms of like what things people are linking to and what you happen to find, just thanks to page rank kind of finding the better articles over time.

40:24 Michael Kennedy: Yeah, it's starting to stabilize a little bit as well.

40:27 Glyph: Yeah, it's definitely been a long slug to get here though and where we are is definitely not ideal. One of the points I think that I didn't make in the talk that's probably worth talking about is that Python is not a deployment platform, and so everything that we do, the things that I talked about in the talk, the things that we're talking about now, distutils setup.py, virtualenv, right? These are mostly developer tools. They're for assembling the software into some shape but they're not really for shipping it. One of the reasons that cloud Python has gotten a lot better is that the assumption about where you're deploying your stuff is shifting towards containers. Docker is a deployment platform. That is a system with like a pretty clear contract of what goes into a software artifact. It's a Docker image, that's it. There are really no other options. How do you run it? Well, it's got an entry point, and that's it. There's a couple of things you might need to document. I have a whole separate series of blog posts and stuff about type system for Docker and how that needs to be more expressive. But basically, if you hand an ops person a Docker image, they more or less know what to do with it. It's a clear integration point between Python people and systems people. And the tools that we have at our displosal like pip, like distutils, like virtualenv, makes sense in that context and we can pretty easily script them to do the right thing to build the Docker image. But Docker is just one deployment platform. The problem is we need tools that deploy Python to multiple different deployment targets and not try to come up with a "Python deployment target" that is universal and works.

42:15 Michael Kennedy: Right, like for example in the GUI space, just 'cause you get a Docker container, you can't like ship that to end users and go, now run the Docker container.

42:24 Glyph: Right, there's no Docker run on my iPhone, right?

42:27 Michael Kennedy: Yeah, exactly.

42:28 Glyph: That shows up in a couple of different places. Any kind of GUI front end thing suffers from this. The GUI front end stuff just can't be shipped in that style but also it can't be as easily shipped with Shell scripts. The reason Docker works so nicely with the Python toolchain is that most of the things that we sort of been building up from are a combination of dependency on some Linuxy, Unixy type tools that are the low level stuff and then a script to assemble them into a coherent like runnable shape. That's pretty easy to do in Docker, but that's a lot harder to do on let's say Windows, right? Where ultimately, you're just trying to get a PE executable or zip file together but the process you need to go through to do that in order to like integrate it with the Windows compiler and all that stuff, you actually kind of have to know about those tools and you can't just like write a little script and copy some binaries around.

43:25 Michael Kennedy: Right, the system's got to be in a place to receive it, in the first place.

43:29 Glyph: Exactly.

43:31 Michael Kennedy: So you kind of taunted the audience a little bit with this thing you called pip.app. And you're like, okay, here's how we're going to deploy it. We're going to run this thing called pip.app, we're going to select our program, we're going to select our dependencies, choose the operating system, and we'll just package this puppy up, right?

43:46 Glyph: Yeah, and why can't we have it, right? This seems like a kind of almost obvious tool. I think that one of the reasons that nobody's made such a thing is that you have to care a lot about the general case and you have to care about the end user experience. And in the kind of open-source world scratch your own itch, you're never that end user, right? You've already gotten to some point that's further down the line than just like, I don't know what Python is. But if you were thinking about making this in like a 1990 style desktop app where like a user finds your floppy disks in the store and can't talk to you after that, you'd better make it work just from the beginning. And if we thought about it more like that, I think that people would end up having to Google for those confusing virtualenv articles a lot less often. Part of the problem is just that we don't have that many volunteers who are working on this stuff.

44:42 Michael Kennedy: I think you're touching on a super important point. I've spoken to several people about this. I sort of open this section by saying, if we could get the deployment, especially the client side UI style deployment better, it would just open up an entire new avenue of growth for Python and where we can really leverage apps. I was talking to somebody who said, look, this is a problem that is super important we need to solve. He's like, well, it's just not the thing that I find people who do Python do very often. We don't run into this problem. We don't run into this problem 'cause it's so hard that we don't even consider doing this most of the time, right?

45:16 Glyph: Exactly. And it's really kind of sad because in fact, it's not that hard. Like 80% solutions exist for so many of these problems. I mean, Dropbox is on like, they always have that huge number. I'm not sure what it is. Like 10 billion computers or something. Like they're running everywhere. That's a Python desktop app. So clearly, it's not going to wreck your company if you decide to use Python. They had to do some work to make that work and they made it work for them. I absolutely don't want to criticize Dropbox for not doing enough to help the open-source community because they've definitely done tons of stuff, lots of really interesting work has come out of there, but at the same time, if somebody else wanted to follow on their footsteps and make another Python app that would deploy to like a zillion machines, they'd kind of have to solve a lot of the same problems again in a different way because the Python ecosystem has moved quickly and has got a lot more and better tools than when Dropbox was first started. There's just problems. Everybody who's like really, really concerned about this problem is already too busy already. I gave this talk to try to help newer people to the community try to undertand the problem because as we spent quite a while talking at the beginning of the show, I've already got a lot going on. I've got to keep the books for our customers and write software to automate financial stuff at work and then I'd come home and then I maintain Twisted and I maintain Klein and I maintain Treq and I do all of these stuff. I try to keep the peace in the community and make sure the mailing list doesn't go down and keep our servers running. At the end of all that, I just don't have any time to start a fairly ambitious project to fix all of these problems.

47:04 Michael Kennedy: It doesn't sound to me from what you described just now that you individually have this problem. I feel like there needs to be a couple of major companies or a couple major projects, like we need to build this app and ship it this way, and we need to just fix the tooling to make it work for us and take the community along with us.

47:25 Glyph: Yeah, I feel like the person who's going to have this exact problem is kind of hard to imagine because if you want to ship a desktop app and you have interesting enough logic and enough investment in Python, there's lots of ways you could just do that today. You can grab py2app and you can grab py2exe and you can work around the same seven bugs that everybody who uses them has to work around, and eventually after you've had a couple of professional programmers hammer on it for a few weeks, you'll have a deploy pipeline that works for you, you put it into your continuous integration system and then you're good.

48:00 Michael Kennedy: Right, once you get around that, you make it work, then you forget how you even made it work, right?

48:04 Glyph: Exactly. When you do something in your code that makes it stop working, then you notice and you back it out. This is actually one of the problems I didn't talk about in the PyCon talk was that one of the reasons this is so pernicious is that we don't tell users to start by deploying. Because Python is so flexible and it's so empowering to just get in there and make a module and then make another module next to it and edit the first one to import the second one and then just change it and maybe even reload it on the fly. There's all this cool stuff you can do with that and with Jupyter Notebook, just type in code into cells and never even put it into a module, that by the time you have something you want to deploy, you might have made a dozen mistakes about how you're loading resources or about how you're getting your code installed or the layout of your repository. It's pretty easy to end up in a place where you can't even write the setup.py because all your modules are expecting to import stuff from like three different paths at the same time. At that point, you have to like back out of whatever weird structure you've got yourself into and then recreate something sensible.

49:16 Michael Kennedy: That's where you throw up your hands and go, I'm never going to ship this thing. Like, I'm not going to rewrite my app for it. It's just not going to work for real apps or whatever, right? That's super interesting. You're basically proposing that like if you know that you want to use py2app or py2exe or the cross platform PyInstaller or some probably combination thereof, write your hello world and then create your hello.app and start building them up together. Is that what you're proposing?

49:44 Glyph: Yeah. For anybody who's out there who's thinking I want to make a desktop app, I want to make a mobile app, I want to make a low level system utility. One of the categories you haven't talked about too much yet is just like if you're making a program that needs to like be a system control thing for Unix systems at a level lower than like Docker, if you want to write your own SSH server or something like that that controls a system, you also can't ship it as a Docker container 'cause you needed to like live on the Debian or the Red Hat system, you need to ship it as a package. So if you're in one of those categories like, I want to ship this command line thing or GUI app, and you're thinking, I'm definitely going to want this to have a GUI, I'm definitely going to want this to ship as part of Homebrew. I'm definitely going to want to ship this as a Debian package. Whatever it is that your deployment target is, deploy your hello world to that target and get like a Travis or some kind of continuous integration system to start running it for you, and doing that build and making sure that at least basically works because it's really, really easy. Once you've got that set up, once you've got that pipeline working, it'll pretty much keep working. You might have the occasional problem where like some third party library has a weird shared object that you need to install in some funny way, but if you know that at that moment when you're adding that dependency or installing that tool, if it breaks right then, you could say aha, I'm not going to use that or I'm going to get in touch with their maintainers and say like, why is this broken? As opposed to writing a big complicated app, getting everything all done, and then ramming headfirst into this brick wall of like, now, how do I put this on somebody else's machine? 'Cause one of these things that you don't realize is you're making all kinds of decisions about how to get it to run on your machine. They seem like totally sensible, right? You have icon.png and it's in the current directory so you just open up icon.png and you read it, except, what's the current directory? How do you know that it's in your git checkout 'cause you're git checkout is not going to be there on your target machine. So unless you're going to ship dev tools to everybody, you have to think about where does that resource come from? And that's exactly the kind of stuff that trips people up with packaging.

51:59 Michael Kennedy: Yeah, that makes a lot of sense. Actually, I hadn't really thought about it that way. Certainly makes a lot of sense. It feels a little bit like, let's write 100,000 lines of code and then run Flake8 on it to find the problems.

52:12 Glyph: Right, just the first time.

52:14 Michael Kennedy: Exactly.

52:15 Glyph: And the day before you ship. Or another way to think about it is like, let's write 10,000 lines of code and we'll make sure that all unit tests pass but we'll never run it. We'll just assume that it will work if we run it. But of course, you'll run it that first time, it's like, oh, you forgot a couple of unit tests. And the build, your build pipeline is the same way. When I talked about pip.app, what I'm really talking about is not just like let's make it easy, but let's make it the first thing you do. So if there bugs in it and there are problems with it, you're just doing it constantly so you notice when there's an issue.

52:54 Michael Kennedy: Yeah, I think that's really a great advice. There might be a lot of people listening here who don't, are not familiar with py2app or py2exe and how these things work. Basically, you can take this py2app utility, so let's stick on the macOS for a moment. It will create, like kind of embed a Python runtime that you need. All of the dependency packages that you need, your code, and package it up in a way that looks just like Safari or something and users doubleclick it and it can run, right?

53:25 Glyph: Exactly, yeah.

53:26 Michael Kennedy: They don't even necessarily, it doesn't even matter what version of Python they have, whether they have Python. They get the app, it runs, because it's self containing like other runtimes like Go or certain Windows EXEs.

53:40 Glyph: Yeah, and this is exactly what I meant when I said that Python is not a deployment target. macOS is a deployment target and the Mac says what you have to do to be an app. And py2app just makes Python be that thing.

53:52 Michael Kennedy: Right, here's you plist, here's your resources, all that kind of stuff.

53:55 Glyph: Exactly, and it just bundles it up into the shape that Mac Os expects without any regard for like what else is on the system. Really like it goes sort of famous for producing static executables where they include all of your Go dependencies, but oddly enough, Python is sort of in a better place for many operating systems than Go would be because what Go let's you do it is lets you put all of your binary resources into one file. But that's kind of a red herring. The thing you actually need is to put it into the shape the operating system expects which is almost never one file. Windows is the one place that that's true, but even that put on Windows, there's a whole system for like tacking on resources into a .exe. I believe Windows loads from the end of the binary so you can actually tack, like you can, your binary can just be a zip file and then you stick the executable on the end and suddenly it works and it launches. I may be mixing that up. I know that Unix is one way and Windows is the other. One is the beginning and one is the end. That kind of thing is like, once you know how you want your app to run, it turns out that actually on Windows and on the Mac, you often need like shared libraries or DLLs or resources or things that can't be put into a single binary. Like an app on the Mac cannot be a single binary. It has to be at least an info.plist in an executable. And practically speaking, you need a dozen other resources like icon files and other things. Similarly, even if you're just talking about command lines, you think single file or binary, you think, oh that's really good for the command line, that's really how I want to run my stuff is I just want to have this one binary. That can make the deployment process easier, but how do people actually deploy stuff onto the command line on the Mac? Do they like curl a binary and chmod plus exit? No, they brew install it. Homebrew has a whole complicated multifile setup where the deployment target is Homebrew.

55:52 Michael Kennedy: It's a really interesting point. In your little py2app graphic you had the various targets and one was Ubuntu, one was Windows, one was macOS, but another was a Chocolatey install. One was a Brew install. One was an aptitude package and things like that, right?

56:07 Glyph: Yeah, and each of these kind of define its own deployment surface. The reason that you don't want to just be randomly curling some Go binary onto your system is because that initial install is just one thing. You've also got to have an update process and that update process probably has to be cryptographically signed or something. Like somebody needs to be in charge of making sure there's a secure pipeline to deliver it to all your users. Then that has to like run on a CDN and to make sure you don't have too much load. By the time you're done with that, you got like JSON files that are describing your metadata and hashes and all kinds of other stuff which don't fit into that one binary anymore. Go can kind of get you started and it can make a lot of parts for this pipeline easier but one of my favorite examples of this which I don't think I mentioned at the talk 'cause I think I found out about it somewhat later is asciinema which is a website, it's a really cool website for sharing like screen casts of terminals. Have you ever used it?

57:03 Michael Kennedy: No, I haven't, but you have to put a link at the shownotes for us that we could check it out.

57:06 Glyph: Definitely. The idea is it's just this little thing. It's like a social place where you can share your various like Emacs and Vi sessions.

57:14 Michael Kennedy: Nice, that's awesome.

57:15 Glyph: When you do this, they have a local TVUI recording tool which they originally wrote in Python. Then of course because Python makes deployment hard, they rewrote it in Go from scratch. Then they rewrote it again in Python because they just went to Python 3 and they shipped it as a thing you brew install because it turns out going into this single file Go binary didn't really help solve their problems that much. Go is a lot lower level language so it didn't like, there were a lot of things that they lost as well. By just kind of going straight to the never mind the single file thing, we just want to make sure that this is brew installable, it got to what their users really wanted which is way to get it installed that's like easy to explain, doesn't require a lot of sysadmin knowledge to keep updated.

58:02 Michael Kennedy: It doesn't require a sudo.

58:03 Glyph: Exactly, yeah. You just type the brew install command. The killer thing about the way that the thing that people think they want when they hear a single file is when you got a Python install and you're doing like pip install to get your tools installed, the problem with that isn't that it's multiple files, it's the sharing all kinds of files. So you do a pip install A and you do a pip install B. And you just need tools from A and B, they can conflict with each other. pip install B breaks A because they're all reading from the same location, all their libraries are piled on top of each other and there's no like dependency checker in pip which makes sure that A isn't going to break. The versioning metadata isn't fine-grained enough for A to even know that it's going to break when B gets pip installed. It's just too fragile to try to share all of the stuff between Python libraries. It's not like a .so where you can have like a very, very strict binary interface that never ever changes. Anything can change in any Python file anywhere and potentially break every user at that library. You pretty much need to do a full on test run of your application to make sure that it's not going to break with a new version of a dependency. It's that sharing that becomes a really big problem.

59:18 Michael Kennedy: Yeah. Sometimes you can't even avoid that because you're using two packages. They might both use a dependency but they were building in different versions of that dependency or something, right?

59:27 Glyph: Exactly. There's nothing really wrong with that even. When you described the problem, it sounds like, oh, that one library should have ran its tests.

59:36 Michael Kennedy: Shame on them.

59:37 Glyph: It's not their responsibility to make sure that whatever random versions of libraries on your system happen to work, they have a continuous integration system that says what they work with. We've gotten a lot of improvements in the Python ecosystem for library developers to put out more and more robust artifacts, but that last mile of like, put the application together, packaging everything together. One tool that I really like that I use a lot is PIPSI, pip S-I, which I don't think is actually an acronym for anything but I pretend that it stands for pip script installer. What it does it you do PIPSI install HTTPie, let's say, or PIPSI install boto3 or awscli or something. It will install a new virtualenv that contains just the script that come from that top level package and then like synlink those scripts into a shared directory but kind of hide the virtualenv from you. So every single tool that you need is installed into a separate virtualenv with its own requirements installed into it. It's still kind of a developer installation tool because you already need Python, you already needed that virtualenv. There's like a little bit of set up that goes into installing PIPSI in the first place and bootstrapping it. But then once you got it, at least your stuff doesn't break all the time.

01:00:59 Michael Kennedy: Yeah, it lets you play with stuff more aggressively or without care, right?

01:01:04 Glyph: Right, it's isolated and it's kind of installed the way that it should be which is that each app has its own dependencies, 'cause again, Python is not a platform. The compatibility guarantees that a library can make in Python are not strict enough that you can have users upgrading their own stuff and then thinking it's going to work afterwards. You need to ship a user all of the libraries that you tested with.

01:01:26 Michael Kennedy: Yeah, that makes a lot of sense. One thing that you held up as a good example is Briefcase.

01:01:31 Glyph: I think that Briefcase is headed in a really good direction. I think that that project has set up some really good infrastructure in terms of like how they think about the long-term solution to the problem in that they have great onboarding for new contributors, they show up at lots of events, they have a great presence at PyCon. They're playing a long game here. They recognize that this is very big complex problem that's going to require a lot of people to solve and that it's also multi-layered. One of the parts of the BeeWare project is not just Briefcase but Batavia. Batavia is this transpiler from Python, or not transpiler, it's a Python virtual machine written in JavaScript so that you can ship your PYCs into a web browser. That doesn't seem like it's really related to Briefcase at first until you realize the vision that they're pursuing is ship Python anywhere, write all of your code in Python all the time, which is really the vision that I'm also sympathetic to. But unfortunately what that means is because they are taking this very long view of the problem, it's probably not the thing that you would use right now for everything. It does have some really great iOS build templates, so if you're going to do a mobile app in Python, it's probably the best place to start.

01:02:52 Michael Kennedy: Yeah, I agree with that.

01:02:53 Glyph: Russell Keith-Magee did a bunch of the build work on Pythonista, so I don't think that Pythonista, the iOS app that runs Python is itself like straight up deployed with Briefcase but it shares a lot of that same technology and so there's kind of at least a proof of concept that it can work really nicely. I don't know if you've played...

01:03:12 Michael Kennedy: Yeah, I've played with Pythonista. It's a cool app for sure. Basically it's a Python editor and runtime for iOS. Maybe also on Android, I don't know about Android.

01:03:22 Glyph: I'm pretty sure it's just iOS.

01:03:25 Michael Kennedy: It's impressive to me that that's even possible, given all the restrictions that Apple puts on executables.

01:03:30 Glyph: Yeah, I mean, they probably wouldn't like it if you started shipping apps to people in the form of like that really, really long base64 encoded.py file. There is some tooling to actually help you write an iOS app inside Pythonista on iOS. You do need to have like have a Mac to kind of bounce your project off of because you need to do the build of an iOS app on a Mac. But I've done that. I've downloaded the templates and like made a little hello world out of the scripts that come on Pythonista. Which is actually kind of another interesting reflection of the same problem from a different direction which is that Pythonista does not have any equivalent to pip, so it's actually very hard to write full apps that take advantage of the Python ecosystem because you can't, your build pipeline can't include that kind of like and install those requirements.txt file first.

01:04:25 Michael Kennedy: Python is less fun when you have no import.

01:04:29 Glyph: Yeah, I mean, Pythonista kind of cheats a little bit because it just gives you a thing that already includes like Requests and like NumPy and some pretty big chunks of SciPy, I think, are also in there. It's just kind of like, here's about a third of the interesting stuff on the Python ecosystem, just in a big box. But yeah, the idea that like you can grab this from the staggering ecosystem that Python has to provide for you, that's really what makes this interesting because if you're just going to write an app, do it in a Swift Playground and then copy the Swift code to a Mac and write all of that yourself, it's not that much different than Python. Python might be a little nicer but once you can pull in stuff, like you can pull in Jupyter or something like that and you can have like live introspection of your objects in your app or you can pull in something like NumPy and have this tremendously powerful numeric library at your disposal, it starts to makes the investment in Python make sense.

01:05:30 Michael Kennedy: Yeah, yeah, it definitely does. I think Swift is a fine language. I just think their equivalent of the standard library is really, really not a great story.

01:05:38 Glyph: It's pretty bare bones, yeah. The way Python can compete with that is by being yet more robust in the library ecosystem and not by kind of cutting itself off of the knee isn't saying like, here's your really streamlined deployment experience that only works if you write all of your code yourself.

01:05:56 Michael Kennedy: Right, yeah, absolutely. We definitely don't want to see that. Glyph, this is a bunch of great advice and hopefully, a lot of inspiration for people that like you said, go that last mile to kind of close the gap and make this really sweet deployment story.

01:06:12 Glyph: I certainly hope that folks get inspired to go do that. I think that if you are kind of, if you feel like you're just a regular developer and you don't feel like you could take on something this ambitious, going and helping out the BeeWare folks get Briefcase into the kind of shape that it can just do a lot of this work itself is probably the best way to help out the community at this point.

01:06:34 Michael Kennedy: Yeah, I agree. They have basically a mentorship program for first time open-source contributors and they're very welcoming there, so it's a nice environment.

01:06:43 Glyph: Definitely. I think that that's one of the things that they might do better than almost any other project I've seen is that they are really focusing on the people problems and making it easy to get involved in the project and make a real impact quickly.

01:06:54 Michael Kennedy: Yeah, it's good to see they're working on this problem as well 'cause they're doing cool stuff. All right, so, I think we probably should leave it there for that, but let me ask you the two questions. We talked a lot about shipping Python code, but if you're going to write some Python code, what editor do you use?

01:07:08 Glyph: We began with the story about my childhood and I feel it's only fitting to end on one. Because I heard you ask this question with a lot of people and they all give you an answer. I have an answer but I think it's important to reflect on where that comes from. When I was 10 years old, my father sat me down, and he said, you're going to grow up and people are going to tell you that you should be a Vi user or you should be an Emacs user. What I have to tell you is they're both wrong, you need to do both editors, because sometimes you're going to be SSHed into some machine that you have, and he probably said telneted, but it's out of the times.

01:07:49 Michael Kennedy: Same basic thing.

01:07:50 Glyph: You're going to be on some console where you can't install anything and Emacs isn't there and the environment is set up wrong, you're really going to want to know how to be able to exit VI. At the same time, Emacs is this tremendously powerful library ecosystem of all kinds of sophisticated stuff that you probably want to be able to leverage to be able to use like an IDE. So I actually learned pretty young both Emacs and Vi and I have tried to take that philosophy to heart in that I have a working Sublime Text configuration. I've been working on upgrading that to Sublime Text 3. I haven't messed around with Atom too much yet, but I definitely intend to. My wife is a big proponent of Visual Studio Code so I feel like we should all learn to use editors in harmony.

01:08:39 Michael Kennedy: Yes.

01:08:40 Glyph: That said, Emacs is the best one and that's what I use.

01:08:44 Michael Kennedy: That's a great story, thanks. A good answer. Notable PyPI packages? I'll throw in Twisted out there for you straight away.

01:08:52 Glyph: Yeah, actually I wanted to talk about, I wanted to bring up one that we haven't already mentioned in the course of this discussion. I think Hypothesis is a package that I would really recommend people check out. I haven't used it very much yet, but what Hypothesis is is it's a generative testing framework. You kind of give it piece of code that you wanted to test and then you give it some variables, some ways that it can generate data or state for that code to be tested in. Then you kind of fire it off and it will figure out what's interesting to test about your code in some ways. I'm making it sound a little bit more magical than it is. It lets you, like if you have, just a function that takes two numbers and like there are range limits on how those numbers are supposed to relate to each other, like there's one case where they're equal to each other and there's one case where the first one's greater and one case where the second one is greater. Hypothesis can, if properly set up, discover that those are the three unit test you need to write and it's a whole bunch of values where they behave one way and a whole range of values where it behaves a different way but it will kind of like minimize and do like a binary search to find the interesting test cases for you. Like I said, I haven't really used it myself yet. But I have been investigating it and it's kind of changed the way that I thought about how tests can interact with each other. I know a couple of my friends have given it a spin on various kind of complex testing projects and it's really helped improve the reliability of their software a lot. It's found bugs in a way that straight up unit testing generally doesn't.

01:10:28 Michael Kennedy: Yeah, it's awesome. I've definitely heard that as well. There's some amazing stuff Hypothesis can do. I actually had David MacIver on Episode 67, so quite a while ago, but talking about that there. I'm impressed with it as well. And it pairs up with the other unit testing things. Like you can use it with pytest or with tox or whatever.

01:10:48 Glyph: Yeah. It's kind of a power up for your tests. It's not like a whole new testing system. People have used it with Twisted's built in test runner and as well as pytest and a whole bunch of other ones.

01:10:59 Michael Kennedy: Yeah. You touched on this just a little bit but one more chance, final call to action. You've got people caring about this deployment story, what can they do?

01:11:08 Glyph: Aside from just like go help BeeWare, the thing to do when you help BeeWare is if you're the sort of person who has like a virtual machine or like the ability to spin up new VMs, run a VM for some GUI operating system, whether it's macOS or Windows and pretend you don't know anything about Python and you want to just write a little GUI application. Find a tutorial and in your fresh new VM with no developer tools installed, try to make an app that you can run on a different VM also with no developer tools installed. The interesting thing about this is, the reason this is my final call to action is if you do this, you will see all of the places that even if you yourself only do like web-based Python and don't do a lot of stuff with GUIs or desktop apps or mobile apps, you will see all kinds of places where you know where the sharp edges are and you know exactly what to do and it's not really written down anywhere. Try to figure out how to take that process, like the parts of the process that you understand that you know how to get done the same way the right way every time. Blog about it, write about it, contribute to packaging.python.org where there's documentation for how to set up packaging, and try to think of whatever you're trying to help with these issues, whether it's BeeWare or just on your own blog, remember where the actual beginning is and don't just write tutorials for people who already have pip installed and already have Python installed and already know the difference between Python 2 and Python 3, just go back to the beginning and think hard about what it would take for a total beginner to get started because there are a lot of total beginners out there that do not have a friend to kind of shepherd them through that wilderness and I feel like were losing a lot of new Python programmers who if only that part were easy, they would be coming to PyCon with us every year.

01:13:11 Michael Kennedy: Yeah, absolutely. That's a great thing for people to try. I totally think it would be helpful to go through that whole exercise. All right, Glyph, thank you so much for being in the show. I really enjoyed chatting with you.

01:13:22 Glyph: It was a total honor to be here and see my name up in the lights next to all those other luminaries that you've interviewed.

01:13:28 Michael Kennedy: I'm thrilled to put it up there. Thanks again. This has been another episode of Talk Python To Me. Rollbar takes the pain out of errors. They give you the context insight you need to quickly locate and fix errors that might have gone unnoticed until your users complain of course. As Talk Python To Me listeners, track a ridiculous number of errors for free at Rollbar.com/talkpythontome. Datadog gives you visibility into the whole system running your code. Visit talkpython.fm/datadog and see what you've been missing. They'll even throw in a free t-shirt for doing the tutorial. Are you or a colleague trying to learn Python? Have you tried books and videos that just left you bored by covering topics point by point? Well, check out my online course, Python Jumpstart by Building 10 Apps at talkpython.fm/course to experience a more engaging way to learn Python. And if you're looking for something a little more advanced, try my Write Pythonic Code course at talkpython.fm/pythonic. Be sure to subscribe to the show. Open your favorite pod catcher and search for Python. We should be right at the top. You can also find iTunes feed at /iTunes, Google Play feed at /play, and direct RSS feed at /rss on talkpython.fm. This is your host, Michael Kennedy. Thanks so much for listening. I really appreciate it. Now, get out there and write some Python code.