00:00 Michael Kennedy: Have you noticed that web development's kind of hard? If you've been doing it for a long time, this is actually easy to forget. It probably sounds simple enough to pick a server-side framework, Pyramid or Django? Create a new project, oh, do I use pcreate, is that cookie cutter, is that manage.py? Install some client-side dependencies with npm, install some server-side dependencies with pip, create a database, well, first, choose a database server, then create a database, and maybe even set up and install the server, the process, and maybe even another machine to do this, connect the app to the database securely, pick a front-end framework, is that Bootstrap or Foundation, integrate the front-end framework into it, pick a theme, now finally, you get to write some code. Ugh, now it's time to deploy. Wait, that doesn't sound that easy, actually, even to me. What if the web were as easy as old-school VB6? You just open a UI, you drag and drop some widgets, you double-click on the things like buttons for events, but with Python code, of course. With Anvil, you'll see that it really is that quick and easy to get started. Meet Meredydd Luff, who's here to tell us about Anvil. This is Talk Python to Me, episode 138, recorded November 6, 2017. Welcome to Talk Python to Me, a weekly podcast on Python, the language, the libraries, the ecosystem, and the personalities. This is your host, Michael Kennedy, follow me on Twitter where I'm @mkennedy, keep up with the show and listen to past episodes at talkpython.fm, and follow the show on Twitter via @TalkPython. This episode is brought to you by Datadog and GoCD from ThoughtWorks. Please check out what they're offering during their segments, it really helps support the show. Meredydd, welcome to Talk Python.
01:50 Meredydd Luff: Thank you very much, thank you for actually having me.
01:52 Michael Kennedy: Yeah, you had some pretty interesting stuff going on at PyCon UK and yeah, I'm super happy to bring it to a wider audience.
01:59 Meredydd Luff: Cool, thanks, yeah, it was really great, that's actually been our first time sponsoring that conference, and it was really quite fun being deluged in people. Yeah, whole bunch of stuff we weren't expecting, we went through a lot of throat sweets in the evening after the conference, but it was really great to see the reaction out there, and I'm really pleased to be talking to you and your listeners, too.
02:17 Michael Kennedy: Yeah, that's awesome. So, I'm super excited to talk about your project called Anvil, which, it's kind of trying to democratize the web again in a really interesting and powerful way, and using Python to do it, but before we get to that, let's just hear your story. How'd you get into programming in Python?
02:32 Meredydd Luff: So, programming I started, I was very privileged, I started around the age of seven, I was introduced to QBasic and for me, it was just this piece of magic, because here I was, sitting at this blue screen, and I could type something in, and it, you know, it was nothing that a seven-year-old's brain couldn't comprehend, and here was this program, it was drawing pictures on the screen, it was interacting with me, and I could make it do what I wanted, and this was magic made real, sitting in front of me. And it was several years before, then, that I worked out that you could actually make a career doing this amazing thing that I'd discovered was the best hobby in the world. Actually, like, literally the reason we started Anvil was that, you know, that experience isn't there in much of the web these days, and as far as I'm concerned, that's borderline criminal negligence from our whole profession, because that kind of magic that can capture a kid's eye is, you know, is what we should be delivering.
03:28 Michael Kennedy: You're right, it absolutely is missing, and so I think you guys are going to bring some of that back, yeah, that's awesome. So that's how you got into programming, how 'about Python?
03:36 Meredydd Luff: So, Python itself actually, I mean, I've like, dabbled, dipped into it, bits and pieces, but honestly, we started Anvil first, it was a very much, you know, we want to bring back this sort of instant-access magic, we want to bring back what you had with like, Visual Basic 6, these tools in the 1990s, and we were like okay, but it's got to be modern, it's got to be an easy-to-use, accessible language, it's got to feel that kind of easy workability in your hands, and we like, looked, and Python was so the obvious candidate. And so I was, I spoke Python as like, you know, fourth language up 'till then, and then I went right, okay, this is clearly the right thing and jumped into it, and I have been loving it.
04:23 Michael Kennedy: Yeah, that's awesome, and so you guys sponsored PyCon UK, it sounds like Anvil's something that you're focused on pretty much day-to-day, is that right?
04:31 Meredydd Luff: Absolutely, so we originally started this a couple of years back now, but it was like an evening and weekends project. It was, you know, we were on our way to a wedding, sitting there in the airport departures lounge going, you know, why don't we have Visual Basic 6 for the web anymore? And that kind of turned into some evenings working on it, which eventually turned into taking one day off a week to work on it, and then last summer we just, honestly, we just stuck it on Reddit and we saw the reaction, went oh, okay, this is really a thing now, okay, we could afford to go full-time on it. And so we worked our way up to it, but now, yeah, this is our thing.
05:07 Michael Kennedy: Yeah, that's a really great story and I think a lot of people feel like the way you get to somewhere where you have, like, your own business or your own sort of thing that you can go do that's not part of your job, is kind of like diving off a high diving board with your eyes closed whereas I think in practice, a lot of these things, like you described, start as like, let's test the waters, let's do some proof of concept, let's slowly build up, and then once you see that it's gaining traction, like, alright, if I could go and do eight or nine hours a day instead of one, think what would happen.
05:36 Meredydd Luff: Yeah, absolute, it's a very much sort of wading in from the shoreline rather than jumping off a diving board.
05:40 Michael Kennedy: That's great. Alright, so we're definitely going to get into the details of what Anvil is, but maybe just give us a really, really high-level picture of what you've built, and then let's talk about your thought that the web is not as accessible as it should be.
06:39 Michael Kennedy: I think you could build a fully-functional web application with hardly knowing any programming. I mean, this could easily be, build like, a really functional web application for your first computer science or programming course in high school or something, right?
06:54 Meredydd Luff: We have people who've done that, right? You know, we've had schools that've just taught their computing curriculum with Anvil precisely because of that, 'cause, you know, for the kids. This comes back to what I was saying at the beginning, you want that instant feedback, you want to have this feeling that I just learned this abstract concept of the if statement, and now here it is in buttons and colors and images in front of me, right away.
07:15 Michael Kennedy: Yeah, so let me see if I can just describe the experience of using this, which, granted, my experience is like, I've only built one little test app and played with it for an hour or something, but still, the idea is, you go to your website, you create an account, you launch a cloud, browser-based IDE, which is really focused on the sort of UI design layer of, UI and data layer of your app, you have a design view and a code view. In the design view, you have a bunch of components and widgets that you can drag over, images, links, buttons, repeaters, columns, all sorts of stuff, you can give 'em names, and then you access them in Python in the code side of your page, right?
07:56 Meredydd Luff: Yeah, that's exactly right. But look, it's not just on the code side of a page, because you can also then pull up a server module, and then you're writing, again, it's just a Python module, but now it's running on our servers. And you can just make function calls as just a decorator for oh, this is something that you should be able to call from your client code, and so you can make Python calls straight from the Python code that runs when, say, when you click a button in that code view, through the server module.
11:51 Michael Kennedy: How do you think we got here?
12:42 Michael Kennedy: Yeah, and then they kind of fixed it in Internet Explorer 6, and then the web kind of suffered from a couple years of stagnation.
12:49 Meredydd Luff: It was years of stagnation, I think that a couple of years of stagnation is, that was actually, arguably, one of the periods of greatest innovation, where it just became obvious how tight the constraints and limits that had been put on it were. 'Cause what actually happened is this wonderful, it's almost a Greek tragedy. You get Microsoft desperately trying to hold back this tide of web innovation so it doesn't become the platform of the future, and then this, you know, around the turn of the millennium, and this is like, famously, product groups in Microsoft, not very well-connected, the Outlook Web Access team, they go, well, wouldn't it be great if you could have, like, Outlook, an email client, in a web browser? And they kind of put enough pressure on to make the Internet Explorer team implement this weird, janky interface for doing real-time calls to the server called XML-RPC requests. And then that gets released in like what, 2002, 2003, and within a year, you've got Gmail and Google Maps.
13:44 Michael Kennedy: Yeah, that just totally broke the dam free, right? That was when the whole Web 2.0 sort of thing kicked in, right, yeah.
13:51 Meredydd Luff: Thing happens, yeah, and so the Internet Explorer 6 stagnation was a bunch of people kind of, it was a belated putting-on of the brakes when the dam had already broken. But the fun thing about this is what it means is, the web, the whole web platform, the whole structure of the web platform is built on the rubble of a platform war of the early 2000s. And so, that's why the ground underneath your feet is so rough, I would argue, because we're still trying to build a functional application platform on top of a system that was originally in documents, and then evolved, against really active opposition, into an application platform, and what's left is this Frankensteinian monster of so many different languages and frameworks having to work together to make something happen that you end up with this huge barrier to entry.
14:44 Michael Kennedy: Yeah, you do, and I feel like a lot of the web is like, well, we could probably do this better, but this is working everywhere, and that's more important than anything else.
14:53 Meredydd Luff: It's the ultimate path dependence, it's how we, you know, we got here this way, so that determines the way things are.
16:00 Michael Kennedy: What I think is really interesting there is like, you opened that statement with, we're improving sort of the accessibility of the web for seven-year-olds, which I think, as you pointed out, kind of downplays the significance of it, but when you get something that kids like that can start to use, what that really means is, non-programmers, you like, crack open a whole new aspect of the web for non-programmers, people who like, hey, I could totally program Excel, and I could like, set up these functions and maybe an if statement that'll make this Excel go, but I'm not a programmer, right, all of a sudden that person can become, like, they could build a web app, totally, with those skills.
16:35 Meredydd Luff: So this is something that's quite interesting, and it's a philosophical thing. A lot of people are interested in making tools for non-programmers, and actually, if you want tools for building a web app for non-programmers, there's actually quite a lot out there, and it's probably better-optimized for non-programmers than Anvil is. We actually take a different approach, we say, programming's not actually that hard, right? Increasingly, people teach themselves to code from websites, they get taught in school, it is something that people think is reasonable to teach middle-schoolers, and a whole bunch of people have learned, themselves. And I would argue, one of the big reasons we have this big mental block around, oh, programming's so hard, we have to make it accessible to non-programmers, is that when we think of programming, we think of the modern 10-layer web application stack, whereas actually, writing code is the best way to tell a computer how to do something, and Python is one of the best ways to get from I'm learning these concepts to cool, I can write some code that gets some stuff done, and that should be all you need. So we're kind of not aiming at non-programmers, we're aiming at, I can do a little bit of code and that should be enough.
17:52 Michael Kennedy: Yeah, I guess that's what I was thinking of as well, people who are like, I'll take a two-hour or five-hour course on Python and put a little effort into learning to code, but I'm not going to take a Computer Science four-year program, right, like that kind of group.
18:05 Meredydd Luff: And you shouldn't need it, right, it's when people talk about coding is digital literacy, this is what they're talking about. You shouldn't need to be actually that kind of expert before you can at least participate in this world. So, funny story, I'm actually taking this interview from borrowed offices of a customer who started with a single data scientist. And, you know, she had the stats knowledge, the marketing understanding, for an ad tech product she wanted to build. As far as she was concerned, she was not a developer, she couldn't build the web app that this so clearly need, and we needed, and was, you know, talking to agencies and so on. And then she found Anvil, and suddenly, actually, she realized she was a developer, it's just that what she could do was, you know, the Python she'd learned in the course of data science. And that's kind of moving the goalpost so that actually, a lot more people can and do know how to code, they just don't think of themselves as being able to build stuff, primarily because of the quality of today's tools, rather than because they actually don't know how to program. And the world's full of people like that, right, people who can write a little bit of code, as you said, like people who'd in previous decades, would have graduated from Excel to Visual Basic are now, they are the people who are writing Python analysis, you're doing a little bit of IPython notebooking, and we're trying to graduate them on to, you can take that and you can build something real that, in this case, handles terrifying amounts of advertising money.
19:39 Michael Kennedy: That's awesome. I do feel like, that people are expected to start software as, they're compared to places like, well this is how Instagram works with Django, and this is how they use microservices, and this is how they've written it to be testable and scalable, people look at that and they go, well, I can't do that, so I can't start, right? And then I feel like sometimes the most important thing is just to get started, I think Microsoft Access was like, one of the most-used small business sort of things for basically, forms over data, build something to work to make my dentistry run, or make my little business run, I'm not a programmer but I can like, drag this stuff around 'till I can get what I need, and that's kind of gone away, right?
20:24 Meredydd Luff: I'm not a programmer, but I can use Access to put this together. I'm not a programmer, but I can create a Visual Basic form such as this one thing that's application-specific. I'm not a programmer, but I've now created this line-of-business application that runs my business, and Microsoft had basically taken one look at this and realized that they had to guarantee support for applications written in Visual Basic 6, out to like the 2030s or something, because you give people this ability to like, slide into building code, and, you know, you give people this power and amazing things happen. Although, yeah, something we've been working on quite a lot with Anvil is to support that whole journey, because I don't want to pretend that good architecture and testing and all these things aren't important, and so a really big design goal for us with Anvil is to have something that you can get started with, without this huge barrier to entry, but is not going to stop you achieving mastery, and proper design, and scaling your app, and scaling your code.
21:30 Michael Kennedy: Right, you don't want to set a really low ceiling--
21:32 Meredydd Luff: Yes, exactly.
21:32 Michael Kennedy: You don't want to let, set a really high barrier to entry.
21:35 Meredydd Luff: Yeah, and we're convinced that this is possible, and we think that we are well on the way if not already there for most people's use cases. And I think that there's a bit of a sense that the floor is so high on the traditional tools, anything with a lower floor must have a much lower ceiling, and I just don't think that's true.
21:54 Michael Kennedy: I think that's awesome, so maybe now is a good time to start actually getting into the details of Anvil. People can check it out at anvil.works, right, you can go there, there's some actually really nice videos and some really intriguing things. But it basically starts as you say, create a new project, it happens on your web IDE, and it drops you basically, why don't you walk us through it, it doesn't just drop you into a blank page, necessarily, right?
22:18 Meredydd Luff: If you sign up at anvil.works, it is free to sign up, you end up in this IDE, and you can choose some of our sample apps and see what we've done before, but if you just create a new app, you start with selecting a visual theme, you create a new form, which is like a page in your site, you've then got this form, it's not fully blank, it's already got, like, the visual structure in there, and for this point, you have a toolbox of stuff on the right that you can just drag on, and visually create your page, and everything you drag on is a Python object, and if you sort of scroll down on the right you can have, you can see its properties, like for example if you put some text on there, so, you know, the font size of the text, there's the string, that's what the text actually is, there is, you know, which font, there's which color, all that kind of stuff. And you may or may not realize it at this point, but you're actually just editing attributes of a Python object that is going to be created at runtime.
23:19 Michael Kennedy: Hey everyone, this is Michael. Let me tell you about Datadog, they're sponsoring this episode. Performance and bottlenecks don't exist just in your application code. Modern applications are systems built upon systems, and Datadog lets you view the system as a whole. Let's say you have a Python web app running Flask, it's built upon MongoDB and hosted and scaled out on a set of Ubuntu servers running NGINX and uWSGI. Add Datadog and you can view and monitor and even get alerts across all of these systems. Datadog has a great getting started tutorial that takes just a few moments, and if you complete it, they'll send you a sweet Datadog T-shirt for free. Don't hesitate, visit talkpython.fm/datadog and see what you've been missing. That's talkpython.fm/datadog. Yeah, like you said, you may or may not know, but it has this very Visual Basic 6 designer feel, right, like you see it, you drag the stuff on there, you have a properties pane, you change the name, and then maybe if there's like, a button, you have these events, right, so there's various things that might have events, like if you double-click a button, it drops you into this code-behind type thing which is the Python, yeah, tell us about that back there.
23:19 Meredydd Luff: If you double-click a button, you're immediately dumped into a function that runs when that button gets clicked, and it's actually a method of a class that represents the whole page.
23:19 Michael Kennedy: All of this is Python, right?
23:19 Meredydd Luff: Absolutely, all Python, and all those things you dragged onto the page are, in fact, accessible as instance variables, so you have self.label1 or whatever, in that Python code, so it's a very natural thing, I mean, you said Visual Basic 6 and Delphi and, no two ways about it, Anvil owes a huge debt to those rapid application tools of the 1990s, because that was a really good way to work.
23:19 Michael Kennedy: They were hugely successful in building quick little applications, rather than hiring, like, a big dev team, but they ran on Windows 95, not on the internet.
23:19 Michael Kennedy: Right, much like writing a Pyramid or Flask web method, you just put a decorator on it and then it's accessible, but somehow there's a kind of a really nice link, both through the autocomplete IntelliSense side of things as well as the calling side, with the Python that is the code-behind for your page that actually runs in the browser and the Python that you write that actually lives in the server, right?
23:19 Meredydd Luff: Not knocking Flask, because Flask is excellent for building REST APIs, but having to squeeze all of your app's state through this choke point of, it must be representable as a REST API and as JSON is one of those things that makes the web unnecessarily difficult, and we explicitly avoid that. So when you mark a function as callable in a server module in Anvil, and then you call it from your client code, as well as being autocompleted right through and, you know, if you get an exception, there'll be a stack trace that shows the call going from the server to the client exactly where it blew up, which is very hard to do over something like REST. You can also pass in or return much richer data structures. I mean, you can pass and return anything that could be JSONed so your lists, dictionaries, strings, numbers, None, et cetera, but you can also do things like returning a row from a database, actually, I'll just explain what that is. So, a lot of apps will require some app-specific storage. And because it's just Python, if you already have a database you can just, you know, import PyMySQL, import Psycopg, connect to a database and go on, you know, many people do. But a lot of the time, you'll require something else, some app-specific storage, and Anvil has a built-in database we call the data tables for that.
23:19 Michael Kennedy: That's based on Postgres, right?
23:19 Meredydd Luff: Yes, and so it's backed by Postgres, and the tables themselves have a directly Pythonic API, so you don't actually write SQL to interact with them. You can if you've got a dedicated instance and you have your own database, so we don't mind you writing Towers of Hanoi in SQL, but this means that, that data table itself is a Python object, so if I create a data table to store all the visitors to my forum, I entitle it visitors, I've then got @tables.visitors accessible as an identifier in my code, just like those components were accessible as Python objects in my code. And I could return that, I could search for a set of rows and those rows will become live objects, you can access them like dictionaries to, you know, get the column values out, or you can update them by going row, bracket, quote, name, bracket is Meredydd.
23:19 Michael Kennedy: Okay, so I think there's some really interesting stuff happening here in that you have Python that looks like it's just regular Python, but actually runs in Skulpt in the browser, and you got Python that runs on the server, where can I access this database? Because it could be like, do I have to do it only on the server? Can I put it in my, like, button click? Not necessarily that I should, but could I?
23:19 Meredydd Luff: Absolutely. The thing I was leading up to, we were talking about the client server calls. The reason that we benefit from not squeezing everything to its choke point of a rest API is you can return a row from a database as a Python object from this server code and it will pop out of your as a return value in the client code. This means you can absolutely just directly access this in client code. In fact, for any data table, you can set its permissions. You can say, "actually this table should be completely readable from client code," and then in your form code, in the stuff that gets compiled to Java Script, you can go, it'll rake through all the rows of a table if it's readable from the client. Why shouldn't you be able to?
23:19 Michael Kennedy: Yeah, that's cool and you also get a visual designer type thing for the databases in the relationship and stuff, yeah.
23:19 Meredydd Luff: Yeah, you can say, "oh, here's a bunch of components, I'm repeating it for every element in the list." Again this is something we gain from everything being Python because that list is just a list of Python objects. You can give it a hint or the auto completer can work out what type the Python objects were and if, for example, it knows that you've given it a list of rows from such and such database, then when you're in the visual designer, you pull up the data bindings and you say, "I want the text on this label to be the name column from this row in the database." It can auto complete that for you because it's all using the same object model, it's all using the same language, and it knows, for example, what options are available for column lookup on that row. That, I think, is a really powerful thing for making this stuff a lot faster.
23:19 Michael Kennedy: I think, also, that that's a super important component for people who are getting started. They look at code and they're like, "I don't know what to type." If you type dot and here's a list, oh it says text. Okay cool, that must mean that's the value that's in the input box. I type customer dot name, that's the name of the customer. That's super helpful.
23:19 Meredydd Luff: There's a talk that I just gave at Python UK last Sunday that will be going up on our website soon, possibly by the time this podcast gets published, where I talk about the auto completer and how we build it, but actually I spend about half the talk talking about how auto complete is just terribly important. Seriously, listeners out there, if you spend your days in Vim or Emacs and you don't use an auto completer, I just recommend just try for a week. Install the jedi plugin for Vim or Emacs and experience what using auto complete is like for your code creation because you're faster, you know what you're doing, you can eliminate whole classes of bugs without ever even hitting the run button, and you get to stay in flow rather than going out to check the documentation every five seconds.
23:19 Michael Kennedy: I use PyCharm a lot and that's one of the main things that I find it's super helpful for.
23:19 Meredydd Luff: You nearly added a question at the end, but yes absolutely. I had to say, we spent a while thinking that we could do without auto completion and creative application development environment was fun to use and we were just wrong. You need it, you need that reassurance.
23:19 Michael Kennedy: What I think is really interesting is you actually spam my client server with auto complete and stuff, as well.
23:19 Meredydd Luff: Yeah, well this is honestly the biggest thing. One of the reasons that a lot of people, again, back in the 1990s your standard tools all had auto complete. Why people have kind of lost it now is because the tools don't know about what's on the other end of this REST call I'm making, what options will be available on the thing that comes back? Actually because it's all in the same system, it's all in the same language, the auto completer goes, "well, you know, I parsed the return statement "that will give me this value five seconds ago. I know exactly what it's returning and I'm going to auto complete the columns available on this database right."
23:19 Michael Kennedy: That's actually really, really nice.
23:19 Meredydd Luff: And we preserve that, and again, because we have this rich interface that's not squeezed over REST connection, we can do things. We have built in capability security model which is a fancy word for a thing that you most of the time just don't need to think about. If you've got one of these rich objects, and you return it from a server function, that was you giving the client access to that object. If there's a data table that the server can access and the client can't, but there's some sort of function that says, "well okay, I will give you the rows that belong to your user." As it returns those, you've now got read access to those rows that belong to your user. Now you can use them directly in client code.
23:19 Michael Kennedy: That's cool. How much ability do you have to structure things other than this page code behind thing in Python? Can I create classes and helper modules and stuff in Python that actually run in Sculpt?
23:19 Meredydd Luff: We've got server-side modules that run purely on a server and that's code that the client never sees and it's trustworthy. Then, we've also got utility modules which can be imported from client and server. Again, they're just Python modules you can import. You know, common code that will be between lots of your pages. We also got a system where you can take your whole app and kind of package that up as a Python package that other apps can depend on so you could say, "oh I've built a form for displaying progress bar," which is something that actually one of our users did on a forum recently, I built a progress bar form. I'm going to export this as a package that somebody else can import. You can also structure your code in that kind of reusable libraries way.
23:19 Michael Kennedy: Right, that's a really cool bit. I can't remember the last time in the web that I had like a sort of drag and drop reusable visual component that had both a visual aspect and a functional aspect, but you guys have that, right? You create these sub-element controls and drop them. I mean, it's been probably ten years since I worked in a web framework that had that.
23:19 Meredydd Luff: Yes, this is the thing, right? People say, "oh it's been ten years." What happened to progress? Why did we go backwards in those ten years? Yes, absolutely. When you import one of these, when you import a dependency that's got some of this stuff in, as well as being able to import the utility modules just like you'd import Python modules in a course they auto complete, blah blah blah, you've also got this server-side code you can bring in so you can make... You've brought in the client side code. You've brought in the server side behavior. You've brought in the visual design, all at once. You can create these reusable components that will abstract away a lot of the fuzzy blur. Had a few people currently have to re implement ten times.
23:19 Michael Kennedy: There could be, actually, almost a marketplace for people to create really powerful little controls. Tell me about it.
23:19 Meredydd Luff: That's definitely on our radar. I don't have anything to talk about yet, but you would not be the first person to ask for that.
23:19 Michael Kennedy: Yeah, I mean definitely in the Windows forums VB space, there were companies that just built these components that you could buy. I need a really cool graphing component or I need, I don't know, a login component or something. You just drop them in there.
23:19 Meredydd Luff: I feel like we've already kind of proved to have that model with the components and services we already provide within Anvil. As well as like, forms and modules and server modules, in Anvil app, you just hit a plus button next to the word services and you get the option of adding a bunch of pre-built interfaces to various external services so I mean the data tables is what I've already talked about, but you could, for example, add the Google API service and that's just a bunch of packaged up functions for dealing with files in Google Drive, accessing Google Sheets documents, authenticating people with their Google accounts.
23:19 Michael Kennedy: So maybe some company that runs on Excel could take all of their Excel documents, drop them into Google Docs, and then you could access them, convert them, access them as Google Sheets and use them as...
23:19 Meredydd Luff: Companies that do that have done that with Anvil.
23:19 Michael Kennedy: Okay.
23:19 Meredydd Luff: For exactly that reason. That's enabled by the fact that there is this drop in client and server and UI service that provides the things you'll need if you want to interact with the Google API. Similarly, we have something equivalent for Facebook and a couple of other things.
23:19 Michael Kennedy: You guys have Stripe? Like a super simple integration with Stripe?
23:19 Meredydd Luff: Oh yes, absolutely. If you want to get money, again, just Stripe integration just off the shelf.
23:19 Michael Kennedy: Yeah, that's awesome. You have email, you have user management. Those are both pretty impressive.
23:19 Meredydd Luff: Yes user management was one I really wanted to talk about. It's the perfect example. The user management service is implemented. I don't think there's any... No, there is no Java Script in the entire thing. It's all implemented using the standard API so the user serves the perfect example of something that you could've written yourself. In fact, of course people do, people did before we have it implemented. But it's just a data table with a column for email, a column for your Bcrypt Hash password.
23:19 Michael Kennedy: Oh you guys are using Bcrypt? That's awesome.
23:19 Meredydd Luff: Yeah, a server function that you pass in and use in a password and it looks up the user and says, "does it match?" If so, it stores something to the session and you know, it can use the Google API servers to say, "log this user in with Google, who's logged in."
23:19 Michael Kennedy: Right, you've got the OAuth 2 stuff.
23:19 Meredydd Luff: It's all something you could build, but you don't want to have to every time you want to build a web application.
23:19 Michael Kennedy: I'm working on a project right now, and I spent literally a full day just working on user management and this is just like the stuff I wrote six months ago, that's also going to be probably what I write six months from now. I feel like I'm always reinventing user management, so that's cool that that's kind of a plug and play thing.
23:19 Meredydd Luff: We tried to pick off some low hanging fruit, the things that people reinvent a lot and user management was pretty much top of that list. How many times have you written an email validation link processor thing?
23:19 Michael Kennedy: Too many, I want my life back.
23:19 Meredydd Luff: Exactly. Really, that's just the low hanging fruit because there's tons of stuff out there that people implement everyday and shouldn't have to. They should be able to create reusable libraries and we're really excited to have that kind of ease of use come to other domains in Anvil.
23:19 Michael Kennedy: So if you're Esri and you're like a company that creates ArcGIS, you want people to access your stuff. Maybe you could just drop in like, "here's how you plug into our web service and just get to your cloud GIS stuff." Right? Things like this? Yeah, that's beautiful. This portion of Talk Python to Me was brought to you by GoCD. GoCD is an on premise, open source, continuous delivery tool to help you get better visibility into and control of your team's deployments. With GoCD's comprehensive pipeline modeling, you can model complex workflows for multiple teams with ease and GoCD's value stream map let's you track changes from commit to deploy at a glance. Say goodbye to deployment panic and hello to consistent predictable deliveries. We all know that continuous integration is super important to the code quality of your applications. Choose the open source local CI server, GoCD. Learn more at talkpython.fm/GoCD. That's talkpython.fm/GoCD.
23:19 Meredydd Luff: I mean, already again, all of this other stuff you could do yourself. We have a bunch of really cool stuff for interacting with web services. I mean, if you're applying to a web service, you probably just want the requests library and you're off. But making that easy and simple and tab completing their whole API is a kind of thing that companies will want. It's something we've talked with companies before and we're really excited about bringing that kind of, "oh okay, I want to do telephony." Fine yeah, that's one important order to complete, exactly.
23:19 Michael Kennedy: Exactly, dropping the module or whatever, whoever it is, right? So this is all super interesting and these component type things are super cool. I guess the thing that probably was most impressive to me, that should like, okay... You have this thing that runs in your environment and is hosted... I want to talk about what that means. Before we talk about that, though, let's talk about Uplink, your final service.
23:19 Meredydd Luff: Oh yeah, so Uplink isn't really service. The Uplink is yet another example of a patent that we do right away throughout Anvil. Everything is built on abstractions, right? You know you can solve every problem in computer science with enough layer of abstraction, except the problem of too many layers.
23:19 Michael Kennedy: If it doesn't work, you add one more.
23:19 Michael Kennedy: Its really quite impressive. Maybe, give us the example that you have on your website with the Raspberry Pi because I think that really highlights how flexible this is.
23:19 Meredydd Luff: If you've got a Python code running somewhere in the world, and you Anvil server connect in, all those things you can do on a server module, you can do from your Python code wherever it is. There's an example tutorial video on our site of me doing this with a Raspberry Pi. So I take a Raspberry Pi. I write two lines of code, if you've ever used a Raspberry Pi, you can get something called a sense hat for it, which is an LED array, and it's really super simple to display some text scrolling across that LED array. It's very nice. I write a function that takes a piece of text, displays it on that. Because I've got the Uplink, all I do is I mark that Anvil dot server dot call upon, just like I would in a server module, and all the sudden I can call it from my web app.
23:19 Michael Kennedy: Just to be clear, this is a function running on the Raspberry Pi is being called by your Anvil app.
23:19 Meredydd Luff: Absolutely, a function running on the Raspberry Pi...
23:19 Michael Kennedy: That's crazy.
23:19 Meredydd Luff: Called from a web browser. Why shouldn't you be able to do that, right?
23:19 Michael Kennedy: That's beautiful.
23:19 Meredydd Luff: It's one call, we don't even use a server module. We call it straight from the browser.
23:19 Michael Kennedy: Yeah, that's really incredible and it even goes straight through firewalls and stuff. You don't even have to open a port because it's an outbound connection to establish that socket, right?
23:19 Meredydd Luff: So you can dry code, obviously the Raspberry Pi is a slightly educational, slightly facetious example but you know, we've got people driving big machine learning jobs that need to run on some delegated somewhere on some server somewhere with like 20 GPUs in. If they tried to do that inside our server modules, they'd hit their CPU quota pretty quickly and that would be the end of that, but because there's a root to plug your code into Anvil wherever it is, they just import the Uplink module, mark some control functions, call them all, and they can drive this big GPU rig from our hosted web stuff, just as easily as you could drive this Raspberry Pi on your desk to display something, just as easily as you could be driving a server function.
23:19 Michael Kennedy: This is a really cool feature. I really like it. Maybe this is a good time to talk about what it means to publish your code, where is it hosted, things like that. Tell me about where I can put this code. When I hit publish on your site, what does that mean?
23:19 Meredydd Luff: It's hosted on our infrastructure. We were talking earlier about this hugely deep web stack you need to understand all of and you need to understand all before you can even do a hello world on the web. We didn't talk about the layers lower on the stack, right? Also, if I want to deploy a hello world on the web, I'm going to need to understand AWS, put up another server, I'm going to have to understand how to administer a Linux computer usually. I'm going to have to publish that. I'm going to have to keep it up to date with security updates because I don't want to be the next Equifax being popped by a months old known vulnerability. That, too, is a barrier to entry. Two Factor Auth can help, but what Anvil does is we do the whole package. You write your code in Anvil, and it's hosted on our platform as well. You hit the publish button from the Anvil editor, even if you just signed up 30 seconds ago, you hit the publish button and you've got a URL that is your app on the web already. By default, obviously it's a private URL. It's like one of those Google Ad sharing links so you know Google Docs, if I send you this link, you can open it but if you don't have it you can't guess it. This is really powerful because it means that your turn around time is really low. You can get something onto the web very quickly and you're not worrying about all that ongoing maintenance burden of, "Oh, have I updated the server this week?"
23:19 Michael Kennedy: Yeah, I think that's really cool. Where is your infrastructure actually running? Is it AWS or somewhere else?
23:19 Meredydd Luff: We run at AWS.
23:19 Michael Kennedy: Okay.
23:19 Meredydd Luff: But I mean, we also run an on-site server.
23:19 Michael Kennedy: Sure, and you guys have an option to let it basically have an on-prem version, right?
23:19 Meredydd Luff: Absolutely, yeah, and of course that is just a few Docker containers, one Docker compose command and you've got a local version of Anvil running on your own machine.
23:19 Michael Kennedy: Yeah, that's cool. So all code runs in Docker on some Linux machine in AWS, basically, when we run it? The server code?
23:19 Meredydd Luff: Absolutely, yes.
23:19 Michael Kennedy: Okay.
23:19 Meredydd Luff: It's a little bit more tightly locked down than that, but yes.
23:19 Michael Kennedy: Yeah, sure. Do I get to pick the data center?
23:19 Meredydd Luff: Not at the moment, you'll want one of... There's a business dedicated hosting plan that says, essentially, we will put the VM wherever you like and you can have it to yourselves.
23:19 Michael Kennedy: Okay, yeah, that's pretty cool. I think what you guys have built really is quite interesting definitely for proof of concept apps, people who are don't really consider themselves web developers, but they feel like they can learn enough to solve their problem. This is awesome.
23:19 Meredydd Luff: Yeah, I think it's not just for those people. I mean, it is for those people, it's not just for those people. Something we do observe quite a bit, for example, is people going, "Oh well, I'll just use this to throw together a visual markup. Oh wait, I can add a little bit of behavior to it. Oh well, I can add a bit more behavior to it. Oh okay, well I've got this first version of my app. I might as well launch it as a prototype, right? Oh well, it's in production, it's scaling. Okay then." Yeah, exactly. The point is that we should be able to grow with you from that proof concept prototype up to deployed application and we have many people who have done that with us.
23:19 Michael Kennedy: That sounds definitely doable, looking at what you guys have built there. One thing I do want to sort of touch on with you, is what about the escape hatches? If I want to use a different editor that's external? If I want to do, like, unit testing? If I decide I actually want to take that code and put it somewhere else, is there a way to move out?
23:19 Meredydd Luff: I talked to you before about the server hosting environment and some of the really quite critical usability stuff we can do because we're not squeezing everything through this REST API checkpoint, a corollary of that is it kind of needs to be hosted on a platform to work because standardized platforms that let you just return a database row from the server of the client and have that give you implicit permissions kind of thin on the ground. If you want to run an Anvil app on premises on your own hardware, you're going to need on-premises license. If you do want to migrate out, obviously, you know we'd be sad to see you go but because of these escape hatches I talked about earlier, it's actually a fairly straightforward process, right? You could change your server modules into REST endpoints. Even if you're not moving out, right? You have, for example, a native isles app that wants to talk to the same back end out, the web app you've already created. Well, that's fine because although by default, we don't go through this REST API layer, you can decorate functions in your server modules to make them REST endpoints and say, this is very Flask-like, "well, this function is available at this URL." That let's you expand outside the Anvil-hosted web client and equally you can use different back ends from an Anvil-hosted client. Yeah, we don't think that you're stuck there but there are tools on our platform that you kind of need the whole thing to use.
23:19 Michael Kennedy: That makes sense. I do understand you guys are doing a lot of magic to wire this stuff together. Without the sort of orchestration behind the scenes, then I can see how that's not so easy to do.
23:19 Meredydd Luff: I have a love hate relationship with the word magic. Obviously, magic is a word that people use when they see something like Anvil working. "Oh my goodness, that's magic." It's the word I used earlier to describe my first experiences with something like Qbasic. "Oh my goodness, this is magic." There's good magic and bad magic, right? There is bad magic as in this is an Angular app and I'm not really sure what it does underneath to make this lever I pull over here connect to that thing over there that just wiggled. There's kind of good magic which is comprehensible. You can see exactly how it worked. It just saved you some work. For example, the client server calls in Anvil. They're an example of what I would call good magic. You can see, it's just a function call. You can see how it works. You see how you could implement it with REST endpoints, if you wanted to but you're glad you don't have to.
23:19 Michael Kennedy: I mostly meant that in the positive sense. The magic you guys are doing, but just that it's hard to replicate outside the environment, right? So you have a business model for this that is more or less like a freemium. Here's a basic version but if you want your own domain, then you've got to pay some amount. It seems pretty reasonable to me, but one of the things that came up is like what if I'm working on an open source project and I'd like to have some kind of open web app hosted with you guys that a bunch of people can come and contribute to, what would that look like? Is that possible?
23:19 Meredydd Luff: It totally would. I mean, even if you have an open source web app, somebody is usually paying for hosting somewhere, but the good news about... The good thing about Anvil is that all of your apps are in fact Git repositories so it is totally a thing you can do to pull your app's Git repository out of Anvil. You could stick it on Github, you could collaborate. You asked earlier about if I want to use my own editor, "I'm an Emacs fiend, I absolutely must edit this module in Emacs." That's fine, git checkout, pit pull. Have fun. git push when you're done. Also, that means you're open to collaboration tools like Github so you could totally run an open source app that's based on Anvil. You could have multiple instances that people pay to host or host on our free version.
23:19 Michael Kennedy: That sounds good. The Git integration is pretty nice. I guess you can create multiple upstreams in Git and make one of those be your Github repo, right?
23:19 Meredydd Luff: That's something that we do all the time. You could even do, like, a continuous integration continuous deployment because everything you can do in the Anvil editor is reflected in this Git repository, obviously, so you could do things like, "Oh well, this is my staging deployment and it's got access to some staging data and then I pull that git commit and I push it up to somewhere else and that's my deployed app." Indeed, we actually have a miniature version of this within Anvil itself so you can choose a version of your app if you go into view history, you can see the full Git history of your app. You can select a version that's the published version. If somebody comes to your shared URL, externally, that's what they get. You can carry on hacking and you've got this dev versus production thing already built in, but the thing about Git is of course, you can extend that.
23:19 Michael Kennedy: I think that's really nice. So basically you have this UI that says, "Here's your Git history." That means you went and pressed saved, like save a state here. But you can actually mark like, "This one right here right now is production." And continue working on it and then push production on down the line once you have it sort of tested also, kind of in production and I think that's pretty powerful.
23:19 Meredydd Luff: Yeah, actually if you use that Git interface, you'll discover that's just a branch in Git. This is kind of what I mean about good magic versus bad magic. That's kind of magical, but how we do it is very much open and you can play with it too, if you want to do something more advanced. Yeah, the normal user experience is like one click, okay that's production now, done.
23:19 Michael Kennedy: Alright, we don't have much time left. So much cool stuff to talk about. I guess one last question is like, you have your data tables as a service there. How do I back up my data?
23:19 Meredydd Luff: So if you're using the data tables within Anvil, first thing to say is we do take regular and properly encrypted and stored backups of those so you should not have to worry about the data loss. If you want a copy on your own system, then there's a ton of ways to get it out. I mean, every data table in the editor, there's a download CSV button which is a very straightforward way of getting stuff out but earlier, we talked about the Uplink and of course, from the Uplink, you can do anything you can do in those server modules. It's not just a matter of saying, "Oh, well here's a function that displays some text on my Raspberry Pi and it's called from the web." That script could've also, you know, reached in to say, "Oh I would like to iterate over all the rows in this table" and that's totally legit from Uplink.
23:19 Michael Kennedy: Sync that with some local DB, something like that.
23:19 Meredydd Luff: Yes, so you know, if you had your own representation of data you wanted to synchronize with or import, export, that is really straightforward to do. Again, you know, if what you actually want is something in Amazon Redshift, if that's where you want to sync your data to and that's where you want all your data to live, you could just access it there from Anvil as well because we've got all those Python modules ready for you to use. There's nothing forcing you to use the data table service. It's just something that makes your life more convenient in a lot of cases.
23:19 Michael Kennedy: Right, knowing the Docker containers are running AWIs, you can basically crack out and get to most of the other AWS stuff, right?
23:19 Meredydd Luff: That would be accessible, that would be possible even if we weren't running in AWS, right?
23:19 Michael Kennedy: Right, lower latency inside the AWS, yeah.
23:19 Meredydd Luff: Does run quite well.
23:19 Michael Kennedy: Nice, alright. Well, I think there's a bunch of other stuff we could dig into. I think we should probably leave it here, just for time sake so let me ask you the two questions. We talked a lot about editors, actually so let's start with your favorite editor.
23:19 Meredydd Luff: My favorite because it's my baby and also I spend a lot of time in it, is of course the Anvil Editor. Its got a full autocomplete. It can do full stack development, and its plugged into full version control on the back end. So I spend a lot of my time, obviously, working in that. For building the Anvil backend itself, I actually use IntelliJ, and I really love the Jetbrains tools. IntelliJ PyCharm are fantastic. The, again, the autocompletion is top notch.
23:19 Michael Kennedy: Cool. Alright. That sounds awesome. Yeah, the editor you built is really really nice. I definitely like it.
23:19 Meredydd Luff: Yeah, check out our PyCon UK Lightning Talk.
23:19 Michael Kennedy: Yeah, send me the link. I'll put it in the show notes. And notable PyPI package?
23:19 Meredydd Luff: The selfish answer here is the Anvil uplink because it is on PyPI, and it is remarkably cool in its ability to plug your own code into the Cloud. If I weren't counting that, then it's got to be Pandas, the things that you can do in like three lines of code. I recall using janky libraries to pull that data out of Excel files and process it, or worse, using R. Yeah, incredible, so yeah, Pandas wins.
23:19 Michael Kennedy: Cool, yeah that's definitely, definitely a big one. Alright, final call to action: people want to check out Anvil, what do they do?
23:19 Meredydd Luff: Oh, they go to Anvil.works and they sign up for a free account. We've got tons of walk through videos and tutorials. There's a user forum that's really quite active and friendly and you can sign up for an account for free and start building. And I would really encourage you to do so, and I would look forward to seeing you there.
23:19 Michael Kennedy: Awesome. Well, congrats on building this. It's definitely pushing Python in a direction that I haven't really seen it pushed successfully in quite awhile So, it's awesome. Nice work.
23:19 Meredydd Luff: Thank you very much. It's been great talking to you.
23:19 Michael Kennedy: This has been another episode of Talk Python To Me. Today's guest was Meredydd Luff. And this episode has been brought to you by Datadog and Thought Works. Datadog gives you visibility into the whole system running your code. Visit talkpython.fm/datadog and see what you've been missing. They'll even throw in a free t-shirt for doing the tutorial. GoCD is the on-premise open-source continuous delivery server. Want to improve your deployment workflow, but keep your code and builds in-house? Check out GoCD at talkpython.fm/gocd and take control over your process. Are you or a colleague trying to learn Python? Have you tried books and videos that just left you bored by covering topics point by point? Well, check out my online course Python Jumpstart by Building 10 Apps at talkpython.fm/course to experience a more engaging way to learn Python and if you're looking for something a little more advanced try my Write Pythonic Code course at talkpython.fm/pythonic. Be sure to subscribe to the show. Open your favorite podcatcher and search for Python. We should be right at the top. You can also find iTunes feed at /iTunes, Google Play feed at /play and direct rss feed at/rss on talkpython.fm. This is your host Michael Kennedy. Thanks so much for listening. I really appreciate it. Now get out there and write some Python code.