#260: From basic script to interactive data sci app with Streamlit Transcript

00:00 If you work on the data science or data visualization side of Python, you may have come to it from a scripting side of things, writing just a little Python, using its magical libraries with little structure or formalism to build a powerful analysis tool that runs in the terminal or maybe even in a Jupyter notebook. What if you could take that same code, sprinkle in a bit of a simple API and turn it into a fast and dynamic single page web application that allows your users to dive into the visualizations right there in the app? Well, that's basically what the folks over at streamlet created. And we're going to dive into it with one of the creators, Adrian true. This is taught by me, Episode 260, recorded April 9 2020.

00:57 Welcome to talk Python to me, a weekly podcast on Python, the language, the libraries, the ecosystem, and the personalities. This is your host, Michael Kennedy, follow me on Twitter, where I'm at m Kennedy. Keep up with the show and listen to past episodes at talk python.fm and follow the show on Twitter via at talk Python. This episode is sponsored by linode and Rubin learners Ace Python interviews course, please check out what they're both offering during their segments. It really helps support the show, AJ and welcome to talk Python to me.

01:26 Hey, Mike. So excited to be here. Thanks a lot for inviting me on the show.

01:29 It's super heavy here on the show. A ton of people reached out to me when streamlet was initially launched, saying there's this new thing. It has changed my world, you have to talk to the people behind it. And so now I'm excited to talk to the people behind it.

01:44 Yeah, totally. It's awesome to hear that. And, you know, definitely, for someone who's spent a lot of time, you know, in a room programming alone, it's really cool to have made something and hear people say this is awesome. This, this, you know, really changed my life and my work. So that's great to hear. Yeah, it's a super cool product.

02:06 And I think it's gonna be fun to talk about, it really enables data scientists to get certain types of computation out there in a way that people can interact with. So I'm very excited to talk about it, where it came from some of the technology inside of it and whatnot. But let's get started with your story. How did you get into programming in Python?

02:25 My story is that it's never too late to become a programmer. I actually went to college to study international relations. I was came from a bilingual family and I you know, I figured I should do politics or something like that. Right,

02:37 leverage your bilingualism. Alright, what languages do you speak?

02:41 Well, okay, so my dad's French, so I grew up speaking French, in English. And we were, and then, you know, being, you know, growing up in the US, I'm surprising I took Spanish. And then actually, when I got to college, I took Chinese. So, you know, I didn't come from a family that really respected like programming and that kind of thing. And so it wasn't part of my world. And I feel kind of envious of kids who grew up, you know, went to science, high schools and stuff, because in retrospect, that's really who I was. But it wasn't until I got to college, and about a year and a half into studying international relations. I was like, This is not for me. Actually, I always thought college might not be swimming. And then this professor of economics remembered me because I had blue hair. And he was like, you know, he said, Adrian, how you doing? And I said, Well, you know, not so I'm not even sure if I should be here. And he said, you what you need to do it. He said, You're misallocated, which is a very economically very economic way to describe it. Yeah, exactly. And so he was like, what you need to do is you need to study math, and science. And I was, you know, anyway, I thought that that was ridiculous. Literally, I took him up on his offer. And it calculon my sophomore year of college, took intro to programming. I was like, blown away. It was like, these are my people. This is my work. And I'm just so grateful. I found it. Oh, my God. I mean, wow, what a fun job to have, and what a fun world to be in. And also, this is back in the late 90s. It wasn't 100% as obvious as it is today that this is like, generally, generationally changing for all of culture. And obviously it is everyone sees that now. And so I feel really lucky to have gotten it.

04:24 Yeah. How lucky that you had that interaction. And you found your place because I think a lot of people, they don't find their place in the world, they find a place that absolutely then they end up okay. They don't wake up going, Wow, I get to do this.

04:41 Yeah. And I really think you know, being able to figuring out, taking time to figure out like what you're really would enjoy doing and trying a lot of different things. I just can't think of anything more valuable. I really actually think the whole world is a little bit Miss allocated to to certain extent, and if everyone can sort of play musical chairs and you know, get the right job, or a better job, we can just unleash like massive new economic potential. So, you know, that's true for my life. It's also been part of actually a theme of my professional life as well. So I can talk about that. But anyway, yeah, figuring out what you love to do is I think, so valuable. And I'm so grateful for that. Professor.

05:23 That's amazing. I do think there's a little bit. I don't know how to solve this. But I do think there's a little bit of a mismatch, where people tell you, you have to find something that you're passionate about, or you're not going to be happy in your career, I think that you find something that connects with you. And you didn't even know that it was going to connect with you. And then you polish that skill, and then you become very passionate and excited about it. So when you I think when young people will hear advice, like you have to do what you're passionate about, like when you're young, and you haven't had that exposure. Like imagine you hadn't taken that math class, like, you weren't passionate about math, it's not like you're like, well get out of my way, you know, languages, I'm going to math, right. There's this sort of chicken and egg thing. And I think it's really hard to unlock that potential for, for a lot of people in general. Mm hmm. I don't know. I don't know what to give people advice about that. But it's, it's I think it's interesting that it's important to do what you're passionate about, but like, you got to somehow explore to get that figured out.

06:22 I agree. And I also think there may be different. There are different kinds of people, of course, and I don't think necessarily being passionate about what you do is as important to other people, as it turns out, it is to me, I don't want to over extrapolate either. But I do feel really grateful for you know, have been given that advice. And it actually come up in a couple other ways in my life, I created a game called Foldit that allowed hundreds of thousands of people to solve protein folding problems. When I was doing my PhD. I think I've heard of that. Yeah, I'm pretty

06:55 sure I did hear that. Yeah. So it was the gamification of trying to solve protein folding problem. It wasn't like SETI at home where it was like just grid computing, right? It was leveraging human creativity, not distributed computing.

07:10 Right. Exactly. And that sort of that theme. One of the coolest things about that project was that a lot of people found out, hey, this is something that I love doing. And they were able to contribute to society and like a new way that they'd never experienced before. And so I think for a lot of people, it sort of unlocked a little bit, this new skill. Yeah. And then getting coming to streamlet. You know, what we're doing is building software that lets data scientists and machine learning engineers, like productize, their work without the need anyone else to help them so they can build apps that others can use. And again, I feel like for our users, we're giving them like a new form of impact in the world and in their companies. And I think that's where the passion comes from. So in a funny way, it does seem like a theme most of my life personally, and also, in the products that I've been creating.

08:05 Yeah, absolutely. So you went from not sure you should be a college to studying some math to being a professor at Carnegie Mellon? Oh, that's a big, big U turn.

08:13 Yeah, yeah. But you know, actually, in a funny way, the I don't think the quest ended there. Because when I was a professor at Carnegie Mellon, there were some things that I really loved about it. But I felt that I wasn't directly affecting the world the way I do now at streamlet. So, and essentially, we were doing super fascinating, very mathematical, big data, stuff. And it was from an intellectual standpoint, I mean, having to the power of heaven. But I was missing that, that other aspect of things, which is really, you know, having an audience beyond a couple hundred experts in the world in some way. Both are totally valid and extremely important forms of impact each in their own way. So I am a huge believer in academia, and it's important in in society, I personally wanted a little bit more direct connection to the you know, the quote, unquote, the user. And then that has taken me through a variety of different jobs at Google X, working on self driving cars, and zooks. And then finally, starting streamlet. And for the first time, since I created, folded, I'm hearing people say, wow, this is really changing my life. And that's so exciting to have created a product that has that property. That's amazing. You've done it twice, right.

09:36 And with this, you've made something that's made a big impact for a lot of people.

09:40 It is really gratifying. And I think, you know, the gratifying part is to first of all, it gets to, you know, build cool things and work on cool ideas, but also just have other people feel like their jobs are more interesting and more fun, and they're having more impact, because of some piece of technology that we've pulled out and so I think that's just great. I think that's just so exciting. I agree. So today, day to day, you're working at streamlet with the algo. And Amanda, your co founders, and I'm guessing some other folks on the team. What kind of work are you doing there? Okay, so we're about nine people. And we're about 18, including contractors. And we are hiring a lot all across the United States. And we are basically building two products, one of which is streaming it. And the other of which is streaming for teams, which allows people to stream it allows people to build data apps really quickly in Python 2.4 teams allows them to deploy it really easily, and others can use it. And then my day is pretty much Well, my dream day is pretty much I'm chatting with the engineers and working on design docs and thinking about cool problems. Somehow that's like very rarely my actual day. I don't know how it happens. But my actual day is often just like meetings, meetings, meetings, meetings, meetings, talking with investors, I don't know, thinking about payroll stuff. And I'm not sure how, despite my effort over the years to make that not be the case, that is still the case. But that is still the case.

11:16 But here you find yourself but you know, that's kind of what it takes to get some of the stuff going. Right. So

11:22 yeah, totally, totally. Yeah. And I think, you know, ultimately, what has made streamlet great, has not actually been my ideas, per se, but really, actually the ideas of the community, and also of the engineers working on it. And so I try to remember that, even though I have a dream job of working on every aspect of the product, probably everyone else's benefit, I should just get out of the way and let them do the magic.

11:50 Yeah, that's sometimes it's how it is, right? It's just let the people do the work, even though it would be very fun. Now, I wouldn't get into some of the like origins and whatnot. But I also just want to set the stage as people are listening, because this was something I was not entirely clear about as well. Yeah. So you spoke about streamlet, the tool for building interactive UI around data science, and you spoke about streaming left for teams, talking about hiring people. So obviously, this is a business venture. Yeah, the status, which is great. I'm super supportive of you guys. That's awesome. But I just want people to realize that streamlet the tooling in the library is an open source thing that you can use. And the team's thing is like a cloud computing service that is part of the business. Is that the my understanding, right?

12:37 Yeah, that's exactly right. And speaking of the origin story, it wasn't actually meant to be a business, it was really a side project. And it was going to just be an open source project. But what happened was a bunch of people at really, like big companies, like Stitch Fix, started using it. And then we realized, Hey, you know, there is actually a business model here. And so the picture sort of, in a sense, it all kind of snowballed from what was really first just a personal project me at home, coding all day long, and my mom saying, hey, you need to figure out a way to make money off this thing. We're gonna work on it so much, and then it sort of has evolved into a business. And so yeah, there's actually a very simple breakdown in terms of the open source. And the business part, which is open source is lets you build data apps super quickly and easily. It's free, always will be free Apache two license, it's a real open source project. Everything's up on GitHub, the community contribute a lot, we build it a lot and maintain it. So that's the open source part of the project. The closed source part of the project, lets you take your stream with apps and deploy them really easily on our cloud, you're certainly welcome to just deploy it for free, however you'd want. And there's tons of guides online about how to do that. And we totally support that aspect of the community, too, there will always be a way to create and deploy streamlet apps for free using only open source tools. But we also recognize that a lot of our users and especially corporate users are asking for a simpler solution to that side of things, the deployment, not the

14:14 app building. But I think it makes a lot of sense, because the goal of the tool is to make converting like data science, ideas and libraries, and problem solving into an interactive web app. Really easy. And so the last thing you probably want to do is like Well, that was cool. Now you need to figure out nginx you need to figure out SSL you need to figure exactly you need to figure out Linux maintenance, etc, etc. Exactly. I think it's a reasonable thing to do for sure.

14:40 Yeah, I think the experience that that our users have is they you know, download too much. So pip install streamer, follow the tutorial. Literally within 30 seconds. They have an app that they're playing around with within five minutes they've created their own and there's a little button or something and and then I think that what we hope for is delight. And, but then it comes to, I want to share this with others. And you go into the new search around on Google, and it's literally starts to become, you know, create a GCP instance, copy your SSH keys into it, I'll tell you, that's a nerve wracking experience to actually try to put something on the web and have it secure and everything. So we want to bring that delight to the deploy as well. And that's kind of the picture.

15:33 This portion of talk Python to me is brought to you by linode. Whether you're working on a personal project or managing your enterprises infrastructure, linode has the pricing support and scale that you need to take your project to the next level, with 11 data centers worldwide, including their newest data center in Sydney, Australia, enterprise grade hardware, s3 compatible storage, and the next generation network linode delivers the performance that you expect, at a price that you don't get started on the node today with a $20 credit and you get access to native SSD storage, a 40 gigabit network industry leading processors, their revamped Cloud Manager cloud not linode.com root access to your server along with their newest API and a Python COI just visit talkpython.fm/ linode when creating a new linode account, and you'll automatically get $20 credit for your next project. Oh, and one last thing they're hiring go to lynda.com slash careers to find out more, let them know that we sent you. I haven't interacted with the deployment side of things. But I can tell you that the web app side it's got a little bit of magic. So let's get into that. But let's start at maybe the predecessor. Right? A lot of people are coming into Python these days. This is my assessment of looking out at the world as they're coming into Python for a little bit of computational scripting, I hate to call it scripting. Because when people say oh, Python is a scripting language, it's like, Yeah, but you also, it's a programming language that supports scripting, right? Like it's totally there are a lot of people who they don't care about computer science, they definitely don't think of themselves as programmers, they think of themselves as economists, biologists, whatever, but they're like, I know I got it, this data is too much for me to do on my own, by Excel or whatever. So I'm gonna throw it into this, I've learned that if I use Jupyter, I type these five lines using this library. No functions inside, you know, no, unit test method. I just like I do these, these five or 10 lines in order that I have this really cool power. And then they get sucked into Python, right? Like, yeah, they could do a little more in a little more. And like, eventually, they look back and like, wait, I'm programming. How did I become a programmer? What? Yeah,

17:48 yeah. Which is great. But I think, like, where you guys are aiming? Is that audience a little bit, right? The the people who are like, I can put this together it kind of a top to bottom blow of computation and generate results. But I need more than that. But I'm not a programmer, would you say? That's fair?

18:06 Yeah, that's exactly right. So we really embrace Python, as a scripting language, that's the core of streamline, and we embrace the ethos of lead, let's write code, from top to bottom, that does something simple, that's easy to understand and read. And that really gives you a huge amount of power for, you know, a huge amount of bang for your buck. I mean, that's what Python is, right? You know, how do I, how do I download though, some information off the web requests package, boom, boom, boom, all of a sudden, that's happening. And I think actually covers myself very much included are like, shocked when you go to another programming language, and you're like, Oh, I Yeah, totally. You know, you just you do it, you know, they do all these things. And, and then you're like, Oh, my God, this is impossibly difficult. How are these people doing? You know, so there is still I mean, it's an old, it's getting to be an old language. And yet, there's this magic to it. And there's this bang for the buck that it's just unparalleled, and, and streamlet is grounded in that observation. And in that ethos, which is let's write script, and let's extend Python in a new direction, which is really being able to write web apps, especially data web apps, that capture something about what you've done as a data scientist or machine learning engineer and allows lots of people to play with it, and use it and understand it and actually become part of business processes. And yeah, and we've seen, you know, become part of the fabric of your company or of your group or of the world. People are creating lots of COVID apps, for example, both to teach other people about Coronavirus, but also in some cases, University of Penn hospitals released a stream that up to allow other hospitals to predict how many beds they need.

19:57 Yeah, that's very useful, right? That's taking The models and apply them for, you're gonna need to get five more or something like that right?

20:05 very concrete. Exactly, exactly very concrete and very much sort of the next step in data science beyond just you know, I've created a graph once for to paste into a, you know, spreadsheet or into a presentation. It's like, well, what happens when I need to do this repeatedly? And when a lot of people need to interact with it and play with it? I use it, I should say, and then and then how can we do that without embracing this scripting ethos, we want to make app creation as easy as writing scripts in Python. And that's really the core sort of philosophy that motivates streaming. And I think, deep, capturing that magic is what has caused so many people to get excited by it.

20:45 Yeah, I think you're right. And I think it's an interesting blend between notebooks, and more proper programming, but just a little bit. So what I'm gonna talk about some of the examples and whatnot, you have that are really cool. But they are Python files, and they do have functions in them. But they're not, not like super advanced, right? So it feels like they're almost kind of organized like someone might organize a notebook for scripting. But it's a little bit more formal than a tiny bit more formal than that. But what you gain on the other side is like tons of interactivity, which is pretty cool. So when you look at the stream, let you guys talk about it, you say basically, the The goal is to bring three things together, the Python scripting, which we talked about, the next one is interaction to basically make this an interactive thing for users, more or less the product ization of that stuff, right, so users can play with it, not just numbers. What's your final final bit is to deployment that we've kind of touched on. But we haven't talked very much about the interactive bit, which I think is also some of the magic because yeah, honestly, I look at this. And I don't necessarily fully understand it, because I've written a lot of GUI applications. I've written really simulators and a lot. And I, you know, I wasn't super great at it. But I know what the code looks like, I've done tons of GUI apps with callbacks and whatnot. Yeah, you seem to have been able to achieve those types of things without loops. And without explicit, oh, the button event wires up to this callback, and yeah, all that async kind of stuff that's so common around UI. So maybe we should talk about the interaction. But

22:26 that's exactly right. The ethos of scripting is basically, I'm going to write a script that, you know, load some data transforms, it does some other stuff. And then it's done. The ethos of app development is totally different. It's, I'm going to declare a bunch of widgets on the screen, I'm going to wire them up with callbacks, I'm going to have state that changes over time, I'm going to be interacting with the web and have promises and stuff. And it is a truly different form of programming, fundamentally, and typically requires a lot of time to do even fairly small things. I mean, that's, you know, it's not easy to build an app, actually,

23:09 at a minimum, it takes a different way of thinking about programming that is exactly right, these 10 lines, right,

23:14 that's exactly right. It's a different way of thinking about programming. And we really tried to ask ourselves, how could we make writing an app, as simple as writing a script. And so we give the streamlet programmer, a very, very simple to understand paradigm, which is basically like sort of a cross between just a fancy print statement. So you know, in Python, you can always just say, print, blah, blah, blah. And then that gives you an ability to see what's going on and understand, right?

23:52 If you want to level it up, you put some color Rama in there, and like maybe, yeah,

23:56 exactly. Yeah, exactly. Or, you know, if you're, if you're Jupyter, you can say like, let's look at the graph, or, you know, you can start doing some visual output, we basically give you that as a basic primitive, it's called St streamlet dot, right? r i t. And then you can just plop anything you want in there, and it pops up on your web browser in order. And then we say, hey, and if you want any interaction, at any point, you can just say S T dot, and then just give it we've got a whole bunch of widgets, you know, sliders, text boxes, and whatnot. And that pops up on this screen. And then we wire it all together so that if the user types or changes anything on the web browser that gets reflected back in your code instantly without having to do any work. And so the model is just so simple to understand. And therefore, so productive, and we want to bring that Python, simplicity and that Python magic to the app building process, and make it so that others can just build actually easily possible.

24:59 Really the best way to give people a sense of this is maybe to walk through an example, that you guys have a couple example. bits of code out there. Mm hmm. And right at the beginning, it was streamlet. Websites from like.io can go down a little bit, there's like a get started, just pip install Streamlight and a few other things. And then there's a, an example that you can just run that comes with it. And this has to do with self driving cars, which you and your co founders have a lot of background in self driving. vehicle, right? Totally. So the idea is, there's a lot of image processing that goes on around identifying people, other cars, bikers, certain tolerances of whether or not that is a biker, or not, and so on. And so you can basically get this demo and explore those, I like to have up to five but no more than six pedestrians on a screen. And then I want to test like parameters around it and stuff, right? Yeah,

25:58 that's exactly right. So this is a problem that comes up in the self driving car industry. Really, it comes up in all machine learning and all data science. In my experience, I've seen this again and again, across groups at Google, self driving cars, companies across the country, which are now using streamlet, and telling us what their challenges are. And so it's really a perfect example of a simple, but a simple streamlined up, which is I have a whole bunch of data domain specific, and I need a way to be able to browse through it and understand what's going on. And I might be need to look through it, I might need to maybe run some models on it. And we're blend a couple different datasets together. And streamlet makes those kinds of tasks really easy by just taking the basic elements of Python and adding a tiny little bit on top to make it an app. So in this case, the challenge was something that we saw when building self driving cars is that you drive your cars all around the city, and you're collecting like all this data. And at a certain point, even understanding all the data that you have becomes really complicated. And lots of people need to be able to see it not just the engineers, but technical program managers and executives, and everyone needs to be able to see what's going on. And so you eventually build an app to allow many different stakeholders to see it understand this data. And so we built this app to illustrate how that can be done in streaming that uses the Udacity self driving car data set as an underlying data set, lets you visualize where there are restaurants and cars and even lets you run a neural net in real time. And the entire app is built in 300 lines of streaming. And from top to bottom.

27:43 There's also like 50 lines of comments, I would say,

27:45 yeah, yeah, it's 300 lines. And it's not even like we're not even trying to be super, you know, we're not playing like code golf or anything. Like, these lines include like a huge, like Apache two license and a lot of comments. And yeah, so in fact, I think there's only like 20, some odd function calls streamlet function calls in the entire program. So everything else, basically, this very, very rich app is created in 20, function calls and everything else are listed, call it 25. Everything else is just pure data. So that's kind of a glimpse, hopefully, for the listeners of just like this fact, which is that like, you can just do a lot with a little, which is just a great thing to have.

28:31 Yeah, so let me do two things. Let me describe a little bit of what this app looks like when it's running. So people get a sense. So when you run the app, you get a webpage that is running on its own little local streamlet server. And on the left, you get a sidebar that lets you basically do the user interaction, right? Like I can say, what type of object Am I looking for? And then how many of those objects and depending on what you get, you can have, like different types of confidence and whatnot. So there's, there's this sort of cool interactive slider thing on the left, and then on the right in the main pane, there's kind of output from that, like pictures with the selected characters in the picture from the neural net, or graphs or whatever, right? That's exactly right. Yeah. And so you know, it's all the basic elements of streaming that are there. So you get some user input, right? And so if I change like the drop down, or I change the slider, as a user talking about user input, right, then that automatically is reflected back in the changing views of the graphs or the detection of the picture. So that's a super cool type of dynamic interactive web app. It's not posting back the page. It's, it's like a JavaScript spa type of application,

29:42 you would expect That's exactly right. That's exactly right. Yeah. And you know, it actually this is just, this is trying to show what's possible in 300 lines of code with a 20 some odd extremely cause but some of our users, especially corporate users, build really intricate, not just single Page applications, but you know, multi page applications with different levels of authentication for different users, and, you know, interactions with databases, and they go both ways. So it's really the sky's the limit, it is actually possible to build, you know, production quality apps used by many people fully in streamlet. And in fact, to do so, really, as a data scientist or as a machine learning engineer, without having to learn and then wield a pretty complicated, and you know, quote, unquote, pro tech stack, like, yeah, you know, flask, and then CSS, JavaScript, HTML, react, whatever it is, that we love those tools. For that matter. We wrote streamlet in those tools, so we really love them. But we recognize that for a lot of people, even if they do know them, it's a big lift to start building an app in that way. And it's really fun to not have to. And you deserve a lot.

31:04 Yeah, that's cool. The other thing, that's interesting, that's what that's notable, I guess, is a lot of these types of tooling that try to make things more accessible, or more productive. They say, Okay, well, what you're going to do is you're going to work with our little widgets and our little libraries. And the reason is simple, because you don't need to learn about database. So you use our data library, and it magically does data magic. Right, right, right. Yeah. But right at the top you have of this example of import Altair, import pandas import NumPy, import open CV, right? I guess what I'm pointing out is like, these are the tools that data scientists are already using. They already know how to use them. They're already like, very effective and polish. So it's not like, well, you can give up your Altair or your book and your pandas, and you could use our style. Right. That's, that's right, and seem to be the story you're telling.

31:58 Yeah, that's right. I think that there's a really interesting balance between simplicity, power, and, you know, accessibility in regards to using other existing open source frameworks. And one of the sort of strategies that some people have try is, we will be the platform, we will define everything, and you will live in our world, walled garden. And that's how we'll make things quote, unquote, simple. That's not really the ethos of open source, really, the ethos of open source is more like, we're going to do one thing really, really well. We're going to be modest in a sense, in what we provide, but we're going to provide a great version of it that then you can mix and match with other open source tools. And that's the ethos of streamlet. We do one thing, which is let you build apps, we try to do it, the best of the world, of course. And then we get out of the way. And we say use everything that you know about graphing existing graphing tools, just Python, all of the you know, snowflake, pandas NumPy, scikit, learn the entire Python ecosystem is just directly available to you.

33:08 Yeah, that's really cool. It also means you're not like super tied to stream mode, if you decide, actually no, or whatever, right? Like totally Exactly. You could say we could do it a notebook, or we could try and streamline it and or not, right, like it's, I really appreciate about that about what you've built.

33:25 This portion of talk Python, to me is sponsored by me, Ruben Lerner, and my free video course Ace Python interviews. You know, one of the most stressful parts of a job interview is when they ask you to explain a concept, or just solve a problem with Python is Python interviews contains six hours of screencasts, solving 50, beginner, intermediate and advanced Python problems. You can even download the Jupyter notebooks used in the screencasts. Give yourself an edge when you Next apply for a job. Check out Ace Python interviews for free at talkpython.fm/ace.

34:00 I still don't fully understand that how the magic works. So I want to talk people through it. And then maybe you can tell me like, oh, here is where the magic is. Yeah, for sure. Alright, so let me talk through some of the code. I just pulled out a few little examples of things that are interesting as you go through that self driving demo. So let me run by pass you. Yeah. So first of all, like I already talked about this, you have functions, you've got a main function at the top. And then you've got other functions like download the files, like go and get those self driving car model, data's and the images and whatnot. And you've got other functions for like showing what are some of them, or like running one to like run the app one to select different parts of the UI one, yeah, and whatnot. So you have this broken down like a well structured script, I guess you would say like it's all self contained to this file. It's not like a notebook Exactly. But it is broken to function. So I appreciate that. I also like that fact that you put the main method at the top, and then the functions below I always had in mind, because here's the summary of what's happening, here's the right always seems to be reversed a lot of times that drives me crazy. Yeah, but so first thing is like, you want to have some message on the screen that is the UI of the web app. And the way you do that is load a markdown file and parse it and render it using like one line like St streamlet, dot markdown, boom, give it a file name. And then that's part like a portion of the UI. Is that right?

35:30 Yeah, that's right. There's a couple of cool things to note about that. First of all, we do let you write well, structured code, quote, unquote, well structured code, if you want, we using named functions and stuff is great. You also don't have to it literally is just a Python script. So you could get rid of all these function calls and just write it as a script from top to bottom. And that works, too. And you get all the interaction and everything. Yeah, but we think it's cool that you get to that you get to write some function calls and structure your code that way. And if nothing else, it actually mostly just helps me describe what's going on to the user, which is one of the great, one of the great properties of functions. Then the other thing that you pointed out, which is true, too, is that you know, we've did in one line of code, we've put a giant markdown file right on your browser. And that's the instructions for how to use the app. And that's pretty cool. Yeah, it's pretty simple. And it does exactly what you expect. And then later on, we say, hey, if you a couple lines down, we say once they select which mode they want, we just get rid of that those instructions. And that's pretty simple, also. So it's a pretty cool little guy, it's a pretty simple little way to do things.

36:35 Yeah. So that doesn't take over the whole screen that kind of just goes up there as like part of the UI. And then you say, probably when you run it first, you don't have these files. And they're like 400. Meg's, which is now time to download, but they're not they're not on a super fast surfer. So they

36:49 do take a little while to get down Actually, yeah.

36:52 And so what's really cool is you you're just using requests and standard file IO to read and write them. But as you're looping through that, reading off the stream, you have two simple lines, you have a progress bar. Yeah. And you have like a textbox area, and you say progress bar dot progress, and you give it like how far it is in the download, and then say, the weights area, you could say, we're downloading this amount, you know, like five of 10, five or 400. Meg's or whatever, yeah, what if I do. And as you're looping, you're changing those values. And that's just dynamically changing the web app. But yeah, that's part of the magic right there. For me, I think,

37:33 Oh, my God, I'm so excited that you're, you're reading the code at this level, I mean, but delight to talk about this. Yeah. So that's the that's the nudge bit of magic, which is, you can think of it as like a sort of, you know, magical print statement. Yeah, instead of saying print, blah, you say st dot write, blah. And then you can do pop things on the web browser screen from top to bottom, markdown, data, frames, graphs, anything you want. But the next bit of magic is that those things that you put on the screen are not just stuck there, you can reach out to any one of them and say, hey, I want to change you. And for example, I want to take this progress bar and move it from 51% to 52%. Or I want to take this text and I want to change it from foo to bar, that ability to reach into the webpage that you've just created in pure Python and just update things is animation. And then you throw that into a loop and all of a sudden you have progress bars going from left to right and download information. And it's really no more complicated than just writing a script.

38:37 And it looks to me like the visual layout story here is like kind of a column layout on the main screen. So I just throw stuff into the UI. Like I say, I want a dialogue, I want a text area. So I just create a like a warning text area or whatever. And I created a progress bar, I just say I'm creating one and it just it pile sort of visually piles up like one after another vertically, right? So you don't there's not a lot of position this at x y here or like set this to like dock this style. It's kind of you put them there. It doesn't feel like I'm doing UI, it kind of just figures it out. Guess what I'm talking about?

39:15 Yeah, exactly. The paradigm is just like a print statement. It's just from top to bottom. And this is a you know, I think that's the right default behavior. It's worth pointing out that the community has been saying for months, we want horizontal layout. Try to put something to the right of something else, not just below it. And I want to say to all you programmers out there, we can totally hear you. We are super excited about horizontal layout. It's coming up in the lab. Were working on some other really cool stuff like custom components that lets you insert your own custom widgets into the into the stream that page those are coming first with Still a small team. We're also really trying to hire as fast as possible. But But you know, great programmers. So all of these, we feel a lot of pressure to get things out. And we really hear you and we're getting it out. Because give us some time. And if you're, if you feel like you're the right person for the job, come join us. And we'll work with you to that way.

40:19 Yeah, very cool. It's awesome that you're hiring one other thing. So I guess I can see how that works. So I have a progress bar, I call progress, it kicks off some Asynchronous JavaScript event that's going to go over and tell that progress bar to update. Yeah, the other part that this is where I was thinking more of the magic is like, if I want, say, a slider that people can interact with, I can go over there. And I could say, go and put a slider on the sidebar, and then just ask some questions about it. And I don't see anywhere, some kind of callback for when the slider changes. I'm just getting that information. Yeah. And yeah, it's really interesting how the user interaction kind of comes back into the script. me Tell me how that kind of works, because I don't totally see like, where the user interaction response starts versus like setting up things and so on.

41:12 Yeah. First of all, I'll point out that this was just a suggestion by a user. And so this is an example of our user community, really just having been the source of the best ideas and streamlet. And he said, Hey, you just you should have a checkbox. That should just be like, you know, the answer equals checkbox. And that's how it should work. Blocking What am I yeah, like, That's impossible. There's no way to do that. But we, we then went back to, you know, the whiteboard, basically ask yourselves, how can we do just that. And so the way it works is that if you ever want to have an interactive element in your app, you basically just say, the answer equals, and then the widget types are the widget type could be a checkbox, and then the answer will be true or false. It could be a text input, and the answer would be a string, it could be a number input, or a slider, and you'll get numbers and so on and so forth. You even have file inputs, you can get works on a color picker. So there's a lot a lot of cool stuff. Okay. But there's really simple model, no callbacks, just, it's like, at this moment, I want to materialize this variable. And I want to let the user select it.

42:22 It's similar to like, if you had done like, print, which of these three options do you want input parts and internet, right? Like, it's the

42:30 looks like blocking calls, it looks like a blocking call? Yeah. And in fact, the goal is that you could just is that you can just say, you know, this is a very typical thing when you're scripting is you say, like, you know, number of iterations equals 50. And then you just keep coding, coding, coding, coding, coding, and then you say to yourself, you know, hey, it would be really nice. If someone could select the number of iterations, I'd have to keep editing this text file. So then you just go to that one line of code, you say, number of iterations equals st number input, default equals 50. And then boom, in your app, suddenly, there's a place where the user can select that. And if they do change it, what happens is the app runs from top to bottom. And at that exact moment, when that call happens, we substitute 50 for whatever number the user typed in. I see. So the function is basically rerun. That's right. It's sort of been seated to say, if they asked, tell them it was this. Okay, that's exactly right. So the script is rerun. And so the as a result of that, it is helpful if your scripts Run quickly. And so we save it. But if you're gonna rerun it a lot, it's really Exactly, exactly. And so we provide a bunch of primitives that help you sort of reuse bits of your script, so that from the previous run, so that it does run quickly.

43:47 Yeah,

43:48 I did notice that from the code there that there's a lot of caching type of things, but I didn't realize that it was to facilitate what you're talking about here. Yeah, these functions, and you can put a decorator a cache decorator, you can even say show spinner. True show spinner. False. Here's the hashing function to determine if the input is the same or not the same. It feels a little bit like the func tools LRU cache last reason, the LRU cache. Yeah, to basically just say, look at the arguments, if the inputs the same, then you already have the output just here.

44:23 Yeah, that's right. That's exactly right. So it is very, it is exactly that idea, which for, you know, CS nerds out there. It's also called memoization. And it really, it comes down to a super simple idea, which is you write a little function, and it takes some arc inputs, and it gives you back some outputs. And then you decorate it with this little Python decorator. And then lo and behold, if you run it again, on the same inputs, it just returns the same output without actually doing the computation. And that's a very useful thing in general, and it's a very useful thing in particular in streamlet, because it lets you Right, it's a much more performant app. It's also I would say, one of the greatest sources of confusion about how streaming works, you know, as people who are really trying to create easy and simple to understand tools, it is a source of some degree of chagrin and embarrassment for us that this part of streamlet is a little bit complicated. And so we have been working on a bunch of other simpler abstractions that we're going to release that will hopefully make this easier to understand.

45:34 Cool. Yeah. So maybe that's a good place to talk about the future. You wrote you published a roadmap for 20.

45:39 Yeah, yeah, there's two basic things that we're working on. So one of them is streaming for teams. And so that is the sort of Auto Deploy version of streamlet. And that lets you basically just check, you know, your apps into GitHub, and then deploy them instantly. And in fact, we are planning to release a version such that if your app is open source on GitHub, you get some amount of free hosting, which we're super, super excited about. Yeah.

46:09 So you're basically trying to leverage GitHub actions and notifications to just do continuous deployment, I'll try to do is just committed, right?

46:18 That's right. That's right. That's right. Yeah, really excited about that. It's, there's a lot of community apps that people put out there for the benefit of others. And we just feel like this is gonna just totally unblock that workflow and make it really simple.

46:31 Yeah, you're really trying to sneak in some of these like, proper CS ideas for non tsp? Yeah,

46:37 yes, yeah, you

46:38 could just simple scripts, but you're gonna have to set a GitHub repo and you have to use source control, or whatever. But you probably should have a few functions, right? Yeah.

46:46 No, I hey, I love I love that you say that. I think that's a that's an element of truth to that. But you know, it goes both ways, too. We talk to so so so streamlet really caught on in the, like the enterprise basically, too much, you know, in some ways, to our surprise how fast until we've talked to the data science teams from like, 711, Caterpillar and all these companies. And one of the things that we asked them is like, do you guys use source control? And often basically, the answer is yes. Or it's no, but we want to Yeah, so in a sense, I don't think we view this prescriptively so much as we think this is really where things are going anyway. And also, we want to use the best tools out there. And we want to weave them together. That's, I think all that, you know, basically, if you look through the GitHub, I mean, if you look through the stream of source code, it's almost like a roadmap of the coolest tools in Python, and then they're out there in the world, and so react and all this stuff. So we want to weave those together, you know, using the best possible tools in 2020. And from our perspective, git, which is not just GitHub, it's also Git lab. And you know, there's many incarnations is, these are amazing, amazing tools that hugely make people more productive, actually, once you understand them. So being able to embrace them and not reinvent the wheel, but rather be sort of get out of the way and let them do their magic is totally part of the stream of effects.

48:16 Yeah, that's cool. Other stuff, you got it on the roadmap and prove caching, which you just spoke a little bit about some DNS layout? You've done? Yeah, that is, yeah, then user programmable state is familiar. Yeah, Japs, all those things

48:30 are very useful. So the the, I think right now, the experience that people have of streamlet is, this thing is awesome. If I have this specific use case, which is an app with sliders, and you know, I don't care too much about the layout. And if that's what you want to do, then streamlet is just super cool. But there are limits to that framework, there are in terms of what's possible. And we believe that we can sort of relax those limits to the point that streamlet is really the best general purpose app framework for machine learning and data science. But in order to do that, we need to think really carefully about how to introduce new sort of AP abstractions like state industry, and that really carefully so that it maintains that simplicity that people like, but gives you a little bit more power. So the goal for 2020 is to roll out to the zeros of each one of these pieces of puzzles that by the end of the year. If people can say with some degree of confidence, hey, you know, I can build this app. It's going to go all the way there.

49:38 That's a great mission. What's the story on authorization? authentication? Oh, yeah, I would like somebody to interact with this app. But I want to remember for them, yeah, what they did or something like that.

49:50 That is part of the paid product. People who want to create it themselves in an open source, purely open source way are welcome to do that. But we're going to basically Create a really simple deployment that gives you those that functionality sort of with a couple of clicks. So you can say, scope it to these people or scope it to my company or use this existing identity provider, a sample or whatnot, and then basically create an app that is a scope to a certain number of users, both in terms of accessing the source code, and also in terms of who can use Yeah.

50:23 Okay, cool. Well, Adrian, we're getting long on time here. But it's been really, there's two, two quick things I just want to touch on before we move on picture. One is, we were talking a little bit before we hit record, and how, like many companies, you guys are finding yourself reinventing yourself as a remote company?

50:41 Yeah, how's it going? I have to say that it has been a really interesting and instructive experience. And as much as I find the whole, you know, COVID-19, very, like scary. And I worry about my parents and my fiance's parents, and all of our you know, and I just ended, we're getting a little stir crazy being at home all day long. It's also a silver lining is that we've had to really think about how to build a company that works asynchronously. We use notion all the time, which is like the coolest app of all time. We use streamlet to coordinate things. And it's been an amazing experience. And I think we are going to, we've actually made a decision to start hiring anywhere in the US now. Instead of just in San Francisco. I'm just so excited about that. I mean, we recognize that there's so much talent everywhere. And we really just assumed that they will have to be in the same room. And I think that these past couple of weeks have kind of shattered that assumption, which is kind of mine timeline. Yeah, it's

51:48 boy, it's a tough time in a lot of ways. But it's also, I think, shaking up everybody's assumptions about reality for work totally family, or even simple stuff like food, right? They take out a lot and whatnot. And like, we're just gonna have to start making more arresting food. So yeah, I cooked like a really good dinner last night. And, but it's like, totally, it's just all these different stages or parts of our life or just like Shake, shake, shake. Let's see what shakes out what we really need or what it really is the right fit. And so it's really cool to hear that you guys are finding the silver lining in it.

52:25 Yeah, yeah. And one other one is my son, who's eight. Now he has like literally three playdates a day over like Google Hangouts. Like he plays like, yeah, you know, fortnight and then he plays Minecraft. And it's like his, he has a whole new social life that he never had before. And he actually told me the other day, he's like, I'm sad. I'm gonna be sad when COVID is done, because I want to have all these playdates over hanging out with all my friends. And I was like, You know what? You still can? that's never gonna go away. We just, we just figured out how to do that. And you'll have it forever.

52:58 Yeah, that's really cool. My daughter's definitely been doing that as well. Like, she'll sit down at the Xbox with her friend and like prop up an iPad next to the

53:07 character in

53:08 there. It's like it's in that regard. I'll touch on something I'm a fan of and I just learned about reading your bio that you were involved in is the lo and behold, documentary. Oh, yeah.

53:26 Yeah, people have not seen lo and behold, it is an incredible documentary on the history of the internet and technology in that era. Yeah. What was your involvement there? That

53:36 sounds fun. I was interviewed. I was one of the sort of like Internet pioneers because of my work with folded I was probably like the least famous person by far. I think it was like Ilan musk and all these people. And so I'm sure that they when they watched it, they're like, Who's this? You know who this guy, but it was because I fold it and you know, the one of the coolest things for me was being able to meet Verner Hertzog, who's just like, you know, he has this sort of shamanistic way of being in the world. And so, you know, he gave me advice like, Adrian, if you want to understand some part of the world, you must walk through it on two feet. Like that says Like, it was really cool meeting him and obviously it's legendary.

54:22 Yeah. Oh, how cool. I just if you are looking to watch something interesting, lo and behold is a fantastic option there. Yeah. All right. Well, final two questions before I let you out of here. Maybe you're gonna work on streamlet or other Python code?

54:36 What editor do you use? Oh, VS Code all the way. All right, right on

54:40 although although I've tried in COVID one of my little side projects has been to try to figure out how to code on my iPad. Yeah, and so now I'm if anyone knows like how to do this, like tweet me that your secrets because I'm like really trying to figure this out just kind of for fun. But on the iPad, there is no VS Code. So I love If it were VS Code, and please, someone make that happen, I'll give

55:03 you two things that you can play with it. I know I'm sure they could put it at the bottom of the show notes. Yeah, comments on the episode page. Make sure you get those as well. Yeah, but one is Carnot car in ETS local Jupyter notebooks for iOS. Like no network everything, which is, if you will. indigene notebooks. It's Yeah, JupyterLab on your iPad. It's pretty cool. I think it might even be open source, I'm not sure. And then you can set up Visual Studio Code online. So Visual Studio online, and then run that I think you can run that in your web browser and basically get VS Code. I've been trying to do that. And I cannot get into working now. Yeah, I can. Yeah, I can. I pad browser just doesn't quite like habit. It's missing a couple of things that. So I've been that's been my I've been trying to do that. And maybe I'm getting something wrong. So anyway, maybe I do feel like that's actually one of the big mistakes that Apple's made is not allowing any innovation on web browsers on their platform. Totally. bizarre to me. So anyway, yeah. Yeah. Cool. Cool. All right. And then notable pi package something, some Python package that you're like, maybe it's not the most popular, like, wow, people should know about this. And I mean, obviously, pip install streamlet is one option. But if it comes to I

56:30 guess, I don't know how I'm famous. This is I think it's semi famous, but I would really tell people, I want to shout out to Altair, basically, you know, a lot of people know matplotlib a lot of people know plotly a lot of people know bouquet think Altair is a little less famous and that the short story is, they took d3, which is legendary thing. They simplified it into a declarative JSON language called Vega. But that was way too complicated to simplify that into an even simpler language called Vega light. And then someone created at Google created Python bindings for that. And it's called Altair. And this thing is its own philosophy, man. But when you learn it, it is so cool and create beautiful graphs, and they're interactive. And they just look great. And they're easy to create. And I just think Altair is great. Awesome. Yeah,

57:19 I've definitely heard good things from Altair out there. So good. Agent, final call to action. People are excited about streaming that want to check it out. do more with it. What do you say?

57:30 Go for it. super fine. stream.io or just pip install streamlet and then type stream with that Hello on the command line. And you'll be playing with interactive stuff in minutes. Awesome.

57:41 Well, thanks for being on the show. It's been really great to talk to you about your project.

57:43 Yeah. Thanks so much, Mike. I really appreciate it. You're welcome. Bye. Bye. All right, take it back.

57:48 This has been another episode of talk Python. To me. Our guest on this episode is Adrian trail. And it's been brought to you by linode. And Ruben learners Ace Python interviews course. Start your next Python project on the nodes state of the art cloud service. Just visit talkpython.fm/ linode li n o d, you'll automatically get a $20 credit when you create a new account. As that Python interview, let Raven learners course help you get that next Python job just visit talk Python dot f m slash Ace to take his free course. Want to level up your Python. If you're just getting started, try my Python jumpstart by building 10 apps course. Or if you're looking for something more advanced, check out our new async course the digs into all the different types of async programming you can do in Python. And of course, if you're interested in more than one of these, be sure to check out our everything bundle. It's like a subscription that never expires. Be sure to subscribe to the show, open your favorite pod catcher and search for Python. We should be right at the top. You can also find the iTunes feed at /itunes. The Google Play feed is /play in the direct RSS feed at /rss on talk python.fm. This is your host Michael Kennedy. Thanks so much for listening. I really appreciate it. Get out there and write some Python code