#218: Serverless Python functions in Azure Transcript

00:00 Michael Kennedy: Do you have stateless code that needs to run in a cloud? The clear answer years ago was to create an HTTP or even, gasp, a SOAP service, before then. While HTTP services are still very important, serverless code can be moved entirely away from the framework that runs it and over to serverless programming and hosted functions. On this episode, I meet up with Asavari Tayal to discuss serverless programming in the cloud. This is Talk Python To Me, Episode 218 recorded live at Microsoft Build on May 8th, 2019. Welcome to Talk Python To Me, a weekly podcast on Python, the language, the libraries, the ecosystem, and the personalities. This is your host, Michael Kennedy. Follow me on Twitter where I'm @mkennedy. Keep up with the show and listen to past episodes at talkpython.fm and follow the show on Twitter via @talkpython. This episode is brought to you by Command Line Heroes from Red Hat and Datadog. Please check out what they're offering during their segments. It really helps support the show. Asavari, welcome to Talk Python.

01:10 Asavari Tayal: Hi, Michael, thank you.

01:11 Michael Kennedy: It's super to have you here. It's great to meet you live at Build. So, it's going to be a lot of fun to talk about serverless.

01:17 Asavari Tayal: Yeah, excited to be here.

01:18 Michael Kennedy: Yeah, it's cool. So, it feels like everyone I've had on the show lately is doing this sort of dual life, right? Like, we're at PyCon, and then now are at Microsoft Build.

01:28 Asavari Tayal: Yep, yep.

01:28 Michael Kennedy: Same for you?

01:30 Asavari Tayal: Same for me, we were just in Cleveland at PyCon and it was great, so, now I'm coming back to Seattle, back home, and enjoying Build.

01:38 Michael Kennedy: Yeah, it's, both of those are great conferences. What was your thoughts on PyCon, did you enjoy it? You had a good time there?

01:43 Asavari Tayal: Yeah, this was my second PyCon ever. Definitely a lot of fun. I really like the sense of community that Python developers have. It's different from all the marketing conferences we go to. Build being one of them, so, always a pleasant experience, always good.

01:58 Michael Kennedy: I would say one of the biggest differences I felt was when you walk onto the expo floor at PyCon, it's like, everybody there is kind of doing their own thing, their own separate thing. It's just a sampling of the community, whereas, places like Build, they're great, but, they're like, it's more like, here's the team for this or here's the team for that, right? It's much more sort of focused.

02:19 Asavari Tayal: Right, I like the fact that at PyCon everybody was there to share their stories of hey, this is how we use Python, this is what we build and maybe it's useful for you as well, so I'm going to share my story with you. That was definitely my favorite part of PyCon.

02:32 Michael Kennedy: Yeah, that's awesome. I had a great time there as well. I always enjoy that conference. So, let's go ahead and get started with your story, though. How'd you get into programming and Python?

02:40 Asavari Tayal: Yeah, I started web development when I was 15 and definitely my first programming language was JavaScript, but eventually, as I got exposed to Python and remember, I think my first web application that I developed was in Django, was developing...

02:58 Michael Kennedy: Oh, right on.

02:59 Asavari Tayal: Like a portal of some kind for farmers to go up and look up various grain imaging systems that was being developed by a computer vision lab. So anyway, that's how I started Python and eventually, when there was an opportunity to make Azure great for Python developers, it just happened that I was there and I had some experience with Python and was basically asked to take over Python for functions and app service.

03:25 Michael Kennedy: Yeah, that's super cool. Tell me more about this Django app.

03:27 Asavari Tayal: Yeah, so the Django app I was building was back in college. I was working with the University of Illinois Computer Vision Laboratory and they were building an algorithm that could take as input the image of some grain and basically identify the quality of that grain based on a machine learning model or computer vision model. So, I wasn't as involved with the data science part of things but at the end of the day, the algorithm needed to plug into a portal that the farmer could go interact with. Plug in their grain and get a market value for it. So that portal was something I built using Django.

04:04 Michael Kennedy: Yeah, that sounds like a really fun first project, actually. Nice. Okay, so that was then. Did you study computer science at University of Illinois?

04:12 Asavari Tayal: Yes.

04:12 Michael Kennedy: Yeah, cool. And now you're at Microsoft working on the Azure team.

04:17 Asavari Tayal: Yes.

04:17 Michael Kennedy: All right. Tell me what you do there?

04:18 Asavari Tayal: Yeah, so I'm a product manager on Azure Functions. And currently, I'm leading both the Python experience in Azure Functions and Java experience as well. So really, what that means is, everything that needs to be built, either on the platform tooling or the runtime, I go in and build. So, trying to understand the story of the journey of a Python developer on the Azure Functions platform. How they come in, what their problems are. So, on a daily basis speaking to lots and lots of customers about why they're trying to use serverless, why they're trying to use functions, what are some problems they face today and how we can solve them. And really taking those requirements back to the engineering team, building POCs and prioritizing features on the product.

05:02 Michael Kennedy: Yeah, that's cool. Did you have a lot of those conversations at PyCon?

05:05 Asavari Tayal: At PyCon, yes, also at Build the last two days. It's great to see that the number of folks who are developing with Python are increasing at Build every year.

05:14 Michael Kennedy: Yeah, it's pretty cool. Did you guys have a expo stand area?

05:18 Asavari Tayal: We do, it's pretty, there's a lot of hustle-bustle around the functions stand. It's been great to see all the interest.

05:26 Michael Kennedy: Yeah, I'm sure it's a fun area to work on.

05:28 Asavari Tayal: Yep, for sure.

05:28 Michael Kennedy: Cool. So, before we get into the details of Azure Functions and the Python story, let's focus on just serverless. We had servers and then we had the cloud. Had kind of virtual servers and then we have serverless. Walk us through that.

05:43 Asavari Tayal: Right, so we almost think of serverless as the evolution of application development on the cloud. You know, we used to have on-prem, or on-premise, and then we moved on to IaaS, then we moved on to PaaS, and now serverless is even the next level of PaaS. What that means is, hey, PaaS already took away your problems of having to manage physical hardware when it comes to servers. Now, you only need to worry about what's running on that hardware but you still needed to worry about what OS you're using. What packages or dependencies are installed on that OS. Patching those OSes, upgrading those machines or even making sure that you're doing the security patching, and right you make sure you're taking care of any vulnerabilities. Serverless takes away all of that because at the end of the day, what you need to do as a developer is give us your application code and we'll run it in the cloud for you. Very important concept in serverless is that you also don't no longer need to estimate how many app servers do you need to run your application. So typically, when you were building an app, you would have to think, hey, my app's going to get 20,000 requests. and so I estimate I'll need about maybe 100 servers to serve those requests coming in. You no longer have to do that with serverless because we're going to automatically scale based on events that your application's receiving.

07:05 Michael Kennedy: Right, you don't have to worry about memory. Like, your server running out of memory 'cause it's doing too much and things like that, right?

07:11 Asavari Tayal: Exactly.

07:11 Michael Kennedy: Except the individual level, maybe.

07:13 Asavari Tayal: Exactly, and that's why when we talk about serverless, we also start to talk about event-driven serverless. Which is really important because if you write an application that's going to be triggered off an event in the cloud, something like an HTTP request or an event in some source system somewhere, could be storage, could be an eventing mechanism, we are constantly watching for those events, and when those events occur, we'll go ahead and spend the resources you need to run your application. And when more of those events happen, we go spend more resources. So, you're really never out of compute or memory in that model.

07:45 Michael Kennedy: Yeah, that's pretty cool, so maybe some scenario like that fits in my world. Maybe I have some videos from an online course but they're in a raw form. I want to get them transcoded for the right kind of streaming. I could upload them to a storage. That could trigger an event, a serverless function which grabs it, runs it through a transcoder service, and then drops it back maybe like in a file in a location, something like this, yeah?

08:06 Asavari Tayal: Exactly, both, I think in the scenario that you described, there are two important factors. One, your scenario was event-driven. You were going to kick it off and then process something and drop it in a queue somewhere. And the other one, that it seemed like it was lightweight job that you maybe didn't need to stand dedicated servers up for.

08:24 Michael Kennedy: Right, can you do, like, timing? Like, can I say, run this function every hour?

08:28 Asavari Tayal: Yes, absolutely, we've got the timer trigger, so, if you go ahead and set the time that you'd like your function to run on, we can always do that.

08:34 Michael Kennedy: Cool, so, I guess maybe I would put that into like an automation category. Like, what are some of the other main scenarios?

08:40 Asavari Tayal: I guess this was automation. The other two scenarios that we talk about with Python, the first one is web scripting or APIs. If you, for instance had a Cosmos DB instance and you wanted to go ahead and stand up a REST API in front of that Cosmos DB, you could do that using Functions.

08:58 Michael Kennedy: Okay, so it might be like an easy way to build an API endpoint that serves up JSON or something like that.

09:03 Asavari Tayal: Exactly.

09:04 Michael Kennedy: How would I, I guess in the Python space, what technology would I use that? Like, if I were building an API and I was hosting it in uWSGI, I would do that with Flask or Pyramid or API Star, something like that. Like, do I work at that level in these functions or what's the story there?

09:20 Asavari Tayal: Yeah, so when you develop a function you would have to pick a trigger for your function, right? That's the event that we were talking about earlier. In this case, as long as you tell us, hey, it's going to be an HTTP trigger, you can figure the parameters you want for that request, things like, are we accepting a GET, POST request or are we authenticating against an API key or an AD or an OAuth two mechanism. As long as you give us that configuration and give us the Python script that you'd like to run when that trigger occurs, we'd be able to run it as a function for you. You no longer have to go in and define routes in a Flask app.

09:55 Michael Kennedy: Right, do I return a string? Do I return a dictionary? Like, what do I return?

09:59 Asavari Tayal: It's up to you. We have this concept called bindings in Functions. So, you can actually return or you can bind your return value to a data source. So for instance, if you were to say, hey, I'd like an output binding for my Function app, which was Azure Queue storage, for instance. And I'd like to bind that to the return value function. Now, whatever you return, whether it's a string value or a byte array or a JSON string or a stream, we'd go ahead and use that to create a message in Azure storage queue.

10:31 Michael Kennedy: Okay, that's pretty cool. That's the web side and what's the third one?

10:35 Asavari Tayal: The third one that we're actually seeing very often is ML inferencing. So, I mentioned I've been talking to a lot of customers for serverless, specifically Python applications and serverless and really seeing two personas. One, lot of data scientists who are coming to us and telling us, hey, I trained a machine learning model. Now, I'd like to deploy it as a web service. Serverless sounds like I no longer need to worry about all those things that typically I don't care about and I would have to go spend time on figuring out.

11:08 Michael Kennedy: Right, as a data scientist you don't care about configuring uWSGI for high performance or whatever.

11:14 Asavari Tayal: I'm not a data scientist but I imagine for a data scientist, those problems are not even fun. You like thinking about the algorithm you want to use for training your model or the libraries you want to use. Probably not about what packages to install or when to security patch your OS, right? Serverless is a great scenario there because you give us your model, you give us your scoring script, tell us what event to use and we go ahead and provision the application for you in the cloud, right? And then there are also second kind of category of the people or customers that we speak to who are seem maybe not mainstream Python developers. They might say hey, we work at a large enterprise and we're using Java for our mainframe application but our data science team gave us this thing called a model and we need to run it in our application or use it in our application to make some predictions. Doesn't look like I can do that with Java. I'd like to do that with Python. Serverless is a great way to create a microservice that you can invoke via an HTTP endpoint all written in Python that you can call from your mainframe Java application if you wanted to.

12:16 Michael Kennedy: That sounds pretty cool. Do you find students using it a lot for like testing out their code or creating little projects?

12:23 Asavari Tayal: That's a good one. I still think serverless is very new to the student community. We're trying to create more awareness in that space. But definitely spoken to a couple of people who've taken an interest and started to use serverless more on the cognitive services side of things but I think we still have a long way to go there.

12:40 Michael Kennedy: There's also kind of a lag between academic courses and stuff like that, right? They don't seem to change technologies super fast. So, I can imagine that. So, you're talking about Azure Functions and obviously, it supports Python. And that's great but just to kind of round out the story, what other languages does it support?

12:58 Asavari Tayal: Yeah, Functions supports Javascript, Typescript, .NET, Java, and we recently announced PowerShell, as of a week ago.

13:07 Michael Kennedy: That sounds pretty cool. What does a workflow look like? If I write regular functions in a Python file, I know what that is. But, how do, I don't edit this on Azure or AWS or whatever I'm working on with my functions? What is the workflow kind of look like there? Give me a sense.

13:23 Asavari Tayal: To help you develop functions, we basically have two tooling options. And these sort of walk you through the process of building a function. The first one is, we've got an extension with Visual Studio Code where you can go in and say, hey, I'd like to create a project which is a Function project. Pick a language for your project, that would be Python. Pick a trigger for your very first function. It could be an HTTP trigger. And then it will populate a template for you that you can start modifying. So, when I say a template, it's basically a combination of two files. One's a template Python script which pretty much has a single function. It's a Hello, World! function, takes an HTTP request, returns hello, name as the response.

14:04 Michael Kennedy: Right, so that's where you write your normal code, right?

14:06 Asavari Tayal: Exactly.

14:07 Michael Kennedy: You can do pretty much what you want between the beginning and end there. As long as it conforms to the shape?

14:13 Asavari Tayal: Exactly, you can do pretty much what you want, and you can also load up additional modules within that folder itself. So you don't have to have all your Python code living in that file. If you had dependencies or in our ML case, if you wanted to throw in a model as a dependency in your function, you can go ahead and add all of the data within your function project.

14:31 Michael Kennedy: And what about packages I'm using? Suppose I'm using requests or NumPy or whatever?

14:35 Asavari Tayal: Right, so we've got a requirements.txt format that we support. You automatically get that when you generate a new function project. Just go and type in your packages and the versions you need and we'll go ahead and install them for you when you deploy.

14:48 Michael Kennedy: Yeah, it's cool, so basically it creates a little virtual environment for the thing?

14:52 Asavari Tayal: Right, when you're developing locally, we actually leave it up to you for how you'd like to manage your dependencies as long as you're running in some environment where the Python path does have the packages you need. We'll be able to identify the packages on the path and run. So, we leave it up to you whether you're using venv, pyvenv, the global Python installed on your machine. Definitely feel like Python developers are passionate about how they manage their virtual environments.

15:17 Michael Kennedy: For sure.

15:17 Asavari Tayal: In the cloud, the only contract is that you give us a requirements.txt file and we'll use that to install the packages.

15:25 Michael Kennedy: This portion of Talk Python To Me is brought to you by Command Line Heroes. Command Line Heroes is an original podcast from Red Hat and it's back for its third season. This one is all about the epic history of programming languages. The very first episode explores the origin and evolution of Python. Let me tell you, this show is really good. Its has a great host, it's highly produced and edited. Imagine if Radiolab made a tech podcast. Yeah, it'd be like that. Even better, this particular episode has a bunch of cool Python personalities, such as Guido van Rossum and I even make an appearance in there. Listen and subscribe wherever you get your podcast or just visit talkpython.fm/heroes. That's talkpython.fm/heroes. Do you pre-compute that little environment or something? So, 'cause you don't want to obviously reinstall it per request.

16:14 Asavari Tayal: Yeah so, it basically gets installed during deployment time. When you deploy your function, we take your code, install the dependencies that it needs, package it up as a ZIP package, store it at Azure Storage, and when your application's invoked, or the first call comes in, that's when we go ahead and run that ZIP package on the platform.

16:34 Michael Kennedy: How's it run? Does it run in a Docker container? Does it run in a VM? How is it shared? What's that story look like?

16:41 Asavari Tayal: Good question. You shouldn't worry about it because it's serverless but since you asked nicely, I'll share it with you. We do run in a container underneath the covers. Good thing is our Python offering is actually running on Linux, which is something that's brand new to Azure Functions. Traditionally, we only supported Windows hosting. Just given the kinds of scenarios that the developers are bringing in, we decided to go ahead and host Python on Linux. So, underneath the covers things are running in a Docker container on Linux. We actually go ahead and even open source that image. It's called the Azure Function's Python image on our Docker Hub profile. And you can use that to build your own custom container or if you're just giving us code, we'll go ahead and run that code in that container for you.

17:27 Michael Kennedy: Oh, that's cool. Could I use it to test locally? Could I Docker pull, Docker run that image locally and play with it?

17:34 Asavari Tayal: Yeah, you can absolutely do that. If you wanted to have the container approach the cool thing is you don't need to. We've got something called the Azure Functions Core Tools. It's a CLI experience as long as you install the Azure Functions Core Tools on your local machine and VS Code in fact does that for you, you can test your functions locally simply by hitting F5 in VS Code. It automatically starts the function process in fact, it'll even identify that it's a function process and go ahead and attach the Python debugger to the functions process. So you can start serving your function on the local host endpoint and start testing against real Azure events.

18:12 Michael Kennedy: That sounds pretty cool. I saw that it had a command line. So you can do things like go in and say like, it's the func command, right? So you can go and say func new and then you can say, what is it, func run or something like this to like...

18:26 Asavari Tayal: Yeah, func host start.

18:26 Michael Kennedy: Yeah, func host start. That's right, it sounds funky. But yeah so, so you can do that in the command line as well. If you don't use Visual Studio Code. That sounds pretty cool.

18:36 Asavari Tayal: Absolutely, so you can use the command line in conjunction with your favorite editor. So if you were using PyCharm that's what some of the users are using as well. You can definitely go ahead and attach.

18:46 Michael Kennedy: Yeah, that's cool. Like one of the things that drives me crazy about working with the cloud, any cloud, right, whatever, is the sort of airplane scenario as I see it. All right like I'm working on this project and now I'm somewhere without internet or I'm in a hotel with bad internet or whatever and it's just like well, now you're done, you know? It doesn't work anymore. And so it's cool that you have this way to kind of like run it locally regardless of that.

19:12 Asavari Tayal: Yup and it's the same run time that's running in the cloud as well so there's nothing different about it.

19:16 Michael Kennedy: Okay, well that seems pretty cool. We talked about some of the events. We talked about HTTP. We talked about, I ask if I can do it on a timer like cron job equivalent which is pretty cool. What are some of the other ones?

19:27 Asavari Tayal: All the storage events. So, Azure storage, queue blob, Cosmos DB events. Also, the eventing and messaging systems. So Service Bus queue, Service Bus topics, Event Hubs, any IoT hub messages coming to an Event Hub, Event Grid and the cool thing is as of two days ago, we also announced Kafka. Kafka Triggers are supported as well. Going back to functions being open source I feel like sometimes I can't say that enough. We've got an extensibility model where we do support any trigger or binding that you want to bring in. An example would be somebody recently wrote a SignalR binding for Python. And so pretty much any binding that you bring can plug in to our extensibility model as well.

20:09 Michael Kennedy: That's pretty cool. Could I build a Slack one? So anytime a certain Slack message comes in?

20:13 Asavari Tayal: Yes, I actually just built a demo for that yesterday.

20:16 Michael Kennedy: Oh really?

20:17 Asavari Tayal: We should talk about that.

20:19 Michael Kennedy: That's pretty awesome. What was the scenario you were like laying out there?

20:22 Asavari Tayal: Yeah, so we wanted to build a demo when the data of issues filed, I'd liked to go ahead and process that issue, do some sort of sentiment analysis on it and go ahead and post to our Slack channel. So that was the simple scenario was building, but definitely felt like just having those bindings to these data sources or just being able to write my Python code and not focus on anything else made things really easy for my demo.

20:47 Michael Kennedy: Of course that's super cool. You've mentioned the IoT stuff, and that seems like a really interesting angle right, people create these little IoT devices and they've got to be sitting back like tons of data in some cases. So, I can just somehow subscribe to that in particular and not necessarily have to write in HTTP endpoint, what was that story there?

21:07 Asavari Tayal: Right so the IT events are generally are routed through IoT hub to event hub, so as long as these events are coming into event hub, we've got an Event Hub triggered so every time a message is received on Event Hub we'll go ahead ahead and trigger or invoke your function script, and pass the message coming from, or the data coming from your IoT device for the function to process. So, that actually makes a very popular scenario for severless as well, because IoT scenarios are generally burst nature. So you settling 1000 events that you need to process, send spin up functions for, and then it might just go down to zero. So really it doesn't probably make sense for you to be paying for VMs that are running 24 hours to serve that one time burst.

21:53 Michael Kennedy: Well, you talk about paying and pricing, right like if I go and spin up virtual machine Linode or somewhere like that, I know that it's $5 or $10 a month flat right, there's no surprises. If I leave it on, it's $10 let's say. What's the story with functions? I feel like of some many of the cloud resources, it's not always sure about what functions, it's like really not clear, like all the time like what is this going to cost. There's a lot of factors right, not just how many times it's called but how much work do these function do, things like this. How do I know what this might cost or is free tier, things like that.

22:32 Asavari Tayal: Yeah, what functions we actually support few different hosting plans, and the price is typically tied to what hosting plan you pick. So the first and the most popular one is the serverless, so the truly severless would be called the consumption plan. What that means is that when a function execution occurs, there is a flat rate for that function execution and you only get charged for the compute that you use GB per second to run that function. So your actually compute consumption determines the cost. And of course any other resources that you might be using during that consumption, memory, etc, that's the truly serverless plan, on that I believe we give you a, and I do need to go back and double check this ones, but I know we give a million executions free to get started, I forget the other thing that we also give you free with the memory. I should go back, double check that. The other hosting plan that we have is called an app service plan, and this is different from what a lot of the other serverless vendors actually provide because we had a bunch of customers who told us "Hey, I love the functions programming model, but you know I already have this set of VMS that I'm using to run a web application maybe, can I just use the extra compute available to me on this VMs to run my function as well, and scale within those set of VMs?" And so that's what we call the app service plan, hey, it's not truly serverless but if you already have a set of VMs tell us about those VMs and what plan they're running on called the app service plan and we'll go ahead and run the function on that for you. It's not serverless scaling, it's CPU based scaling, so it's scaling based on CPU metrics how much CPU's being used or how much memory's being used. But it's kind of for a determined set. It does give you advantages like now you can put your functions within a V-Net, so there's networking options, you can connect to data in a V-Net, you know those kind of capabilities, so we more or less had these two groups of people the serverless plan people really happy about their true serverless...

24:34 Michael Kennedy: Right they just have to think about it, it does its magic.

24:36 Asavari Tayal: Exactly.

24:38 Michael Kennedy: It's the cloud.

24:39 Asavari Tayal: And then we had the folks who were super happy with the app service plan because the performance is great because things are already provisioned for you, and all the networking capabilities. So in the last few months we actually went and did some work to enable a third hosting plan which we're very very excited about actually, it's called the elastic premium plan. And in the elastic premium plan what you can essentially do is, you can tell us how many reserved VMs you want for your application. So if you said hey I have an app, and I almost expect it to consistently get a certain load that I serve with a certain reserved number of instances. Say those instances, even if it's just one instance, I always want to have my application warm and running on that one instance, but if I receive traffic that's more than that instance can handle, I'd like to scale out to meet that demand, and scale back down to one when the demand's gone. Which is cool, because now you get the benefits of really good performance, because reserved instance, networking options because you already have something that's provisioned there, and the serverless scale.

25:47 Michael Kennedy: Yeah that sounds pretty cool, most of the time probably just one VM may do it and one of the things that I think that I'm always a little suspicious of in the serverless world is there's how long your code takes to run, and then how long it takes end to end for the request to come in, for the temporary docker image to spin up and get created and all that kind of stuff and it seems to me like you might be able to avoid a lot of it in this scenario you're talking about here, if you have your own VM that's dedicated to it.

26:20 Asavari Tayal: In fact, what you explain right now, we're already optimizing for that so the whole point of us running in the docker container which we are certain what container it is is the fact that it's already warm and running even when the first call comes to your application, you're actually running a dummy Python site, and when we open up your application we go ahead and specialize that site with your code and with your application settings in order to evoke your code.

26:46 Michael Kennedy: A lot of that's already been done?

26:47 Asavari Tayal: A lot of that's already been done, so we're optimizing for performance even on the serverless plan, that said though even doing that job takes a little bit of time because again, it is serverless, we're not provisioning anything for you, so when you do specialize something it takes a little bit of time. If you absolutely had a workload that could not function on that kind of latency either, premium plan is a great option.

27:10 Michael Kennedy: Interesting, there's a lot of talk about docker and docker images and stuff in here and it's really cool and obviously gives you a decent level of isolation in your environment and stuff like that, but would I be able to create my own docker container, say instead of taking the one that we talked about that we could get docker pull and mess with, could I base a custom image on that and get it exactly the way I want, and then run a?

27:35 Asavari Tayal: Yeah you can absolutely do that. So, like I said, our docker image is open source, you could go ahead and extend that image to add whatever requirements you wanted to add, and the typical reasons why we see somebody would do that is if you had dependencies that weren't essentially pip installable, but something that was an OS level dependency, maybe you wanted to install the PyTorch package we apt-get on your container right, in that case custom container is a great scenario. The other one is if you were using a distribution of Python that we didn't support out of the box, something like the Anaconda distribution. Again go ahead and customize the container with the Anaconda distribution, and you can still use functions with it.

28:16 Michael Kennedy: Okay then I put it like bob storage then point to it in my container, or my function constitute, how do I do it?

28:21 Asavari Tayal: You can simply push your Docker image to either Docker Hub or ECR and we also support private repositories, but as long as you bring us the image itself, we'd be able to use that to create a function app.

28:33 Michael Kennedy: This is the next thing, a lot of these cloud and networking ideas together right?

28:38 Asavari Tayal: Right and that's why the only career I will mention though with custom containers is we will not be offering the consumption or the serverless plan with custom containers, we are sticking to the premium plan and the app service plan to optimize for performance benefits.

28:52 Michael Kennedy: Sure, so we talked a little about being able to run with that func host start command, run locally R code, but if you're going to do a production release of your code you probably want to have continuous integration, maybe continuous employment, like how do I test this stuff, can I test it in some kind of CI system, or something like that?

29:17 Asavari Tayal: We integrate with Azure DevOps, Azure DevOps is basically a CICD tool, specifically Azure Pipelines is the CICD component of DevOps. And it provides you with multiple client touch points, you can go to this portal where you'd say "Hey I'd like to see up a DevOps Pipeline or a CICD Pipeline for my function app." We're already populating functions in Python template in there, so it'll be able to generate the structure of the various steps you'd need to take in order to do CD, and you can go ahead and integrate your own CI in there as well. We've also got the command line experience, which is starting from if you've already developed a function app, you can go ahead and run the easy function app DevOps create command, it's documented in the functions documentations. But that'll automatically identify that the app that you're using is a Python app, it contains requirements.txt which has certain packages, and go ahead and generate what's called an Azure Pipelines YAML file, that contains exactly the script that needs to run for your CD step.

30:23 Michael Kennedy: Okay, and then how do I test it? Can I test it with like pytest, or what is the story?

30:28 Asavari Tayal: You can definitely write unit tests against pytest, for each of the triggers and bindings that we integrate with, we're including in the library of mock interfaces.

30:37 Michael Kennedy: Right okay so if I want to have like say a Cosmo DB or like a Blob storage one, I guess that would be hard to test. Like it would be hard to simulate something going into blob storage right?

30:48 Asavari Tayal: Exactly so in that case you'd use that mock rich type that we're providing to create an object that you can test against in a CICD kind of setting. If you were to be testing locally however, we do let you run against real time Azure events. So if I started a function that's going to be triggered by a message being added to queue, when I'm running it locally on my local host end point I can go to Azure, add a message to queue, and it will trigger off the function.

31:16 Michael Kennedy: Yeah that sounds like a pretty cool way, so that's more of a development side of things.

31:19 Asavari Tayal: It's more like the inner loop, outer loop story. So we think of hey when you're developing locally just try to get your function to work in the way you want it to, in a loopers create you develop with the CLI, you test against real time Azure events, now when you think you're ready with your function app and you'd like to move to the cloud, that's when you go ahead and initiate the CICD or DevOps create command.

31:41 Michael Kennedy: Okay, I'm happy with it, I want to put it out there, how do I do that without suppose, I have a function, it's getting 100 requests per second, right even if it only takes half a second to deploy this thing, that could be a problem. Is there a way to roll it out so that it kind of upgrades without breaking inbound requests?

32:03 Asavari Tayal: We've got this feature called deployment slots, and essentially what that means is hey, you can have different kind of canary environments that you're publishing your application to, for instance I could have an app that's running in a production slot and 100% of my traffic is coming to that slot, now I've got an updated version of that app a , I'll go and create a separate slot which is, you know my canary slot say, and publish my application to that slot. And I can start navigating some of my traffic, the incoming traffic to that slot, so that maybe 80% of my traffic's coming to production, 20% is going to canary. And so on and so forth I start to navigate the traffic until I feel very comfortable that my staging slot or my canary slot is doing what I expect it to do, and bring my number of requests on the production slot down to zero before I swap.

32:57 Michael Kennedy: Yeah that's actually cooler than I expected I figured there might be some kind of Kebernetes sort of upgrade story type of thing, but ability to send 1% of your traffic at it and just go let's just make sure that this is going to hang together, yeah that's pretty cool. This portion of Talk Python To Me was brought to you by Datadog. Get insights into your Python applications and infrastructures with Datadog's fully integrated platform. To bug and optimize your code by tracing requests across web severs, databases and services in your environment. Then co-relate and pivot between distributed request traces, metrics and logs to troubleshoot issues without switching tools or contacts. Get started today with a 14 day trial and Datadog will send you a free t-shirt. Visit talkpython.fm/datadog for more details. So, at Build there's always a lot of announcements from folks at Microsoft, this is one of the conferences that you hold back all of your big news for, right, until you let it out here. So I guess maybe give us a rundown on what's the big news for you and the serverless and Python.

34:08 Asavari Tayal: Yeah, it's interesting because we've actually been releasing pretty exciting announcements over the last month, month and a half almost. So the elastic premium plan that I spoke about went out about a month ago, and that's the biggest thing even at Build we're talking about it and a lot of people already knew. So I would say that's the biggest announcement of it all. The second one also super exciting is support for Kebernetes. So, functions was already open source, the runtime and the programming model and the framework itself, but there was still a component of the platform that was still novel to what Azure was doing, that was called the skill controller. And just to give you a little rundown, skill controller is the component that's responsible for event based scaling, so for instance if you have a function that was triggering off messages in a queue, our skill controller's always watching that queue, how many messages are on that queue, what's the size of each message, how long is it taking to process. And depending on that actually scaling your infrastructure. Now, we actually went in and rewrote that skill controller in Golang and made it open source, so you can now deploy your functions containers or function apps to Kebernetes and also take advantage of the event based scaling. It's a project we're calling Keda, and it's an attempt to really open source a lot of what we're doing in functions and try to reduce the window lock in that may still be remaining. For instance I mentioned earlier in the podcast that we added support for the Kafka trigger, that was motivated by the Keda project. We're also looking at adding support for things like RabbitMQ as triggers.

35:51 Michael Kennedy: That sounds super cool, because to me I feel like the cloud is the new lock in, right for both good and bad, not just Azure, but AWS and the other ones. The more you take advantage of these really cool features, the more you are kind of committed right, but it sounds like with this as long as you're willing to run a Kebernetes cluster you can kind of bring it wherever you want it to go.

36:13 Asavari Tayal: Exactly, and it's more for the audience I'd say who are already brought into Kebernetes. You know you don't have to do that if you really, truly are serverless. Just leave the infrastructure to us and let us do what we do best, but if you're a big enterprise and everything you have is going to be running on Kebernetes and you want functions to be a part of that, it's a great way to still be able to take advantage of the eventually serverless functions.

36:38 Michael Kennedy: Okay so I already have a Kebernetes cluster, I can just take this and run it on the cluster as one of the things and then start calling the endpoints there. Where's the rest of the infrastructure, where are the HTTP endpoints that tie it back to the functions, you know stuff like that. Is that enough to just sort of have my own little private serverless thing in some cluster or what?

37:00 Asavari Tayal: Yeah absolutely I'd actually, without going too much in detail on Keda I would encourage you to look at the documentation it's got some great references on how things are actually working in the Kebernetes space there is a lot going on in terms of your cluster, your pods, I'm still learning about Kebernetes to be honest. There is great documentation online I encourage you to go look up Keda, Keda project.

37:23 Michael Kennedy: Okay cool, what else?

37:25 Asavari Tayal: We introduced this concept called extension bundles, it's a little underneath the cover so you don't really need to know about it but really, historically what happened was because our core runtime is written on .NET, there's a lot of .NET that kind of flew out or snuck up on you when you were developing Python, and really we don't want that for our Python developers you know we want, if you're a Python developer let us come and plug into your ecosystem, rather than trying to drive you to .NET. Extension bundles abstract away a lot of that stuff so if you're using a extension or a binding that's written in .NET, you no longer have to have the .NET SDK installed, all you need is Python if you're developing in Python. And we'll take care of a lot of that for you.

38:11 Michael Kennedy: Yeah kind of what you would expect but I guess it's nice that it's there, right?

38:14 Asavari Tayal: It's an evolution of the platform, I think from functions V1 to where you could only develop on Windows and host on Windows to V2 where you can now develop on any platform be it macOS, Linux machine and host on Linux, to finally being able to abstract the few fritters that were still left, I'm really excited and looking forward to a place that we're at right now, I think we're getting pretty close.

38:38 Michael Kennedy: Yeah that's cool. So speaking of environments, what versions of Python are supported?

38:44 Asavari Tayal: Today we support 3.6, Python 3.6 so we're looking to add 3.7 support in the future as well, we've already got a container image available if that's of interest.

38:52 Michael Kennedy: And I guess if you can do your own container then put whatever else you want out there right?

38:56 Asavari Tayal: Exactly, our core run time is actually already compatible with the 3.6 and 3.7, we heard feedback from specifically some of the ML users that there were packages like TensorFlow that weren't yet compatible with 3.7 or were still working on the wheels for 3.7 and such. So we just decided to delay that by a few months before we go ahead and make a big release.

39:19 Michael Kennedy: Sure okay, cool. One of the things that I saw people doing on AWS Lambda is using, they're abstracting away the deployment of something like Flask, right. So there's a project called Zappa, and Zappa will let you take what looks like a Flask function and turn it into a bunch of serverless functions. It's an interesting idea, I don't know, I saw that in the docs you guys have something kind of like that but it's much more manual than working directly with Flask, do you have any plans for something like that?

39:55 Asavari Tayal: Not yet, and honestly we're still thinking about it. We don't know if that's a direction that most users want to go in, just speaking to customers and we've got a very good community of preview users, they've given a lot of feedback. This hasn't cone up that much, so really I'd encourage you and everbody who's listening to give us feedback on that, tell us if that's an area or something that's important for you, and help us just understand how we should think about these things.

40:23 Michael Kennedy: Yeah honestly I look at that and it's super interesting, and it's kind of cool to think about but in practice I'm like do I really want to maintain my web app as 100 separate little things or does it make more sense to maybe build 10 specialized serverless bits and then host the rest of it as a regular web app or something I don't know, it's an interesting idea though.

40:44 Asavari Tayal: Right, right.

40:44 Michael Kennedy: Yeah yeah, cool. Well, I guess that's probably about it. Anything else you want to share about the service that you guys got going on right now?

40:53 Asavari Tayal: We've got few things in motion, I would say the biggest one definitely Python is something that is our biggest focus right now for the Azure functions team, so just keep your eyes and ears open for what we have coming next in terms of improvements for the preview itself and GA offering. That's really my biggest call to action here.

41:15 Michael Kennedy: Yeah cool, one of the things I thought was really nice is I went to the Azure serverless functions bit for Python, and it said "To subscribe to our announcements, just go watch The GitHub Repository." That's a pretty cool way to, instead of like yeah we'll going to shoot you a newsletter or something, nah just, it'll be in GitHub, subscribe to it.

41:33 Asavari Tayal: Yeah we definitely spend a lot of time and lives on GitHub these days. Open source is great, any time we want to speak to the community we pretty much post an announcement even something like "I'm not sure if you've heard of durable functions in Python, we're thinking about hey, does it makes sense to have durable functions for Python? Do Python users have scenarios that they need durable for?" Something I posted the announcement about a GitHug Azure bug, and there was a ton of interest. It's a great way to just go watch out repository, see what we're doing every day, and the questions we're asking and participate in the discussion.

42:07 Michael Kennedy: Yeah, it's cool. What do you mean by durable functions, like functions if they fail they'll restart, kind of like a transaction, what is this?

42:13 Asavari Tayal: No actually, so, going back to a little bit of principles of serverless per say, functions, serverless functions are typically supposed to be short lived and stateless. Right, but what we heard was there can be scenarios that are long running because maybe you are running some kind of an orchestration, maybe you are doing a function like a map reduce scenario, a fan in fan out scenario.

42:36 Michael Kennedy: I see, and that can be a problem because a lot of the server infrastructure, severless infrastructure will stop your function if it takes too long, right?

42:44 Asavari Tayal: Exactly in fact on our infrastructure it's, on our platform it's 10 minutes default timeout, but you can tweak it to be 20 minutes at the max. The new elastic premium plan actually can run forever, that's a premium benefit of the premium plan. But with durable functions what we do is we are basically providing a framework for you to be able to orchestrate your function executions. There can two concepts in durable, there is something called an orchestrator function, which can run forever. And orchestrator function basically calls into what are called activity functions, which are basically just regular functions and have the 20 minute timeout. So if I were doing a scenario like function chaining, I could have my orchestrator function kind of coordinate calling function A, taking the response from function A and using it to invoke B, similarly B to C and so on. So in both function chaining scenario or something where I was actually doing a fan in fan out would be more complicated, I could orchestrate various functions with the long running orchestrator function, or even the other day somebody built a scenario which required an OTP pin for instance, human interaction with a function which, we humans take time sometimes to respond, right? So those kind of scenarios are possible with durable functions, it kind of solves the problem of being able to maintain state and run for a longer duration.

44:02 Michael Kennedy: Does it serialize the state while it's waiting and bring it back, or?

44:07 Asavari Tayal: It basically does, as somebody on Twitter today just posted about the black magic of durable, so it's a little bit complex and I could go into the details but yes it's basically using storage to maintain its state.

44:16 Michael Kennedy: Okay cool that's exciting, that's a cool idea. All right well thanks for sharing all these ideas, there's just a couple of questions I'll ask you before I'll let you go. One is if you're going to write some Python code, what editor do you use?

44:29 Asavari Tayal: Yeah so good question, because I was traditionally using PyCharm and recently, when I say recently, since about the last year or so I started using VS Code. And I'm really appreciating right now the VS Code ability to be able to work with Azure because I work a lot with the Azure cloud, the extensions do a great job at it.

44:47 Michael Kennedy: Yeah that's cool, there's definitely a lot of integration there. Do you have plugins for PyCharm as well?

44:53 Asavari Tayal: I was using few plugins, I honestly don't even remember what plugins they were right now.

44:58 Michael Kennedy: Yeah sure it's been a while, all right. And then notable PyPI package that maybe you ran across and you're like "Aw that's cool, people should know about this."

45:05 Asavari Tayal: So yesterday I ran into something called Flask dance.

45:09 Michael Kennedy: Flask dance.

45:10 Asavari Tayal: And really what it helps you do is allow the OAuth dance that you would do manually typically, I mentioned this Flask application to you right, a lot of the OAuth dance that you would do typically yourself it makes it really really easily to be able to do that, and I was super thankful for that package yesterday when I had to do this in a short time.

45:29 Michael Kennedy: Yeah that's awesome, 'cause OAuth can be kind of annoying. You do this, and then it calls back to this function and then it's just, it gets to be...

45:35 Asavari Tayal: And of course every API provider does it it differently.

45:39 Michael Kennedy: Yeah, of course, yeah that's no fun. So, awesome, Flask dance, that's cool, I hadn't heard about that one. All right, final call action, people are excited about serverless, what should they do?

45:46 Asavari Tayal: I think you should go check out Azure Functions, we've got some great getting started material. Like I said if you're a CLI person, go ahead, download our CLI tool, start developing. Or you can also use VS Code. We are all eyes and ears right now for any feedback that you can give us, and given that we're out there in the open, all of our discussions, ideas, feature requests, you really have a great chance to influence the Azure product, I'd encourage you to come participate and talk to us, yeah.

46:15 Michael Kennedy: All right cool, well thank you so much for being on the show and sharing all of what you're up to here.

46:19 Asavari Tayal: Yeah, thank you so much for having me.

46:20 Michael Kennedy: Yeah, bye. This has been another episode of Talk Python To Me. Our guest on this episode was Asavari Tayal, and it's been brought to you by Command Line Heroes and Datadog. Command Line Heroes is a podcast telling the story of developers. This season is all about programming languages, and starts off with Python, of course. Subscribe at talkpython.fm/heroes. Datadog gives you visibility into the whole system running your code. Visit talkpython.fm/datadog and see what you've been missing. They'll even throw in a free t-shirt for doing the tutorial. Want to level up your Python? If you're just getting started, try my Python Jumpstart by Building 10 Apps course. Or, if you're looking for something more advanced, check out our new Async course, that digs into all the different types of ssync programming you can do in Python. And of course, if you're interested in more than one of these, be sure to check out our Everything Bundle. It's like a subscription that never expires. Be sure to subscribe to the show, open your favorite podcatcher and search for Python, we should be right at the top. You can also find the iTunens feed at /itunes, the Google Play feed at /play, and the direct RSS net /rss on talkpython.fm. This is your host, Michael Kennedy. Thanks so much for listening, I really appreciate it. Now get out there and write some Python code.