WEBVTT

00:00:00.001 --> 00:00:04.480
How do we host and run server software? Let's consider the progression we've been on over the

00:00:04.480 --> 00:00:10.180
past 15 years or so. We've gone from software and operating systems that we manage running on

00:00:10.180 --> 00:00:15.760
hardware that we own and babysit, to virtual machines, to infrastructure as a service on the

00:00:15.760 --> 00:00:21.840
cloud, or even platform as a service on the cloud. And then on from there, we've moved to containers,

00:00:21.840 --> 00:00:28.240
usually Docker, maybe running on someone else's cloud. After that, maybe we want to put these into

00:00:28.940 --> 00:00:33.060
microservices, which are conglomerates of these containers working together managed by something

00:00:33.060 --> 00:00:39.060
like Kubernetes. Well, where do we go from there? I can't tell you what the final destination of this

00:00:39.060 --> 00:00:44.060
whole progression is going to be, but I believe we have reached a leaf node in this hierarchy with

00:00:44.060 --> 00:00:50.940
our topic today. On this episode 118 of Talk Python To Me with Ryan Scott Brown, we're going to explore

00:00:50.940 --> 00:00:55.480
serverless computing. It's an interesting paradigm shift, and I hope you enjoy the conversation.

00:00:55.480 --> 00:00:59.020
It was recorded on May 24th, 2017.

00:01:23.020 --> 00:01:28.980
Welcome to Talk Python To Me, a weekly podcast on Python, the language, the libraries, the ecosystem,

00:01:28.980 --> 00:01:33.740
and the personalities. This is your host, Michael Kennedy. Follow me on Twitter, where I'm at

00:01:33.740 --> 00:01:39.280
mkennedy. Keep up with the show and listen to past episodes at talkpython.fm, and follow the show on

00:01:39.280 --> 00:01:45.560
Twitter via at Talk Python. This episode is brought to you by Rollbar and us at Talk Python Training.

00:01:45.560 --> 00:01:49.100
Be sure to check out what we're offering during our segments. It really helps support the show.

00:01:50.840 --> 00:01:52.080
Ryan, welcome to Talk Python.

00:01:52.080 --> 00:01:53.440
Hey, Michael. Glad to be on.

00:01:53.440 --> 00:01:59.900
Yeah, it's going to be super fun. We're here to talk about running code out on the internet with

00:01:59.900 --> 00:02:04.280
no servers, which seems kind of impossible and awesome all at the same time. That's going to

00:02:04.280 --> 00:02:04.660
be exciting.

00:02:04.660 --> 00:02:07.220
Yeah, I mean, there are still servers. We'll get around to that.

00:02:07.220 --> 00:02:12.080
Of course. Of course. Yeah, of course. Awesome. Before we do, though, let's hear your story. How'd you

00:02:12.080 --> 00:02:13.240
get into programming in Python?

00:02:13.500 --> 00:02:20.380
So I had kind of a pretty vanilla path. I actually started with my interest in networking. So I was

00:02:20.380 --> 00:02:27.140
majoring in systems administration and network admin and actually got brought into the programming

00:02:27.140 --> 00:02:34.140
world by a Perl class that I took. And then there was an open source club at my college that everyone

00:02:34.140 --> 00:02:40.280
was using Python to write programs for the One Laptop Per Child project. And that was my introduction to

00:02:40.280 --> 00:02:40.820
Python 2.

00:02:40.820 --> 00:02:43.320
Oh, that's really cool. What school is that?

00:02:43.320 --> 00:02:44.740
Rochester Institute of Technology.

00:02:44.740 --> 00:02:49.380
Oh, yeah. Okay. That's a great school. I know a couple of people that have gone there and it sounds

00:02:49.380 --> 00:02:50.680
like a pretty neat program.

00:02:50.680 --> 00:02:56.680
Yeah, they have actually an open source software minor now. They didn't when I went, but the program's

00:02:56.680 --> 00:02:58.540
kind of slowly evolved and grown over time.

00:02:58.540 --> 00:03:07.560
Oh, my gosh. I've never heard of an actual official university program that is focused on open source.

00:03:07.560 --> 00:03:09.760
Tell us just a little bit about that. Do you know anything about it?

00:03:09.880 --> 00:03:13.480
What does a person who takes that degree do or that minor do?

00:03:13.480 --> 00:03:18.620
It's not a full degree. So usually it'll be in software engineering or in computer science.

00:03:18.620 --> 00:03:24.880
In my case, I actually TA'd for some of the classes that then were added to the minor. So we had a bunch

00:03:24.880 --> 00:03:30.640
of pilot courses first, and then you get a certification from the university that you're a real minor, and then

00:03:30.640 --> 00:03:31.920
you can have real students enrolled.

00:03:31.920 --> 00:03:32.300
Awesome.

00:03:32.480 --> 00:03:38.360
All of the courses focused around either open source software or the idea of open communities

00:03:38.360 --> 00:03:43.000
in different ways. Like we had some Wikipedia editors that would come in and talk and be

00:03:43.000 --> 00:03:50.220
TAs. We had contributors to the BSD project, one of whom lives in Rochester and would teach

00:03:50.220 --> 00:03:55.780
one of our classes on open source software. And for course credit, you could do things like contribute

00:03:55.780 --> 00:03:57.360
to existing open source projects.

00:03:57.360 --> 00:04:00.980
All right. So that sounds like a really cool program that you were in in the university.

00:04:00.980 --> 00:04:02.240
Now, what do you do now?

00:04:02.240 --> 00:04:07.240
Yeah. So now I actually work for Red Hat, who obviously makes all of their money on open source

00:04:07.240 --> 00:04:13.020
software and support and consulting around that. And the team that I work on is the Ansible

00:04:13.020 --> 00:04:20.540
team. So we make a engine and a kind of language that people write in so that they can orchestrate

00:04:20.540 --> 00:04:26.840
all of their servers and cloud services and network devices and all that from a simple YAML

00:04:26.840 --> 00:04:30.820
language. So that's not a full fledged programming language. So you don't need the kind of experience

00:04:30.820 --> 00:04:35.700
you'd need to write things like chef recipes, which are Ruby. And so you'd have to know Ruby

00:04:35.700 --> 00:04:39.880
to write chef recipes. So we try and make that as accessible as possible.

00:04:39.880 --> 00:04:45.440
Yeah. I've heard really good things about Ansible and that sounds fun. And Ansible is a thing that

00:04:45.440 --> 00:04:48.960
recently came to Red Hat, right? Like recently acquired that project?

00:04:49.100 --> 00:04:55.360
Yep. About a year ago. Well, a little more than a year ago now, Red Hat acquired Ansible

00:04:55.360 --> 00:05:00.880
Works, which was the company that was behind Ansible, the open source project. And so now

00:05:00.880 --> 00:05:06.000
Red Hat sells Ansible Tower, which is a web interface that previously Ansible Works sold on top of

00:05:06.000 --> 00:05:11.860
Ansible. And then the Ansible core is still open source and just as kind of community focused as

00:05:11.860 --> 00:05:12.060
ever.

00:05:12.060 --> 00:05:15.460
Nice. Which side did you come from? Were you working on Ansible before or were you at Red Hat

00:05:15.460 --> 00:05:21.340
and then it came there? I was using Ansible, but I was working on OpenStack, which is a

00:05:21.340 --> 00:05:24.280
series of open source projects also all in Python.

00:05:24.280 --> 00:05:26.540
Yeah. That's quite a big Python project actually.

00:05:26.540 --> 00:05:30.060
Yeah. It's actually, I think 50 big Python projects.

00:05:30.060 --> 00:05:34.080
Yeah, exactly. It's an understatement to call it large, right?

00:05:34.080 --> 00:05:40.460
Huge contributors both to obviously the Python community in terms of bringing things back like

00:05:40.460 --> 00:05:47.140
PyPI and other things that become things in the regular Python community, as well as to

00:05:47.140 --> 00:05:50.180
the core language. There are a lot of core Python contributors that work on OpenStack.

00:05:50.180 --> 00:05:55.560
And so split their time between making Python better and then making things on Python better.

00:05:55.560 --> 00:05:58.420
Yeah, it's definitely a symbiotic relationship there. That's great.

00:05:58.420 --> 00:06:05.060
So let's talk about this idea of serverless programming. I feel like we started out with just

00:06:05.060 --> 00:06:11.860
having machines in data centers, maybe co-locating some of our machines in a data center to virtual

00:06:11.860 --> 00:06:19.380
machines, to things like Docker and things getting sort of smaller and smaller. And now we've reached,

00:06:19.380 --> 00:06:24.640
I'm not sure you can get much smaller than a single functions almost running on the internet,

00:06:24.640 --> 00:06:24.960
right?

00:06:24.960 --> 00:06:30.380
Yeah. I mean, that's sort of the direction that we're moving. And it's been a direction in programming

00:06:30.380 --> 00:06:36.740
for a very long time from when you had a system that just ran one job at a time, like on punch cards. So

00:06:36.740 --> 00:06:42.220
you toss in a stack of punch cards and it would churn through them and then churn out results also on

00:06:42.220 --> 00:06:48.700
punch cards. You didn't have multitasking at all. So one program was exactly what was running. So you

00:06:48.700 --> 00:06:53.800
were, you had the whole CPU to yourself and then you have multitasking operating systems. So you had

00:06:53.800 --> 00:06:57.840
multiple people providing code that would run in different processes that would be isolated in a certain

00:06:57.840 --> 00:07:03.720
way. And the operating system would handle scheduling those. And you can sort of think of

00:07:03.720 --> 00:07:11.720
function as a service, which is things like AWS Lambda and mostly called serverless, quote unquote. You can

00:07:11.720 --> 00:07:18.680
think of those as like a multi-user scheduling, but where you have untrusted entities sharing

00:07:18.680 --> 00:07:21.760
compute resources. So you're getting smaller and smaller slices.

00:07:21.760 --> 00:07:26.340
Right. Oh, so that's really interesting to track that trend over time, right? From just

00:07:26.340 --> 00:07:33.400
sort of OSs to, to running these functions on other people's computers. So obviously there

00:07:33.400 --> 00:07:39.840
really are servers out there, like you said, but when it's not your responsibility, it's not

00:07:39.840 --> 00:07:46.880
your problem to like babysit these servers. So for example, yesterday, I think I logged into

00:07:46.880 --> 00:07:54.640
some of my servers with SSH and I had to update like the unattended install software package on Linux.

00:07:54.640 --> 00:07:59.820
So even though of course your code runs on servers, when you don't have to think about it or manage it

00:07:59.820 --> 00:08:04.680
or balance it, it really makes a difference, right? Yeah, exactly. It's serverless in the same way that

00:08:04.680 --> 00:08:10.200
wireless doesn't have any wires involved. There are still wires. They just aren't directly to your device.

00:08:10.360 --> 00:08:16.560
Right. Well, and you know, in wireless, you never care about the wires. Somebody has to care about the

00:08:16.560 --> 00:08:21.160
wires, but it's opaque to you. Right. And so as far as you're concerned, wireless is wireless.

00:08:21.160 --> 00:08:27.160
Same analogy here. As far as you're concerned, you get particular promise from your provider. Like

00:08:27.160 --> 00:08:34.380
we will always have such and such version of such and such underlying dependencies like libc and things

00:08:34.380 --> 00:08:39.360
like that, that you would statically link to and a such and such version of a Python interpreter

00:08:39.360 --> 00:08:45.540
available and such and such version of whatever else you need. And you would write your code such that

00:08:45.540 --> 00:08:52.360
you can rely on that such and such version being there, but you don't install that. You don't deal

00:08:52.360 --> 00:08:58.480
with the sort of provisioning that you would do to install. Let's say you need a specific Python 3.

00:08:58.480 --> 00:09:03.800
So if you only can run on 3.5 or up, you don't have to worry about making sure that's there. You just tell

00:09:03.800 --> 00:09:08.400
your provider, give me this runtime, please. Right. And it just works and you have to worry about it. That's great.

00:09:08.920 --> 00:09:15.780
So can you compare this to like, say working with like event driven programming versus like hooking up

00:09:15.780 --> 00:09:20.100
to cloud events that, you know, something like Erlang? Yeah. And this is kind of the other side of

00:09:20.100 --> 00:09:26.640
serverless. The one side is run this code in the cloud for me. And the other side is hook my code

00:09:26.640 --> 00:09:33.780
up to these other things. So real world or virtual world, I guess, events that you want your code to

00:09:33.780 --> 00:09:39.380
active on. So let's say you're in your podcast world. You have something that you want to happen

00:09:39.380 --> 00:09:45.540
every time a sponsor contacts you. You could hook up a way that when the sponsor fills in the contact

00:09:45.540 --> 00:09:51.780
form, it invokes a Lambda function that then maybe calls you if it's super important to get back to

00:09:51.780 --> 00:09:58.660
them immediately or any number of other things. But the idea is the event source half and then the run

00:09:58.660 --> 00:10:02.560
my code somewhere that I don't have to manage half is what kind of makes up serverless.

00:10:02.820 --> 00:10:07.620
Right, right. It's easy to focus on here's a function run it, but really the diverse triggers

00:10:07.620 --> 00:10:13.840
that trigger those functions are super important. Yeah, exactly. And it maps to a lot of event driven

00:10:13.840 --> 00:10:18.960
concepts that we might be familiar with in Erlang where you have processes that are managed by a

00:10:18.960 --> 00:10:25.760
supervisor. So in serverless, that would be individual functions that are managed by your cloud provider,

00:10:25.760 --> 00:10:30.780
which under the hood deals with scheduling those across diverse hardware and all that stuff.

00:10:31.020 --> 00:10:37.820
And then in Erlang, you also have an event routing system where every process has input and output

00:10:37.820 --> 00:10:45.300
kind of event hooks. And you can get that similar thing in Lambda with using APIs to send data out.

00:10:45.300 --> 00:10:50.980
So you might send data out over like a MailChimp or a Mandrill, or you might send data out into a

00:10:50.980 --> 00:10:56.180
database. That would be your kind of output. And then your input can be triggers from things like

00:10:56.180 --> 00:11:01.580
DynamoDB. It can be HTTP events, any number of things. I'm not going to list them all because

00:11:01.580 --> 00:11:05.900
this would be a long podcast. Yeah, yeah, absolutely. And you know, to bring it back to like, what could I do?

00:11:05.900 --> 00:11:12.180
You know, you're saying something happens. Like maybe I've, I'm going to upload my MP3 file that I want to

00:11:12.180 --> 00:11:19.580
release for the week to S3. And that needs to be, you know, maybe have like the artwork and the little,

00:11:19.580 --> 00:11:26.640
like the description embedded into the MP3 header. And then it needs to be moved over to the content

00:11:26.640 --> 00:11:31.980
delivery. I got to like flip it to publish all these different things. Like I could possibly set that up

00:11:31.980 --> 00:11:37.360
serverless, you think? Yeah. And individually, you would have something like the S3 event source. So you would tell

00:11:37.360 --> 00:11:43.580
a Lambda function that when a new object appears in S3, and then S3 would notify your function when

00:11:43.580 --> 00:11:49.520
that happened, it could download from S3 your file. It would put in the, I forget what the name of the

00:11:49.520 --> 00:11:55.960
MP3 metadata format is. Yeah. It would dump in whatever author metadata, show notes. I don't know if that

00:11:55.960 --> 00:12:01.740
goes in there. Yeah. And then it could save it to another S3 bucket that would either kick off yet

00:12:01.740 --> 00:12:06.660
another Lambda and you'd have this chain, or that would be the S3 bucket you're serving public content from

00:12:06.660 --> 00:12:13.420
to your CDN. And it doesn't have to be linear either. You can have kind of forking things. So

00:12:13.420 --> 00:12:19.100
you could have this Lambda see, Oh, a new shows up here. I'll put the metadata in and I'll also kick off

00:12:19.100 --> 00:12:25.980
the job that will add it to the list on the front page. And I will kick off the job that will add it

00:12:25.980 --> 00:12:30.860
to the RSS feed. And those can be a bunch of Lambda functions that each have a small focused purpose.

00:12:30.860 --> 00:12:35.960
It sounds really amazing to think of how that kind of opens it up. And to me, it feels like almost like

00:12:35.960 --> 00:12:42.060
the event sources and the workflow bit is more important than, Hey, run this function.

00:12:42.060 --> 00:12:46.340
Yeah. Because I mean, when you think about it, we've been really good at running small bits of

00:12:46.340 --> 00:12:51.700
code for a very long time. And a lot of the value comes in when you can hook those small bits of code

00:12:51.700 --> 00:12:58.200
up to something that's a useful thing in the real world. Right. Cause I could make a little program

00:12:58.200 --> 00:13:03.280
that simulates a vending machine that will, you know, take pretend quarters and things, and I can type in on

00:13:03.280 --> 00:13:07.540
the terminal, but until I'm hooking that up to hardware, that's actually giving someone a soda

00:13:07.540 --> 00:13:15.320
or whatever, it's much less useful. Right. Right. Absolutely. Absolutely. So you're talking about

00:13:15.320 --> 00:13:21.780
S3 and I'm assuming that that probably ties right back into AWS Lambda, which is probably the most

00:13:21.780 --> 00:13:27.520
popular one of these, but it's not the only one. Like I think there's Azure functions, for example,

00:13:27.520 --> 00:13:32.200
what are, what are some of the implementations and places where we might find this type of

00:13:32.200 --> 00:13:36.580
programming model? There's a load of different providers. There's obviously the big three cloud

00:13:36.580 --> 00:13:41.980
providers all have their offering GC or Google cloud platform has what they call cloud functions.

00:13:41.980 --> 00:13:49.680
Microsoft Azure has what they call Azure functions. AWS has Lambda, which is a kind of a pawn on function,

00:13:49.900 --> 00:13:56.560
right. Nothing to do with, Python Lambdas, right? No. And I, yeah, that's been a problem

00:13:56.560 --> 00:14:01.980
sometimes cause you'll pull something into Python to automate. So if you want to automate the deployment

00:14:01.980 --> 00:14:06.880
of a Lambda function and you pull that into Python and then you name a variable Lambda, you can have

00:14:06.880 --> 00:14:11.780
problems if you're not careful. So why is this keyword gone? I don't know if that actually happens.

00:14:11.780 --> 00:14:16.980
There we go. It does. You can override keywords in Python. So you can override true, false, Lambda,

00:14:17.540 --> 00:14:21.920
all kinds of things that you should. Yeah. These are not good. And recently I can't speak to the

00:14:21.920 --> 00:14:27.580
other cloud providers. I just don't track them that closely, but AWS Lambda recently switched to

00:14:27.580 --> 00:14:33.060
Python. Well not switched, made available Python three six, which is pretty cool. Yes, that is

00:14:33.060 --> 00:14:38.660
absolutely right. And it has been great. Yes. Cause surprisingly until like a month ago,

00:14:38.660 --> 00:14:44.120
this was a Python two only option, right? Yeah, exactly. And I mean, if you think about when it came out,

00:14:44.120 --> 00:14:50.420
Lambda was out, I believe in towards the end of 2014 when, you know, Python three has been out for

00:14:50.420 --> 00:14:56.360
six years at that point. Yeah. That sounds about right. And I would love to be able to go run a

00:14:56.360 --> 00:15:01.460
second experiment and release it with Python three only and see what that does for adoption. Yeah. Yeah.

00:15:01.460 --> 00:15:04.880
Yeah. That would have been awesome. These would just be really interesting experiments to run,

00:15:04.880 --> 00:15:09.980
but for these other providers, there are some smaller providers that only do JavaScript. So,

00:15:09.980 --> 00:15:16.160
Auth0 is an example of one that has a function as a service, but it's JavaScript, Node JS only.

00:15:16.160 --> 00:15:16.680
Mm-hmm.

00:15:16.680 --> 00:15:22.420
If you look at IBM's OpenWhisk, you can actually provide an arbitrary Docker container as the runtime

00:15:22.420 --> 00:15:27.680
for your function. And so it manages the scheduling and it will invoke as many as needed to handle events.

00:15:27.680 --> 00:15:33.180
So it's still got the function as a service going for it, but you get a lot more control over what

00:15:33.180 --> 00:15:39.900
run times you give it. And so you can not only do Python three, but you can do entire Ruby installs.

00:15:39.900 --> 00:15:45.260
You can do Apple Swift, any language that you can fit in a Docker container, which is any language.

00:15:45.260 --> 00:15:48.600
Right. You could probably do something like Fortran if you really wanted to torture yourself,

00:15:48.600 --> 00:15:52.300
right? Like you could do basically if it can accept a request, you can do it, right?

00:15:52.300 --> 00:15:57.260
Yeah. I mean, as long as it accepts, I believe the rules for OpenWhisk is it has to accept data on

00:15:57.260 --> 00:16:02.100
standard in and provide a result on standard out when the container is invoked as a process.

00:16:02.100 --> 00:16:07.580
And so as long as you can do those things, you're good to go. So COBOL in the cloud.

00:16:07.580 --> 00:16:15.940
How interesting. Yay. Let's get more COBOL in the cloud. So one of the things that immediately comes

00:16:15.940 --> 00:16:23.700
to mind when I think of this and the Docker variant obviously would be one way to basically get an escape

00:16:23.700 --> 00:16:31.140
patch errors. Like how much do you have control over dependencies and requirements? So for example, what if I want to

00:16:31.140 --> 00:16:37.680
use like NumPy or requests or something that doesn't ship with Python? Yeah, sure. But I want to use that.

00:16:37.680 --> 00:16:43.620
Like, can I go to AWS Lambda and go, Oh yeah. And like, here's my requirements.txt make that happen.

00:16:43.620 --> 00:16:49.120
Unfortunately, you can't just upload a requirements.txt as much as I would love to. You can with a Google

00:16:49.120 --> 00:16:54.140
cloud functions, you can give them your code and a requirements.txt and they'll do it for you.

00:16:54.140 --> 00:16:59.820
That's pretty cool. In Lambda, you provide a zip file, which contains, can contain any dependencies

00:16:59.820 --> 00:17:05.240
you want, whether that's C object libraries, if you're using something that links directly to C like

00:17:05.240 --> 00:17:12.880
NumPy, or it can just be Python files like requests. So you would use pip on either your local machine or a

00:17:12.880 --> 00:17:18.660
build server that would build in all of your requests and pack them into a zip file. And I've

00:17:18.660 --> 00:17:24.340
had to do some backflips to get SciPy and SciKit learn working in Lambda. Wow. But you've, you've done

00:17:24.340 --> 00:17:30.240
it? No, it absolutely works. And the nice thing is that AWS now provides a Docker image that duplicates

00:17:30.240 --> 00:17:35.560
the Lambda environment. So you can download that Docker image, build stuff in there. So it's all going to be

00:17:35.560 --> 00:17:40.860
built just right. And then dump it to a zip file on your machine and then upload it to Lambda. So you can

00:17:40.860 --> 00:17:46.820
have exactly the same build environment locally for Lambda. And then you build all the SciKit learn

00:17:46.820 --> 00:17:51.820
and optimize it for that hardware or for that, that environment. And then you can strip out

00:17:51.820 --> 00:17:59.100
everything that isn't needed. So you can do things like compact the .o files. You can strip out a bunch

00:17:59.100 --> 00:18:05.620
of a TXT and MD and RST documentation files to get your size down because that relates to how fast your

00:18:05.620 --> 00:18:10.840
function can run. Yeah. Cause it's got to somehow take that thing apart and work with it. Right.

00:18:10.840 --> 00:18:14.360
I mean, it's got to unzip it, but it's also got to download it to whatever machines running your

00:18:14.360 --> 00:18:20.960
code. Cause they don't all have it. Right. Right. Absolutely. Does it get, like warmed up as it

00:18:20.960 --> 00:18:26.580
runs? Like how many machines might be involved in executing your code? No. And does it, if it's

00:18:26.580 --> 00:18:30.880
running a while? Yeah. It depends on your number of invocations. So the way that it works under the hood

00:18:30.880 --> 00:18:36.900
is they spin up basically a container that will handle events and each container will handle one event at a

00:18:36.900 --> 00:18:41.960
time. So let's say that you have five events per second and they all take half a second.

00:18:41.960 --> 00:18:47.380
Okay. Then on average, you're going to have about three hosts running or three containers handling

00:18:47.380 --> 00:18:51.460
requests because if they each take half a second, you've got two that are fully utilized and one

00:18:51.460 --> 00:18:55.920
that's half utilized. If that makes sense. Yeah. Yeah. That makes sense. And so if you have a bunch

00:18:55.920 --> 00:19:01.720
of events that come in in a very spiky kind of way, so you have 500 requests this second and then

00:19:01.720 --> 00:19:06.520
nothing for 10 minutes, it's going to spin up on a bunch of machines to handle these events and then

00:19:06.520 --> 00:19:11.700
they'll all spin down basically. And eventually they get evicted if they're not used. What's the

00:19:11.700 --> 00:19:16.340
payment model for this kind of stuff? How much difference does it make from say like, I'll just

00:19:16.340 --> 00:19:22.960
fire up a VM and just run my stuff there relatively. Yeah. That depends a lot on what your utilization

00:19:22.960 --> 00:19:28.560
level can be. So if you get, let's say you have a VM and you get one request a day, you're paying for

00:19:28.560 --> 00:19:35.520
24 hours of computer to handle 50 milliseconds or a hundred milliseconds or something. Yeah. And whereas

00:19:35.520 --> 00:19:43.540
in Lambda you pay by the unit, I believe is the megabyte second of memory. And that also scales your CPU.

00:19:43.540 --> 00:19:52.380
So they have tiers of anywhere from 128 megabytes of RAM up to, I believe a gig and a half. And each of those

00:19:52.380 --> 00:19:59.080
is build per hundred millisecond slice by the tier. Yeah. Let's suppose I've got a function that doesn't

00:19:59.080 --> 00:20:06.000
really use much memory, right? It's pretty basic and it's run five times a day for 10 milliseconds.

00:20:06.000 --> 00:20:14.060
Is this like a penny, a dollar, $5? Oh, that's in the hundreds of pennies. The way I had a blog post that

00:20:14.060 --> 00:20:19.940
I calculated out the cost to quote unquote more sensible unit, the, Pico dollar per bite second.

00:20:19.940 --> 00:20:30.020
Okay. And if you run, I believe it's, you get a million invocations for something like 14 or a million

00:20:30.020 --> 00:20:37.320
milliseconds for something like 1.4 cents. Wow. And so there's a lot of, I mean, you get,

00:20:37.400 --> 00:20:44.400
it's more expensive if you were to run a 24 hour EC2 instance and then run a 24 hour Lambda function,

00:20:44.400 --> 00:20:49.100
which you can't because there's a timeout, but if you ran them one after another for a total of 24 hours.

00:20:49.100 --> 00:20:53.800
Like if it was under a super heavy load, that was basically equivalent to a continuous, right?

00:20:53.800 --> 00:21:00.660
Yeah. If you were to run 24 Lambda hours, if you will. So if you run, you know, 24 of them all for one hour solid,

00:21:00.660 --> 00:21:06.200
then that would be the equivalent of a little more expensive than the equivalent EC2.

00:21:06.860 --> 00:21:12.200
But that would be assuming that you can fully utilize your EC2 instance, which is not all that

00:21:12.200 --> 00:21:17.960
common for most workloads because you've got kind of extra capacity to handle random spikes and

00:21:17.960 --> 00:21:22.520
variations in what users are doing. So if you're going to get a little bit more traffic, you don't

00:21:22.520 --> 00:21:26.680
want to spin up a brand new server every time you get just slightly over the threshold. Right.

00:21:26.680 --> 00:21:29.980
Yeah. Yeah. That makes a lot of sense. Whereas Lambda can match very precisely,

00:21:29.980 --> 00:21:33.100
you know, one-to-one with the events that you've got.

00:21:35.100 --> 00:21:38.760
Hey everyone, Michael here. Let me take just a moment and thank one of our sponsors who makes

00:21:38.760 --> 00:21:42.700
this show possible. This portion of Talk Python in May has been brought to you by Rollbar.

00:21:42.700 --> 00:21:46.200
One of the frustrating things about being a developer is dealing with errors,

00:21:46.200 --> 00:21:50.540
relying on users to report errors, digging through log files, trying to debug them,

00:21:50.540 --> 00:21:53.820
or a million alerts just flooding your inbox and ruining your day.

00:21:53.820 --> 00:21:59.440
With Rollbar's full stack air monitoring, you get the context, insight, and control you need to find

00:21:59.440 --> 00:22:06.520
and fix bugs faster. Adding the Rollbar Python SDK is just as easy as pip install Rollbar. You can start

00:22:06.520 --> 00:22:11.420
tracking production errors and deployments in eight minutes or less. Rollbar works with all the major

00:22:11.420 --> 00:22:16.440
languages and frameworks, including the Python ones like Django, Flask, Pyramid, as well as Ruby,

00:22:16.440 --> 00:22:23.020
.NET, Node, iOS, and Android. You can integrate Rollbar into your existing workflow, send error alerts to Slack

00:22:23.020 --> 00:22:28.460
or HipChat or automatically create new JIRA issues, pivotal tracker issues, and a lot more. They have a special

00:22:28.460 --> 00:22:34.720
offer for Talk Python To Me listeners. Visit talkpython.fm/Rollbar, sign up and get the bootstrap plan free

00:22:34.720 --> 00:22:41.040
for 90 days. That's 100,000 errors tracked for free. But you know, just between you and me, I hope you don't encounter

00:22:41.040 --> 00:22:46.500
that many errors. Give Rollbar a try today. Just go to talkpython.fm/Rollbar.

00:22:47.420 --> 00:22:52.640
There's all these different event sources. I understand how AWS can invoke our function.

00:22:52.640 --> 00:23:01.000
If like say an S3 thing changes, it's all like inside AWS, but you could wire up even like an API to this thing,

00:23:01.000 --> 00:23:05.900
right? Like if somebody hits this URL with a post with this JSON body, run this function, right?

00:23:05.900 --> 00:23:13.920
Yep. Just like you would put a handler in your Pyramid or Django application, you can attach a Lambda function

00:23:13.920 --> 00:23:22.720
to a, what's called in AWS an API gateway. In other serverless platforms, they call it, I believe, a HTTP event.

00:23:22.720 --> 00:23:28.760
But it's all the same. Basically, they take, they have something that's running all the time that's waiting for HTTP requests.

00:23:28.760 --> 00:23:34.300
And then when someone hits it, it will invoke your Lambda function, get the output, and then send it back.

00:23:34.300 --> 00:23:41.980
And you only pay when people are actually using it. So you can run HTTP API that's available that costs you pretty much

00:23:41.980 --> 00:23:44.540
nothing in base cost. It's just for requests.

00:23:44.540 --> 00:23:52.460
Yeah, that's pretty awesome. And can you do things like map, like SSL certificates and custom domains and stuff to those URLs?

00:23:52.460 --> 00:23:52.780
Do you know?

00:23:52.780 --> 00:23:56.520
Yep. Yep. You can do custom SSL certificates. You can do custom domain names.

00:23:56.520 --> 00:24:03.000
And in your function, you can figure out what domain name it was sent to. So you can do special things in your template, for example.

00:24:03.000 --> 00:24:07.740
Let's say you have a short domain and a long domain, but they both show the same site.

00:24:08.240 --> 00:24:14.400
If your Lambda function is rendering things or rendering links, it can give the correct URL back.

00:24:14.400 --> 00:24:15.000
Yeah, I see.

00:24:15.000 --> 00:24:19.600
By looking in the event and seeing all the information about where it was sent and all that stuff.

00:24:19.600 --> 00:24:20.840
Okay. That sounds great.

00:24:20.840 --> 00:24:25.040
And it has access to all the data sent to it, right?

00:24:25.040 --> 00:24:29.360
Like the URL, the query string, the post body, the headers. Is that possible?

00:24:29.360 --> 00:24:34.120
Yeah, absolutely. But there's kind of two ways you can do it in Amazon's API gateway.

00:24:34.120 --> 00:24:40.880
You can use what's called the Lambda proxy integration, which gives you this giant JSON event that has everything in it.

00:24:40.880 --> 00:24:51.360
So it'll have the forwarded by, the X forwarded for, all any other HTTP headers like auth and the query string and the path and everything.

00:24:51.520 --> 00:24:56.680
Or you can use subset language that AWS calls a velocity template language.

00:24:56.680 --> 00:25:00.440
And you can select very specific little bits.

00:25:00.440 --> 00:25:04.940
You shape the event that you get in your function to only the stuff that you need.

00:25:04.940 --> 00:25:07.960
And this means that your function runs faster because it's decoding less data.

00:25:08.640 --> 00:25:14.980
And there's less kind of a security attack surface area because you're only letting through one or two little things.

00:25:14.980 --> 00:25:18.240
And then that's happening before it gets to your code at all.

00:25:18.240 --> 00:25:19.280
Right. You don't want to.

00:25:19.280 --> 00:25:23.760
There were those mass assignment injection attacks, for example.

00:25:23.760 --> 00:25:32.120
Yeah. Or you could just have someone who sends a massive regex or something else that's hard, particularly expensive to decode.

00:25:32.660 --> 00:25:36.520
And make your Lambda function slower and, you know, cost you more money in the end, right?

00:25:36.520 --> 00:25:42.220
Right. Yeah. It's interesting to think that the more we use these cloud resources on a consumption-based model,

00:25:42.220 --> 00:25:48.460
how distributed denial of service has a direct monetary component.

00:25:48.460 --> 00:25:51.480
Yeah. I mean, it's more like a banking denial of service.

00:25:51.480 --> 00:25:52.600
Yes, exactly.

00:25:52.600 --> 00:25:58.760
So if I have, like, my VM running at, say, DigitalOcean or something, and somebody decides to attack it and pound on it,

00:25:58.760 --> 00:26:01.780
well, it may degrad or even kill my service.

00:26:01.920 --> 00:26:05.680
But I'm still going to pay the $10 a month or whatever I pay, you know what I mean?

00:26:05.680 --> 00:26:07.680
Whereas this, it could vary, right?

00:26:07.680 --> 00:26:09.660
It can vary, but there are limits.

00:26:09.660 --> 00:26:12.140
AWS puts in place two kinds of limits.

00:26:12.140 --> 00:26:18.640
They call them, the first ones they call safety limits, which are relatively low just so that you don't outbill,

00:26:18.640 --> 00:26:20.020
you don't bill yourself out, right?

00:26:20.020 --> 00:26:20.280
Right.

00:26:20.280 --> 00:26:24.220
And then they have what are called soft limits, which are limits that they say,

00:26:24.220 --> 00:26:30.040
okay, most of our users never hit this limit, but the ones that do can just call us and we'll raise it right up for you.

00:26:30.900 --> 00:26:37.840
And then they have hard limits of services that there's some technical limitation where they just can't go above, you know, 40 gig Ethernet, for example.

00:26:37.840 --> 00:26:38.280
Sure.

00:26:38.280 --> 00:26:39.140
Okay.

00:26:39.140 --> 00:26:39.760
That makes sense.

00:26:39.760 --> 00:26:44.860
And, of course, in AWS, I'm pretty sure the others have this as well, but you have, like, billing alerts.

00:26:44.860 --> 00:26:45.220
Yep.

00:26:45.220 --> 00:26:45.500
Right.

00:26:45.720 --> 00:26:47.060
You have billing alerts.

00:26:47.060 --> 00:26:50.000
You can also monitor specific things about Lambda.

00:26:50.000 --> 00:26:56.320
And that's another thing that's really nice in serverless is the provider needs to monitor really well to bill you correctly.

00:26:56.320 --> 00:27:02.680
And so you also happen to get really good monitoring because they need good monitoring to bill you.

00:27:02.680 --> 00:27:03.060
Right.

00:27:03.060 --> 00:27:04.300
And they just surface that for you.

00:27:04.300 --> 00:27:05.680
So that lines up real nice.

00:27:05.880 --> 00:27:06.080
Oh, yeah.

00:27:06.080 --> 00:27:06.500
That's cool.

00:27:06.500 --> 00:27:16.340
So maybe that's a good place to look, maybe compare and contrast with traditional web frameworks like Pyramid, Flask, Django compared to, like, this programming model.

00:27:16.340 --> 00:27:18.900
I mean, obviously, the way you set up the server is different.

00:27:18.900 --> 00:27:20.480
Like, you don't deal with N2NX and whatnot.

00:27:21.300 --> 00:27:24.180
But, you know, sort of the paradigms, what do you think?

00:27:24.180 --> 00:27:26.640
There's a lot of things that you don't get.

00:27:26.640 --> 00:27:35.560
So, for example, you can't just have some super long-running API request because, for example, Lambda has a five-minute timeout maximum.

00:27:35.560 --> 00:27:40.220
But usually you'll set that lower so that you don't bill yourself out or so you don't cost yourself too much money.

00:27:40.220 --> 00:27:46.000
Because, you know, if your average web request doesn't terminate in five seconds, the user's gone anyways.

00:27:46.000 --> 00:27:47.340
So you want to just stop that.

00:27:47.680 --> 00:27:52.340
And you also get a lot more control over what's shared and what's not.

00:27:52.340 --> 00:28:01.180
So in Django, a Flask, or a Pyramid, you have sort of a shared state that's internal to the server that isn't persisted out to a database, for example.

00:28:01.180 --> 00:28:01.620
Right.

00:28:01.620 --> 00:28:06.800
Maybe some in-memory static caching and stuff you pre-computed at start and you just can reference that, right?

00:28:06.800 --> 00:28:07.020
Yep.

00:28:07.020 --> 00:28:13.340
So you can build up pretty expensive caches locally that in Lambda don't make sense to do quite so much.

00:28:14.000 --> 00:28:22.920
And so what you would use for that is some other really fast storage system like DynamoDB or a Redis or even Elasticsearch.

00:28:22.920 --> 00:28:23.340
Sure.

00:28:23.340 --> 00:28:27.000
All of these things give you a really low latency way to just get data back.

00:28:27.000 --> 00:28:29.520
And then that would be accessed by your Lambda functions.

00:28:29.520 --> 00:28:31.640
And then you would be able to get stuff really quick.

00:28:32.080 --> 00:28:34.660
And so that's different in that you don't have the persistence.

00:28:34.660 --> 00:28:40.240
And then the other thing is that you don't have the same limitations on language boundary.

00:28:40.240 --> 00:28:42.280
And I know this is a Python podcast.

00:28:42.280 --> 00:28:52.500
So we'll flip around this example to imagine that if you're writing something in Express, which is a Node.js framework, and then you want to write a function Python because you like Python more.

00:28:52.740 --> 00:28:55.620
You're kind of SOL unless you make a new microservice.

00:28:55.620 --> 00:28:56.100
Right.

00:28:56.100 --> 00:29:00.360
Whereas in Lambda, you can go as granular as you want with languages.

00:29:00.360 --> 00:29:11.240
So I can say, oh, well, the user create endpoint is in Python 3.6, but our profile image generator only runs in 2.7 right now because I haven't gotten around to it.

00:29:11.340 --> 00:29:16.820
And so you can actually make a more granular migration between languages because you're doing one feature at a time.

00:29:16.820 --> 00:29:17.260
I see.

00:29:17.260 --> 00:29:24.280
You upgrade a function at a time and execute a function more or less in its own isolated environment, right?

00:29:24.280 --> 00:29:25.040
Yep, exactly.

00:29:25.040 --> 00:29:31.020
And then if you compare that to a Django project or something, then you've got one Python interpreter for your whole app.

00:29:31.020 --> 00:29:34.680
So everything either has to be Python 3 or nothing can use Python 3.

00:29:34.680 --> 00:29:35.080
Right.

00:29:35.080 --> 00:29:35.720
It's all or nothing.

00:29:35.720 --> 00:29:36.260
Yeah.

00:29:36.260 --> 00:29:36.740
Okay.

00:29:36.740 --> 00:29:37.660
That's pretty interesting.

00:29:37.660 --> 00:29:39.480
So the granularity is really cool.

00:29:39.480 --> 00:29:43.760
And then you also get the ability to make your dependencies separate.

00:29:43.760 --> 00:29:52.240
So if you have certain dependencies that you only want to run against very trusted data, you can make those only in the functions that are invokable by very trusted things.

00:29:52.240 --> 00:29:59.680
So you get a lot more security firewalling, not literal firewalls, but a lot more compartmentalization between functions.

00:29:59.680 --> 00:30:05.040
And you can even do AWS resource permission distinctions between functions.

00:30:05.040 --> 00:30:09.820
So you can do things like, say, this function's allowed to write to S3, but it's the only one.

00:30:09.820 --> 00:30:10.920
Everyone else is denied.

00:30:10.920 --> 00:30:11.340
Right.

00:30:11.340 --> 00:30:11.980
Okay.

00:30:11.980 --> 00:30:13.080
That's actually pretty awesome.

00:30:13.080 --> 00:30:19.960
Because, again, in like a traditional whiskey app, you have to put those walls up yourself.

00:30:19.960 --> 00:30:24.180
And it can be tricky because it's still the same memory in the end anyway, right?

00:30:24.180 --> 00:30:24.460
Yeah.

00:30:24.460 --> 00:30:28.260
I mean, regardless of how tricky it is, it's just, it's easy to make a mistake.

00:30:28.260 --> 00:30:28.620
Sure.

00:30:28.620 --> 00:30:35.160
Or to accidentally add an endpoint that probably shouldn't get right to S3 and then so on and so forth.

00:30:35.840 --> 00:30:40.780
Sounds like serverless code might be a little bit more safe by default.

00:30:40.780 --> 00:30:42.360
It's as safe as you make it.

00:30:42.360 --> 00:30:42.580
Yeah.

00:30:42.580 --> 00:30:49.740
You can just say, oh, I'm just going to give all my functions full admin and they're just going to execute arbitrary Python that comes in off the internet.

00:30:49.740 --> 00:30:52.020
You can just eval every request.

00:30:52.020 --> 00:30:54.080
Here, give me that pickled object.

00:30:54.080 --> 00:30:54.820
I'll work on that.

00:30:54.820 --> 00:30:55.220
No problem.

00:30:55.220 --> 00:30:55.540
Yeah.

00:30:55.540 --> 00:30:58.320
I also take arbitrary pickled objects and just, yeah, let's go.

00:30:58.320 --> 00:30:59.500
That'll be fun.

00:30:59.500 --> 00:31:00.360
Let's try that.

00:31:00.360 --> 00:31:01.960
But you can do a lot.

00:31:01.960 --> 00:31:03.620
There's a couple of really good talks.

00:31:03.620 --> 00:31:15.440
One is gone in 60 milliseconds as an example of how even in a serverless context, if you over permission your functions, attackers can still get things that they shouldn't from your Lambda functions.

00:31:15.440 --> 00:31:15.900
Of course.

00:31:15.900 --> 00:31:16.440
Yeah.

00:31:16.440 --> 00:31:20.400
It just sounds like it might be a little easier to exercise some least privilege.

00:31:20.400 --> 00:31:21.320
Yeah, definitely.

00:31:21.320 --> 00:31:22.360
Type of stuff here.

00:31:22.360 --> 00:31:22.920
Okay.

00:31:22.920 --> 00:31:33.600
So talking about the dependencies and persistence and caching and things like that, it sounds to me like to really, if we're going to have kind of complicated programs that are

00:31:33.600 --> 00:31:38.440
running in this serverless architecture, you kind of need to go a little more all in on the cloud providers.

00:31:38.440 --> 00:31:43.700
Like, let's just stick to AWS because we've been talking about Lambda, but, right, this applies generally.

00:31:43.700 --> 00:31:48.180
So AWS has DynamoDB and some kind of caching.

00:31:48.180 --> 00:31:49.080
I'm guessing Redis.

00:31:49.080 --> 00:31:51.100
I haven't played with their Redis option.

00:31:51.100 --> 00:31:54.140
Yeah, they offer Redis or Memcached as a service.

00:31:54.140 --> 00:31:55.200
Yeah, exactly.

00:31:55.360 --> 00:31:56.740
So there's that.

00:31:56.740 --> 00:31:57.760
There's RDS.

00:31:57.760 --> 00:32:00.820
Storage would go to S3 instead of the file system.

00:32:00.820 --> 00:32:01.820
Right.

00:32:01.820 --> 00:32:07.460
So do you feel like to be effective with this stuff, you kind of have to go a little more into the various APIs?

00:32:07.460 --> 00:32:10.980
Whereas I could use EC2 and, like, basically forget I'm on AWS.

00:32:11.280 --> 00:32:12.380
Yeah, yeah, you could.

00:32:12.380 --> 00:32:27.400
The downside of doing EC2 and you forget that you're on AWS is that you've, you know, forgotten Amazon's, you know, zillion man years or a zillion developer years that have gone into creating all these higher level services that are basically commodities.

00:32:28.060 --> 00:32:32.660
So things like S3, oh, store this blob for me and then let me get it later.

00:32:32.660 --> 00:32:37.240
When you're running just on EC2 and you're storing it to disk, you need a backup strategy.

00:32:37.240 --> 00:32:40.780
You need to make sure that if that server goes down, it's still available.

00:32:40.780 --> 00:32:41.940
So you have to do the replication.

00:32:41.940 --> 00:32:47.120
And S3 is just one example, but you have all these services that make your life easier.

00:32:47.120 --> 00:32:49.880
And so you do have a trade going on.

00:32:49.880 --> 00:32:58.100
So you can choose to use as few provider-specific services as possible, but then you don't get the benefits of using those.

00:32:58.100 --> 00:33:09.740
So the example that I like to use is there's an online training company called A Cloud Guru that they built their prototype over about a week on Lambda and using Firebase.

00:33:09.740 --> 00:33:10.380
Okay.

00:33:10.380 --> 00:33:13.300
And the downside is that they would be locked into that forever.

00:33:13.300 --> 00:33:16.840
The upside is that if they couldn't have done that, they couldn't have started.

00:33:16.840 --> 00:33:18.960
And so they wouldn't even exist, right?

00:33:18.960 --> 00:33:19.200
Right.

00:33:19.400 --> 00:33:30.000
And so every time you get up, you're making a trade between going and killing your own hand-raised, going out and growing your own food versus getting it from someone else where you're locked into that provider somewhat.

00:33:30.000 --> 00:33:37.920
So you're making a trade-off between what you're able to do in a short amount of time versus how easy it would be to switch to another provider, basically.

00:33:37.920 --> 00:33:38.360
Right.

00:33:38.360 --> 00:33:38.720
Of course.

00:33:38.720 --> 00:33:41.280
And it doesn't necessarily mean you have to stick with serverless, right?

00:33:41.280 --> 00:33:48.440
Like you could go and use RDS and Redis and S3 and then switch to EC2 and still use those, right?

00:33:48.920 --> 00:33:52.240
You're just kind of just stuck to AWS at that point, but not to Lambda.

00:33:52.240 --> 00:33:57.300
Yeah, because a lot of the services that you'll use alongside Lambda, you would use from a traditional application.

00:33:57.300 --> 00:34:07.600
So you'll see people that write web applications and then some of the functionality is in Lambda because they didn't want to deal with something like a resource overrun on that particular item.

00:34:08.040 --> 00:34:12.280
Or they wanted a special event source that wasn't an HTTP type event.

00:34:12.280 --> 00:34:17.020
Or they just liked that context better because it's a language they don't normally work with.

00:34:17.020 --> 00:34:20.440
There's all kinds of reasons that you would have kind of a hybrid.

00:34:20.440 --> 00:34:21.060
Right.

00:34:21.060 --> 00:34:21.600
Of course.

00:34:21.600 --> 00:34:23.860
I have a lot of faith in these cloud providers.

00:34:24.160 --> 00:34:26.400
Like they very rarely go down.

00:34:27.480 --> 00:34:30.100
And, you know, when they do, it's usually really, really short.

00:34:30.100 --> 00:34:34.720
But what if you wanted to have some flexibility to say move?

00:34:34.720 --> 00:34:37.460
Like maybe could you speak to lock in a little bit?

00:34:37.460 --> 00:34:45.820
Everywhere you're going to go, you're going to have some lock in, whether that's just the time it would take to move your data or the time it would take to write your app to make changes.

00:34:45.920 --> 00:34:54.740
An example of not very lock in lock in would be something like file systems on Linux because you can switch between X4 and ZFS.

00:34:54.740 --> 00:34:59.360
And that's, as far as your code is concerned, the same because they both provide the same interface.

00:34:59.360 --> 00:35:15.060
Whereas when you migrate from, like you've talked about on previous podcasts, from Postgres to MongoDB, you've got to do a change to your code to deal with the different modeling that MongoDB does of your data and queries versus what Postgres does.

00:35:15.060 --> 00:35:15.740
Right, exactly.

00:35:15.740 --> 00:35:20.220
Yeah, that took a couple days of work and a few bugs you had to hunt down, right?

00:35:20.220 --> 00:35:21.540
Yeah, that was different, of course.

00:35:21.540 --> 00:35:22.280
Yeah, but you made it.

00:35:22.280 --> 00:35:24.420
So, and serverless is the same way.

00:35:24.420 --> 00:35:35.920
You're going to be embedded in whatever cloud provider you're in because in most contexts, you kind of want to be embedded like that because then you take advantage of their work so that you don't have to.

00:35:35.920 --> 00:35:36.360
Right.

00:35:36.360 --> 00:35:42.960
But if you're worried about transitioning cloud providers, there are a few things that you can do, like never handling an event directly.

00:35:43.240 --> 00:35:46.060
Is you always transcode it into a sensible format.

00:35:46.060 --> 00:35:54.540
So instead of relying on the Lambda proxy event format, you have something that transcodes that into just the stuff that you need in a format that makes sense to you.

00:35:54.540 --> 00:36:00.000
So that way, at least, you just have to rewrite these shims and then your internal code makes sense.

00:36:00.000 --> 00:36:01.220
Sure, that makes a lot of sense.

00:36:01.220 --> 00:36:02.680
Or your internal code still works.

00:36:02.680 --> 00:36:03.320
Right, right.

00:36:03.340 --> 00:36:08.400
When you rewrite the shim and to handle the, say, the Google format of the event or the Amazon format of the event.

00:36:08.400 --> 00:36:12.600
Yeah, I guess, you know, probably some proper architecture makes a lot of sense here.

00:36:12.600 --> 00:36:24.980
Like, for example, your primary code could directly write to S3 or it could call some other code that says save this file to my thing, wherever that is.

00:36:24.980 --> 00:36:30.080
And then that could be implemented to do it to S3, it could be implemented to Azure Blob Storage, whatever, right?

00:36:30.080 --> 00:36:35.200
Azure actually has a really cool implementation of this that I like a lot that I hope that more providers will copy.

00:36:35.200 --> 00:36:38.360
You make a trigger, they call it a trigger, an event source.

00:36:38.820 --> 00:36:44.220
And you can have Azure Blob Store be the trigger, or you can have something like Dropbox be the trigger.

00:36:44.220 --> 00:36:54.020
And what it'll do is it will pull down that file, put it in a temporary directory where your function is running, and then invoke your function and tell it about the local file path.

00:36:54.020 --> 00:36:54.700
Oh, nice.

00:36:54.700 --> 00:36:59.600
So you don't actually deal with, like, the Dropbox or Azure Blob Storage APIs to get files.

00:36:59.600 --> 00:37:04.560
It puts them in a local directory for you, and then you use your language's regular file support.

00:37:04.720 --> 00:37:08.940
So events like that would be amazing for more providers to implement.

00:37:08.940 --> 00:37:10.920
And I think they've done a great job there.

00:37:10.920 --> 00:37:13.540
Yeah, yeah, that actually sounds really quite interesting.

00:37:13.540 --> 00:37:23.440
Another thing that I guess I see is a bit of a challenge for serverless is what if I want to work on this locally, right?

00:37:23.440 --> 00:37:30.820
Like, I want to just fire up my local CPython and run this code and see what happens, you know?

00:37:30.820 --> 00:37:42.680
Like, maybe I'm on a plane, and I want to, you know, do a little bit of work before I get to this conference for something I'm doing or some customer demo or something, right?

00:37:42.680 --> 00:37:44.780
And it involves this app.

00:37:44.780 --> 00:37:48.660
Like, what's the story on local or offline or any of these things?

00:37:48.660 --> 00:37:49.660
You've got some options.

00:37:49.660 --> 00:37:59.020
For local, there's the kind of what I would call the first-degree local, which is you can develop locally and deploy to a dev environment that just takes your code directly.

00:37:59.020 --> 00:38:02.660
So that's kind of first-degree local, but the code's actually not running on your machine at all.

00:38:02.660 --> 00:38:06.540
If you want more than that so that you can have a faster feedback loop, because deploys do take time.

00:38:06.540 --> 00:38:07.860
Not a lot of time, but time.

00:38:08.300 --> 00:38:18.280
You can use something like Atlassian has a thing called local stack that will fake out API Gateway, Kinesis, Dynamo, S3, and a bunch of other services on your local machine.

00:38:18.280 --> 00:38:19.540
So they'll be on local ports.

00:38:19.540 --> 00:38:26.480
And then you can run code that would use AWS services and point them at your local, local stack.

00:38:26.480 --> 00:38:26.940
Nice.

00:38:26.940 --> 00:38:31.720
Yeah, maybe just change, like have a different, like a dev local config that has different endpoints.

00:38:31.720 --> 00:38:36.680
It would be a config change that would point your code at those local services.

00:38:36.680 --> 00:38:38.680
So you can then develop locally, like on a plane.

00:38:38.680 --> 00:38:42.260
So once you have local stack downloaded, you would be able to do that.

00:38:42.260 --> 00:38:48.360
Other thing is plugin for the serverless framework, which I'm a contributor to the serverless framework, but not to this plugin.

00:38:48.360 --> 00:38:58.300
That it runs Docker locally and will use that container image that I talked about and simulate different event invocations to your function.

00:38:58.600 --> 00:39:08.100
And so you can either use that with, they have a built-in thing for the DynamoDB developer mode, which is a little jar that you run that puts up a fake DynamoDB.

00:39:08.100 --> 00:39:15.180
Or you could use that in conjunction with local stack to get the runtime and the service simulation together.

00:39:15.180 --> 00:39:15.740
Yeah, I see.

00:39:15.740 --> 00:39:24.600
Yeah, I wonder how much Atlassian came up with local stack to like solve their own local cloud testing problems and just open source it.

00:39:24.600 --> 00:39:25.160
Do you know?

00:39:25.160 --> 00:39:31.400
I don't actually know the people that made local stack, but it definitely seems like the kind of thing that someone Atlassian size would say.

00:39:31.400 --> 00:39:38.860
We have so and so many developers and it would save us so and so much time to not deploy every time they want to test something.

00:39:38.860 --> 00:39:46.460
So let's take that time that we're going to save and invest in this tool that's going to, you know, make our developers a lot happier down the line.

00:39:46.460 --> 00:39:53.040
Yeah, that actually made me think of continuous integration and testing and verification and stuff like that.

00:39:53.040 --> 00:39:58.540
Like, how would I basically verify my stuff in a CI style?

00:39:58.780 --> 00:40:03.800
Yeah, so there's some things that are hard, like simulating actual chains of lambdas.

00:40:03.800 --> 00:40:10.720
So let's say you have that podcast lambda that we used as an example earlier where it takes an MP3 file, it does stuff, it puts it back in S3.

00:40:10.720 --> 00:40:14.760
And then the next function picks up the new file and does something else and so on.

00:40:15.120 --> 00:40:24.100
It's hard to simulate that chaining, but because each function is pretty focused, using a regular testing framework to test, given this input, what happens?

00:40:24.100 --> 00:40:29.820
And there are several libraries that will pretend to be AWS for you in Python unit tests.

00:40:30.440 --> 00:40:40.960
I really like placebo, which you can run it and it will record your interactions with AWS, the interactions of your code, and save them all in order.

00:40:40.960 --> 00:40:43.880
And then you can rerun it in playback mode.

00:40:43.880 --> 00:40:49.260
And so it will insert itself before your calls go to AWS and just send back the recorded response.

00:40:49.260 --> 00:40:57.040
So you can make sure that things are called a certain number of times and that your code handles these responses from AWS the right way.

00:40:57.040 --> 00:40:57.920
Okay, that sounds cool.

00:40:57.920 --> 00:41:15.640
You'd use unit level testing for that and then you'd have an integration level test where you deploy to a staging environment and then you have scripts that exercise kind of the full life cycle with all of those AWS services that you can't simulate locally or want to make sure that your local simulation isn't different from the current AWS behavior.

00:41:15.640 --> 00:41:15.980
Right.

00:41:15.980 --> 00:41:20.420
It sounds like mocking might be an important part as well for certain parts.

00:41:20.580 --> 00:41:28.980
Actually, with placebo, you don't have to do the mocking because it injects itself into the AWS client library and sort of does that mocking under the hood for you, which is pretty cool.

00:41:28.980 --> 00:41:29.400
I see.

00:41:29.400 --> 00:41:30.100
Yeah.

00:41:30.100 --> 00:41:31.160
It's involved.

00:41:31.160 --> 00:41:32.140
You just don't have to write it.

00:41:32.140 --> 00:41:33.540
Yeah, it's involved, but you don't write it.

00:41:33.540 --> 00:41:33.780
Nice.

00:41:33.780 --> 00:41:34.920
I guess that's mockless now.

00:41:34.920 --> 00:41:36.160
Yeah, it's mockless.

00:41:36.160 --> 00:41:37.240
Mockless unit testing.

00:41:38.660 --> 00:41:41.340
This portion of Talk Python To Me is brought to you by us.

00:41:41.340 --> 00:41:47.240
As many of you know, I have a growing set of courses to help you go from Python beginner to novice to Python expert.

00:41:47.240 --> 00:41:49.280
And there are many more courses in the works.

00:41:49.280 --> 00:41:53.520
So please consider Talk Python training for you and your team's training needs.

00:41:53.520 --> 00:41:59.960
If you're just getting started, I've built a course to teach you Python the way professional developers learn by building applications.

00:42:00.660 --> 00:42:05.040
Check out my Python jumpstart by building 10 apps at talkpython.fm/course.

00:42:05.040 --> 00:42:08.160
Are you looking to start adding services to your app?

00:42:08.160 --> 00:42:11.320
Try my brand new consuming HTTP services in Python.

00:42:11.320 --> 00:42:16.760
You'll learn to work with RESTful HTTP services as well as SOAP, JSON, and XML data formats.

00:42:16.760 --> 00:42:18.620
Do you want to launch an online business?

00:42:18.620 --> 00:42:22.820
Well, Matt Makai and I built an entrepreneur's playbook with Python for Entrepreneurs.

00:42:22.820 --> 00:42:28.060
This 16-hour course will teach you everything you need to launch your web-based business with Python.

00:42:28.580 --> 00:42:31.600
And finally, there's a couple of new course announcements coming really soon.

00:42:31.600 --> 00:42:36.980
So if you don't already have an account, be sure to create one at training.talkpython.fm to get notified.

00:42:36.980 --> 00:42:40.820
And for all of you who have bought my courses, thank you so much.

00:42:40.820 --> 00:42:42.780
It really, really helps support the show.

00:42:42.780 --> 00:42:48.840
We talked about developing locally, but there's also some other tools that just help with things like deployment

00:42:48.840 --> 00:42:56.220
and something called Zappa, which basically as soon as AWS Lambda switched to Python 3,

00:42:56.220 --> 00:42:57.860
Zappa's like, hey, we're Python 3.

00:42:58.500 --> 00:43:02.080
Because it's like running more or less on top of Lambda.

00:43:02.080 --> 00:43:03.180
And there's some others as well.

00:43:03.180 --> 00:43:03.980
Do you want to talk about those?

00:43:03.980 --> 00:43:04.300
Yeah.

00:43:04.300 --> 00:43:09.200
So Zappa is a project that will take your WSGI-ish app.

00:43:09.200 --> 00:43:15.560
So it'll take things like API star, now that it supports Python 3, Flask, Django, Pyramid,

00:43:15.780 --> 00:43:23.020
and wrap them up in its own kind of fake WSGI that takes the Lambda API gateway events

00:43:23.020 --> 00:43:30.800
and will put them into the request object for that web framework and then give it to your function that would work as a Django app,

00:43:30.800 --> 00:43:32.840
but is now inside of Lambda, but it doesn't know.

00:43:32.840 --> 00:43:33.260
Right.

00:43:33.260 --> 00:43:36.300
So you write code as if it were Django or something.

00:43:36.300 --> 00:43:42.960
So you pretend that it's Django and then Zappa handles packaging up each endpoint and associating it with API gateway

00:43:42.960 --> 00:43:47.260
and then uploading that code to Lambda and hooking it up.

00:43:47.300 --> 00:43:50.160
And then you get your Django-ish endpoint.

00:43:50.160 --> 00:43:55.280
Or, well, it is a Django endpoint, but now running without Django actually serving the connection.

00:43:55.280 --> 00:43:55.780
Interesting.

00:43:55.780 --> 00:44:01.200
Yeah, because you basically, when you start these apps normally, you say, here's my WSGI app.

00:44:01.760 --> 00:44:06.660
And that's the implementation of WSGI is like, you've received a request, right?

00:44:06.660 --> 00:44:09.400
So they just have to adapt to, hey, you've received a request.

00:44:09.400 --> 00:44:13.480
It just happened to have come not from a WSGI server, but from somewhere else, right?

00:44:13.480 --> 00:44:14.520
Yeah, that's exactly right.

00:44:14.520 --> 00:44:19.300
Because you're getting the same sort of data about the request in a different format from API gateway.

00:44:19.300 --> 00:44:25.780
And so what Zappa does is it makes that format match the format that WSGI expects and then gives it off to your code.

00:44:25.780 --> 00:44:26.440
Okay, cool.

00:44:26.440 --> 00:44:28.520
And that'll do deployment as well.

00:44:28.520 --> 00:44:31.380
So you say, Zappa deploy, and it'll deploy it up for you.

00:44:31.580 --> 00:44:36.360
And it will also let you do some local testing and things of that nature.

00:44:36.360 --> 00:44:44.240
Yeah, I wonder if that actually makes it more locally testable because you could just run it as a Django app or...

00:44:44.240 --> 00:44:46.520
I think you're still really bound to those services.

00:44:46.520 --> 00:44:49.240
So I don't know how much that really helps you.

00:44:49.240 --> 00:44:50.020
Yeah, I guess you're right.

00:44:50.020 --> 00:44:52.820
Because it's really that you're trying to get to that S3 bucket.

00:44:52.820 --> 00:44:55.500
You're trying to get to that RDS instance and so on, right?

00:44:55.500 --> 00:44:55.800
Yeah.

00:44:55.800 --> 00:45:01.560
And for example, you might have a thing that you're calling that's only available inside of your VPC.

00:45:01.560 --> 00:45:04.580
which is a virtual private cloud, which is basically a private little subnet.

00:45:04.580 --> 00:45:04.860
Yeah.

00:45:04.940 --> 00:45:07.400
And if you're testing locally, you're just not going to have it.

00:45:07.400 --> 00:45:07.620
Yeah.

00:45:07.620 --> 00:45:09.540
But Zappa is really cool.

00:45:09.720 --> 00:45:13.820
Chalice is another one that's made by Amazon as sort of a labs project.

00:45:13.820 --> 00:45:15.340
I think it's still point something.

00:45:15.340 --> 00:45:16.560
So zero point something.

00:45:16.560 --> 00:45:25.420
But it looks a lot like writing Flask, but it uses those decorators that you put on to auto-discover how it should connect your code to API Gateway.

00:45:25.940 --> 00:45:33.100
And it has a thing that will try and auto-discover what IAM permissions you need, which is, in my experience, a little hit or miss.

00:45:33.100 --> 00:45:33.500
Okay.

00:45:33.500 --> 00:45:35.080
And then there's also something called Gordon.

00:45:35.080 --> 00:45:35.640
Yeah.

00:45:35.640 --> 00:45:37.500
Gordon is a Python frame.

00:45:37.500 --> 00:45:42.920
It's both written in Python and you can deploy Python with it, but it's just for deployment.

00:45:43.040 --> 00:45:48.240
So you'd write your code like Gordon doesn't exist and then you use Gordon to deploy your code.

00:45:48.240 --> 00:45:49.080
Okay.

00:45:49.080 --> 00:45:57.980
So things like Gordon, Ansible, the serverless framework you use to specify all these resources around your function, but you write your function sort of independent of them.

00:45:58.380 --> 00:46:01.580
So they don't want kind of anything to do with your internal code.

00:46:01.580 --> 00:46:02.660
They just want to deploy it.

00:46:02.660 --> 00:46:03.000
Interesting.

00:46:03.000 --> 00:46:03.400
Okay.

00:46:03.400 --> 00:46:06.480
So let's talk about where people are using this.

00:46:06.480 --> 00:46:09.400
What are some popular deployments?

00:46:09.400 --> 00:46:15.920
We already talked about a cloud guru, which is that online training place for cloud stuff.

00:46:15.920 --> 00:46:16.200
Yep.

00:46:16.200 --> 00:46:18.500
And then you also probably know of iRobot.

00:46:18.500 --> 00:46:21.120
They make the Roomba and they have a new series of Roomba.

00:46:21.120 --> 00:46:22.260
I think it's the nine something.

00:46:22.260 --> 00:46:23.460
Right.

00:46:23.460 --> 00:46:28.940
And that has an associated app and it will map your home over time as it sort of figures out where things are.

00:46:28.940 --> 00:46:32.220
And that's actually all backed with Lambda and API Gateway.

00:46:32.220 --> 00:46:33.280
Oh, that's crazy.

00:46:33.280 --> 00:46:35.740
So is it like you hook it on your Wi-Fi?

00:46:35.740 --> 00:46:36.820
Yeah.

00:46:36.820 --> 00:46:37.320
Yeah.

00:46:37.320 --> 00:46:37.980
It has network.

00:46:37.980 --> 00:46:40.580
It just streams its location and info.

00:46:40.580 --> 00:46:42.000
I don't think it streams the location.

00:46:42.000 --> 00:46:43.880
You'd have to talk to them about that.

00:46:43.880 --> 00:46:46.520
They're also huge Python users, so they might be a good guest.

00:46:46.520 --> 00:46:47.180
Oh, yeah.

00:46:47.180 --> 00:46:48.040
That sounds fun, actually.

00:46:48.040 --> 00:46:52.780
The Roomba maps kind of locations in your home and eventually it'll figure out like, well, this is always here.

00:46:52.960 --> 00:46:55.340
And this chair is only sometimes here, so it must be mobile.

00:46:55.340 --> 00:46:58.800
And so it can figure out the most efficient way to vacuum your house.

00:46:58.800 --> 00:47:01.280
But you can also do things like kick it off remotely.

00:47:01.280 --> 00:47:08.420
So if you're at work and you're like, oh, no, I have guests that are going to be in for dinner and I'm at work, run the Roomba so I don't look like a slob.

00:47:08.420 --> 00:47:16.560
And all of that's going through the iRobot API, both on the Roomba side, because they need to be able to tell the Roomba to go.

00:47:16.560 --> 00:47:20.200
And then on your app side, because your app needs to tell them, I want this to happen.

00:47:20.680 --> 00:47:23.200
And all of that's going through API Gateway with Lambda.

00:47:23.200 --> 00:47:23.460
Yeah.

00:47:23.460 --> 00:47:27.720
It seems like these IoT things would be a really good fit for serverless.

00:47:27.720 --> 00:47:29.540
They just need to talk back.

00:47:29.540 --> 00:47:30.200
Yeah.

00:47:30.200 --> 00:47:32.960
Because you don't know what the usage profile is going to be.

00:47:32.960 --> 00:47:39.700
And you're really cost sensitive because people pay you however many dollars for the device and then they expect it to just keep working.

00:47:39.700 --> 00:47:40.280
Yeah, that's cool.

00:47:40.280 --> 00:47:41.920
Another one is Nordstrom, right?

00:47:42.080 --> 00:47:50.600
They've been speakers at serverless conf several times and they use Lambda API Gateway and Kinesis, which is an event streaming service.

00:47:50.600 --> 00:47:58.800
So you can put as many events as you want on Kinesis and it'll sort of keep them in order to a certain extent and invoke Lambdas for batches of them.

00:47:58.800 --> 00:48:03.400
And they have a project called Hello Retail that I'll make sure gets in the show notes.

00:48:03.400 --> 00:48:05.500
But is a really nice.

00:48:05.500 --> 00:48:08.680
I want to see how an architecture of a real thing works.

00:48:09.120 --> 00:48:14.900
So for them, it's a simple retail platform that's an example of all these services kind of together in one place.

00:48:14.900 --> 00:48:16.220
Okay, great.

00:48:16.220 --> 00:48:20.240
So I guess, you know, all this sounds really good.

00:48:20.240 --> 00:48:26.240
There's a few cases where serverless makes things a little harder, but a lot of places where it makes it much easier.

00:48:26.240 --> 00:48:31.740
When would you say that we should maybe, so we talked about a lot of places to use it.

00:48:31.740 --> 00:48:33.960
When would you say you maybe should not use it?

00:48:33.960 --> 00:48:38.380
Like if I'm trying to do this thing, like it's probably not a good use case for this model.

00:48:38.380 --> 00:48:39.060
What is that?

00:48:39.060 --> 00:48:39.620
There's a few.

00:48:39.620 --> 00:48:43.940
There's things that you have right now, things that have a low latency sensitivity.

00:48:43.940 --> 00:48:48.820
So if you absolutely positively need a response in X milliseconds.

00:48:48.820 --> 00:48:49.480
Right.

00:48:49.480 --> 00:48:51.760
So like high frequency trading might not make sense.

00:48:51.760 --> 00:48:53.500
And this is like single digit milliseconds.

00:48:53.500 --> 00:48:53.920
Yeah.

00:48:53.920 --> 00:48:56.240
So you're not going to high frequency trade on Lambda.

00:48:56.240 --> 00:49:05.180
But even if you have something like an ad marketplace, Lambda is probably not the best because usually your responses are sitting around.

00:49:05.500 --> 00:49:11.600
I believe it's 150 milliseconds to go through API gateway and back and then however long your Lambda function wants to run.

00:49:12.160 --> 00:49:16.400
So if you need 50 millisecond latency, then I think you're going to have problems.

00:49:16.400 --> 00:49:16.840
I see.

00:49:16.840 --> 00:49:22.080
So there's kind of an assumed like 150 millisecond latency just in the whole system.

00:49:22.080 --> 00:49:22.340
Yeah.

00:49:22.340 --> 00:49:22.600
Yeah.

00:49:22.600 --> 00:49:27.280
Just because you're invoking a new container and it's starting and it's loading your code into memory and doing all this stuff.

00:49:27.600 --> 00:49:32.320
And subsequent invocations are faster because the code's already in memory and it's just waiting on more events.

00:49:32.320 --> 00:49:39.500
But you have kind of a base time that you're going to be spending just communicating over the network between API gateway and Lambda.

00:49:39.500 --> 00:49:46.340
Whereas if you're going from client directly to your Django app, there's no like API gateway in between.

00:49:46.880 --> 00:49:50.320
And so you are adding a hop even though it's a hop inside AWS.

00:49:50.320 --> 00:49:51.140
So it's pretty quick.

00:49:51.140 --> 00:49:53.140
The other thing is for WebSockets.

00:49:53.140 --> 00:50:02.140
There's not a great Lambda WebSocket story yet, but you can go over to a place like Firebase and they have awesome WebSocket support and they hook into Google Cloud Functions.

00:50:02.140 --> 00:50:04.580
So it depends on your provider too.

00:50:04.580 --> 00:50:05.120
I see.

00:50:05.120 --> 00:50:05.660
Okay.

00:50:05.660 --> 00:50:07.700
Well, that sounds like good advice.

00:50:07.700 --> 00:50:09.340
Firebase definitely worth checking out.

00:50:09.580 --> 00:50:09.780
Yeah.

00:50:09.780 --> 00:50:17.460
And they have really good Google Cloud Functions integrations now that were, I think some of them were just announced at Google Next a month or two ago.

00:50:17.460 --> 00:50:18.320
Yeah.

00:50:18.320 --> 00:50:18.620
Excellent.

00:50:18.620 --> 00:50:21.960
So this is probably a pretty good place to wrap things up.

00:50:21.960 --> 00:50:27.360
One thing I did want to give you a chance to talk about is you just wrote some video courses on serverless programming, right?

00:50:27.360 --> 00:50:28.080
Yeah, that's right.

00:50:28.080 --> 00:50:38.240
I have two, one from a little while ago that's just on AWS Lambda and function as a service and how to write code for kind of this new architecture and runtime and all that stuff.

00:50:39.080 --> 00:50:49.540
And the most recent one I have is using the serverless framework, which is one of these deployment options, one of these tools to deploy GraphQL APIs, which is a query language.

00:50:49.540 --> 00:50:50.500
It's developed at Facebook.

00:50:50.500 --> 00:50:53.520
I think you've had a show on it unless it was a different Python podcast.

00:50:53.520 --> 00:50:56.580
Yeah, I think it was on a podcast in it actually, but that's, yeah.

00:50:56.580 --> 00:50:57.740
So people can check that out.

00:50:57.740 --> 00:50:58.860
That sounds interesting.

00:50:58.860 --> 00:50:59.220
Yeah.

00:50:59.220 --> 00:51:08.800
So that's the serverless framework to deploy a GraphQL API that you would consume from a web front end, like a single page app, maybe using React, or you could use it from a mobile app.

00:51:08.800 --> 00:51:10.260
Anything that can speak GraphQL.

00:51:11.120 --> 00:51:20.580
And the idea is that you can then write your own backend with a lot less experience than you needed before because now you're not managing like a load balancer and auto scaling groups and all that stuff.

00:51:20.580 --> 00:51:26.660
You just deploy your function that talks to your data store that then gives you access to all your data from the client.

00:51:26.660 --> 00:51:27.620
That sounds really cool.

00:51:27.620 --> 00:51:32.740
So those are both at a Cloud Guru and we can link to that in the show notes so people can check that out.

00:51:33.580 --> 00:51:35.260
All right, Ryan, this is really interesting stuff.

00:51:35.260 --> 00:51:36.680
Let's close it out with the two questions.

00:51:36.680 --> 00:51:39.820
If you're going to write some Python code, what editor do you open up?

00:51:39.820 --> 00:51:41.160
That would be Vim, of course.

00:51:41.160 --> 00:51:42.480
All right, cool.

00:51:43.140 --> 00:51:49.720
And most notable but not really super popular PyPI package you want to draw people's attention to?

00:51:49.720 --> 00:51:56.180
I don't know if it's super popular, but I really love StructLog, which is a library for structured logging.

00:51:56.400 --> 00:52:04.900
So you import it and you put it into your standard library logging configuration and it intercepts stuff and it can take keyword arguments.

00:52:04.900 --> 00:52:11.800
So instead of logging out a formatted line with a bunch of percent S or the curly braces, you log out the name of your event.

00:52:11.800 --> 00:52:16.380
So that can just be a short message and then as many key values as you want as keyword args.

00:52:16.540 --> 00:52:20.620
And then it'll log them as either JSON or a prettified kind of CLI thing.

00:52:20.620 --> 00:52:31.260
And so you can inject a lot more data into your logs and then you can get a lot more out of them if you're parsing them through like the elastic stack for log parsing or using CloudWatch or anything like that.

00:52:31.260 --> 00:52:37.440
So in Lambda, you can also use this and it will get parsed as JSON by CloudWatch, which is pretty cool.

00:52:37.440 --> 00:52:37.940
Yeah, yeah.

00:52:37.940 --> 00:52:39.600
That sounds like a really cool addition.

00:52:39.600 --> 00:52:40.240
All right.

00:52:40.240 --> 00:52:41.200
Final call to action.

00:52:41.200 --> 00:52:42.540
How do people get started with this stuff?

00:52:42.540 --> 00:52:46.280
I have a blog about this that has some Python material as well as Node.js.

00:52:46.420 --> 00:52:50.960
So whatever people are into at serverlesscode.com, they can just kind of go through there.

00:52:50.960 --> 00:52:57.760
I have tutorials, projects, interviews, kind of a mix or hit up those video courses if you're the kind of person that learns from video.

00:52:57.760 --> 00:52:58.280
All right.

00:52:58.280 --> 00:52:59.480
Very cool.

00:52:59.480 --> 00:53:01.760
Thanks for sharing all this serverless stuff with us, Ryan.

00:53:01.760 --> 00:53:02.480
It was very interesting.

00:53:02.480 --> 00:53:02.840
Yeah.

00:53:02.840 --> 00:53:03.360
Thanks, Mike.

00:53:03.360 --> 00:53:03.740
My pleasure.

00:53:03.740 --> 00:53:04.260
Yep.

00:53:04.260 --> 00:53:04.560
You bet.

00:53:04.560 --> 00:53:04.800
Bye.

00:53:04.800 --> 00:53:09.320
This has been another episode of Talk Python To Me.

00:53:09.320 --> 00:53:16.140
Today's guest was Ryan Scott Brown, and this episode has been sponsored by Rollbar and Talk Python Training.

00:53:17.040 --> 00:53:18.980
Rollbar takes the pain out of errors.

00:53:18.980 --> 00:53:26.700
They give you the context and insight you need to quickly locate and fix errors that might have gone unnoticed until your users complain, of course.

00:53:26.700 --> 00:53:33.840
As Talk Python To Me listeners, track a ridiculous number of errors for free at rollbar.com slash Talk Python To Me.

00:53:34.840 --> 00:53:36.860
Are you or your colleagues trying to learn Python?

00:53:36.860 --> 00:53:39.920
Well, be sure to visit training.talkpython.fm.

00:53:39.920 --> 00:53:45.720
We now have year-long course bundles and a couple of new classes released just this week.

00:53:45.720 --> 00:53:46.700
Have a look around.

00:53:46.700 --> 00:53:48.160
I'm sure you'll find a class you'll enjoy.

00:53:48.920 --> 00:53:50.580
Be sure to subscribe to the show.

00:53:50.580 --> 00:53:52.780
Open your favorite podcatcher and search for Python.

00:53:52.780 --> 00:53:54.020
We should be right at the top.

00:53:54.020 --> 00:54:03.320
You can also find the iTunes feed at /itunes, Google Play feed at /play, and direct RSS feed at /rss on talkpython.fm.

00:54:03.800 --> 00:54:08.420
Our theme music is Developers, Developers, Developers by Corey Smith, who goes by Smix.

00:54:08.420 --> 00:54:15.120
Corey just recently started selling his tracks on iTunes, so I recommend you check it out at talkpython.fm/music.

00:54:15.120 --> 00:54:20.460
You can browse his tracks he has for sale on iTunes and listen to the full-length version of the theme song.

00:54:21.300 --> 00:54:22.540
This is your host, Michael Kennedy.

00:54:22.540 --> 00:54:23.840
Thanks so much for listening.

00:54:23.840 --> 00:54:25.020
I really appreciate it.

00:54:25.020 --> 00:54:27.160
Smix, let's get out of here.

00:54:27.160 --> 00:54:27.160
Smix, let's get out of here.

00:54:27.160 --> 00:54:48.620
I'll see you next time.

00:54:48.620 --> 00:54:49.120
Bye.

00:54:49.120 --> 00:54:49.360
you

00:54:49.360 --> 00:54:51.420
you