Do you have a big monolithic web application or service that's hard to manage, hard to change,
and hard to scale? Well, maybe breaking them into microservices would give you many more
options to evolve and grow that app. This week, we'll meet up again with Miguel Grinberg
to discuss the trade-offs and advantages of microservices. It's Talk Python to Me,
episode 121, recorded June 2, 2017.
Welcome to Talk Python to Me, a weekly podcast on Python, the language, the libraries, the
ecosystem, and the personalities. This is your host, Michael Kennedy. Follow me on Twitter,
where I'm @mkennedy. Keep up with the show and listen to past episodes at talkpython.fm,
and follow the show on Twitter via at Talk Python.
Talk Python to Me is partially supported by our training courses. Here's an unexpected question
for you. Are you a C# or .NET developer getting into Python? Do you work at a company
that used to be a Microsoft shop, but is now finding their way over to the Python space?
We built a Python course tailor-made for you and your team. It's called Python for the .NET
developer. This 10-hour course takes all the features of C# and .NET that you think
you couldn't live without. Unity Framework, Lambda Expressions, ASP.NET, and so on, and it
teaches you the Python equivalent for each and every one of those. This is definitely the fastest
and clearest path from C# to Python. Learn more at talkpython.fm.net. That's talkpython.fm.dotnet.
Miguel, welcome back to Talk Python. Thank you. Thank you for having me. It's great to have you back. Previously, we talked about
building Flask web applications and web services, and that was really fun. And I think we're going to take it up a notch in terms of sort of abstraction and talk about a more general idea, which obviously is frequently done in Flask, but could be done in Pyramid, could be done in Django, or, you know, Node.js, right? Microservices.
Yes. So microservices are really interesting, and there's a couple of ideas vying for this, like how do we decompose our big applications, and we'll dig into that. But first, I just want to let people know, if they want to hear your previous episode, that was episode 48, and maybe we could quickly talk about PyCon. We were just both there, right? You have a good time there?
Yeah, yeah. I had a lot of fun. It always surprises me that people recognize me, and they stop me, and they say, you know, thank you, your tutorials, your blog, what got me into programming, or helped me advance. So I always end up with a smile.
It's really amazing how many people I was around who were like, wow, people actually really appreciate what I do.
Right. Yeah, yeah. Likewise. I mean, I just do it because I enjoy it. I mean, to be honest, I do it for myself.
Yeah. But it's really great to see that you make a difference, like some of the other podcasters at our podcaster booths were like, wow, people really appreciate what we do. I like, you know, you do this work often, really, very much in isolation.
I don't mean it in a bad way, but like, you sit down to write your book, or you write your series of blog posts, or you record a course, or even the podcast is kind of, you know, just two people, right? And then you go to one of these events, you're like, wow, there's a lot of people who this actually affects how cool it is that I'm doing this.
Yeah, that's something that I really enjoy.
Yeah. And you did a tutorial, right? On microservices, right?
I did a tutorial. So these are the three hour long tutorials that happened before the main conference. So yeah, this year, this is the fourth consecutive year that I do a tutorial. And this one was on microservices.
Yeah, excellent. So we'll definitely have to put that into the show notes.
Sure. Yeah, it's been on YouTube, like the day after I gave it, they put it, they're very efficient.
It's amazing. They really seem to have the AV work down at PyCon these days.
They're very good. Yes.
I think it's also part of their outreach mission, I would guess. Like there's 3,300 people that came to PyCon, but there's, you know, already, I think some of those videos have more people watching it online than actually attended the conference.
Right. Yeah.
Yeah. So what were some of your favorite sessions, like some talks or experiences?
Always dictionaries. Those are always interesting. For example, there were a few this year. So, and I always end up learning something new that I didn't know.
It's amazing, right?
Yeah. So, and I always end up learning something new that I've been doing.
I always end up learning something new that I've been doing.
Yeah, I always end up learning something new that I've been doing.
I always end up learning something new that I've been doing.
You know, some people know you, some people don't.
What do you do day to day?
Where are you coming from?
What's your perspective?
In the last few years, I have been working.
Well, actually, for a very long time, I've been working as a software engineer.
In the last few years, I've been building cloud applications.
Right now, I'm working for Rackspace.
And I am helping build some of the services that our customers use when they go to the Rackspace
control panel.
More specifically, the services that I work on are ones that allow customers to deploy
applications that then we manage.
They deploy applications very easily by clicking and dragging stuff.
Basically, we do all the magic.
And we, of course, we use microservices for all of this.
Yeah, that's really cool.
And you have a book, right?
That we talked about just a little bit in the previous podcast.
Right.
And then I have a book.
The book is called Flask Web Development.
It's now, I think, a little bit over three years old.
And I'm currently working on the second edition.
So, probably later this year, hopefully before Christmas, we'll see.
The second edition will be out.
That's going to basically refresh.
The books largely are going to be the same.
It's going to update a few things that changed in Flask or some of the extensions and related projects that I referenced.
Yeah, progress is great.
But when you create things like books or video courses, it's really frustrating when they change, actually.
Yeah.
I mean, it's really amazing, to be honest, that after three years or a little bit more, large parts of the book are still updated.
Flask, thanks God, is not a framework that likes to change things in big ways.
Yeah, that's right.
It was a pretty mature framework when you got to it.
If you did like Jopranto or Sanic right now, you might be rewriting that thing in a year.
Right.
Yeah.
Actually, I do have one of my open source projects.
I have support for Sanic and AIo HTTP, as well as Flask and Django and many others.
It's a socket IO server.
It's called Python socket IO.
And I find that AIo, HTTP and Sanic require more attention than the old friends from the whiskey world.
Yeah, absolutely.
Yeah, but it's good.
Those things are changing.
Those things are growing.
Those are the frameworks that are pushing the web forward in the Python space.
Yes, absolutely.
So it makes living around their orbit more work, but I think it's going to make it all better for everyone in the end.
Yeah.
All right.
So let's start with what are microservices?
Like, if I want to just take and create like a keep it on Flask since that's where your book is, if I want to just create like a Flask app, I could just put everything in there, right?
I could do my user management, I could do my data access, I could do my reporting, all that stuff.
I could just stick into like one big Flask app and ship that, right?
Correct.
And I covered many times, it's in the book and in tutorials that I've done in previous years.
You know, with Flask, you have a way to organize your application when it starts to become large.
It gives people some trouble.
There are sometimes issues with secular dependencies, but you can do it and you can end up with a single application.
In the context of microservices, we call these types of applications monoliths because they're one big thing.
Yeah, yeah.
So maybe compare and contrast with what are microservices relative?
I can draw a parallel.
So we all know that if you write your application in a single big function, that's really not good, right?
It's hard to maintain.
Yeah, you should at least use two functions.
Two functions.
You should use two or three, right.
So basically what you do when you're talking about functions is you write small functions.
Each function, at least you should try that it does one thing.
And then the functions call each other and that's how you achieve the big problem, right?
The solution to the big problem, right?
Right.
And functions, the way I think of it is if I can't give it just a simple name that says what it does, it's wrong.
There's something wrong with it.
I need to change.
I need to change the function so I can name it better, right?
Correct.
Right.
So microservices, it's basically the same idea, but applied to a web service.
So the traditional way in which you develop a web application in Python, say using Flask, Bottle or Django or anything, pyramid.
Basically, like you said before, you put all the contents in one application.
And then without realizing it, you have a coupling between the different subsystems, right?
You have a user subsystem that keeps track of your users.
And then you have many others.
And, you know, they all use the same database models.
And you don't realize it, but you are basically making it harder for that application to grow and maintain because of all the these references that one subsystem has into the other.
So solution that microservices bring is that you take all these conceptually separate subsystems and you create a separate web service with each one.
Right. So maybe you've got like a front end web app that still does the back end server side stuff.
But instead of going straight to the database or straight to some sub modules in your web app, it calls these related microservices that sort of implement the functionality, right?
Correct. Right. So that gives you a number of advantages, some disadvantages, too.
But the clear advantage is that each service is going to be very simple.
We're going back to, you know, you know, very small code bases for each service.
For example, with Flask, you can easily write a an entire microservice in a single file.
Right. So give us some examples of microservices that would be reasonable to create.
Like, would there be a logging microservice?
Would there be an authentication or like how would you how have you decomposed this before?
So the example that I used at the the Python tutorial was a chat application.
So the chat application exists as a monolith.
And I showed in class how to break it into microservices.
And basically there were five microservices.
So one was users. That's the name. Basically registered users.
The second one was tokens.
Basically took care of generating authentication tokens for the client side app.
There was a third microservices was messages.
This is the one that adds and basically stores messages when a user types a message.
The fourth was the UI application.
So basically there was a very simple service that served the JavaScript and CSS and HTML files to the client.
Super simple.
And then the final one was the one that did the server push.
So this was based on WebSocket using my socket IO server.
And anytime there was a change, either a new user or a user leaving the chat or a new message, the service knew how to push those updates to all the clients.
So five microservices are all completely independent applications in this case written in Flask.
Okay. That makes a lot of sense.
It definitely adds some complexity, right?
So you're no longer maintaining the configuration for one app, but you're maintaining five or four and then the interplay between them, right?
Correct.
So the complexity, I did mention this in the class, the complexity doesn't go away.
Basically, you're shifting the complexity to a different place.
And now we have an ops.
Yeah.
It's more complicated to deploy an application that's based on five.
And this was a relatively small app, right?
Normally you may have dozens or maybe even hundreds of microservices.
So, yeah, definitely the complexity goes somewhere else.
What I find that I like to shift the complexity into those places because I'm a software developer, right?
So from my point of view, I really like clear code that's easy to maintain.
For example, something that I see done with microservices is if you have a team where you have a beginner, right?
Usually if you have a big, complex application, you're going to be afraid that this person that doesn't have a lot of experience may inadvertently break something.
Right.
And they could break it entirely, right?
Correct.
It could be...
You know, unknowingly, right?
It's because of all this coupling that, you know, from over time keeps increasing in these types of applications.
Right.
And the slightest little problem in like even a trivial part of the app, if it makes it fail to start, like you've taken the entire site down for everyone, for everything, right?
Right.
It's gone for everybody.
Right.
So with microservices, however, you can have a beginner work on one microservice, even own it.
And if there are any problems with that microservice, that's not going to affect the overall application, right?
All the other microservices will continue to run.
So this is in general, not only when a beginner makes a mistake, but in general, if one microservice is sick, it goes down or has problems, that doesn't mean that the whole application goes down.
It's just that system.
And many times if you kill that microservice and start a new instance, then you're back up running and you have more time to fix the problem.
Yeah, that's really an interesting way to think about it.
And you could probably even just force a rollback to the previous deploy and run that.
And that could be super hard to do in your regular application because maybe the UI has changed, maybe the database schema in some little part has changed, and SQLAlchemy freaks out or whatever, right?
Right.
Yeah.
Databases are one of the big reasons why deployments for monolithic applications, it's so hard, right?
Once you migrate the database.
I mean, yes, migration frameworks have downgrades, but very few people use them.
And even those that use them, many times they don't test them, so they're usually broken.
Yeah.
And they could be remove this column which had data in it.
Correct.
Right.
So, yes, the idea with microservices in particular to databases is that each microservice has its own database.
So, if you migrate one database for the messages service, that has nothing to do with the users.
So, it's a much smaller problem if you end up having problems.
Yeah, that's really cool.
There's a ton of advantages to that.
I like the way, gosh, who was it?
Martin Fowler was referring to these databases as the ones from the monoliths and bigger ones.
He called those integration databases.
Right.
And these called application databases.
I'm not sure if that's quite the right term, but I really like to think of it as like, you can take this one big complex database that's trying to represent everything from every part of the app or multiple apps.
So, the user's table is as complicated as it could possibly be, right?
Right.
The order history table is as complicated as it could be because it has to support every single possible option.
But if you break it into these little microservices, you know, you could have a really simple like, here's the microservice that handles orders.
It has a database that handles orders.
Right.
It's just that.
Correct.
Now, there's a problem with that.
You lose the ability to run joins because now you don't have everything in one database.
Right.
So, if you need to correlate users or customers with orders, you have to do it in the application.
Yeah, exactly.
Like you can't join across ACP requests.
Correct.
That doesn't work.
Not really.
You have to do it in the Python space in our case.
Yeah.
I don't find that terrible.
That's my first observation.
My second observation is that even though people that know me know that I'm a big fan of relational databases, when you're working with microservices and your databases are usually one table or two tables, you know, the reason to use relational databases sort of lessens.
And now it's starting to make more sense to go to a NoSQL database.
Yeah.
Especially a document database.
Yes.
The one thing that you get kind of contains the pre-joined data as a hierarchy anyway.
Yeah, that's really interesting.
So, I can definitely see how that makes like rolling back one of these services if it gets sick much, much easier.
And the chance that it gets sick is smaller as well, right?
Because it's simpler.
There's a lot less chances of making mistakes because you're working with a much simpler code base.
How about scalability?
Well, right.
So, if you have a big monolithic app, you need to scale it.
You need to scale the whole thing.
Maybe going back to the chat example, you're probably going to have a lot more activity around messages than around users, even less on tokens.
So, if you were to scale a monolith, you're going to be basically, you're going to have to provision, you know, for the entire thing.
Right.
You have to work.
You have to aim for the worst case scenario.
Correct.
Basically, across any part of it, right?
So, if you're going to have to do a lot of messages from users, you need to run 10 instances.
You're going to have to provision 10 instances for everything, right?
Because it's all one piece.
Now, when you are doing microservices, you can scale each service independently.
That's really, really cool.
So, it's super exciting.
You can scale.
I mean, if you use something like Kubernetes, for example, you can scale across different hosts.
If you have a cluster of container hosts, automatically does it for you.
So, you can have not only scalability, but reliability by having your instances of the same service distributed across multiple hosts.
Yeah, that's really, really neat to think that, okay, I might have two or three of the straight-up web front-ins, maybe five of the orders, servers, you know, three of the message senders, and just to be able to configure those independently is really cool.
And then dynamically, as well, right?
Yeah.
I mean, the concept of auto-scaling also applies to this.
So, you know, the messages can, you know, or orders or whatever, anything that's very active, you can decide, okay, I'm going to start one more.
And some other components we haven't discussed yet help with that dynamicity.
Sure.
One of the things that was striking about the Instagram keynote, which is a really cool story of moving Django from Python 2 to Python 3, while you have millions and millions of users, and doing it on a single branch without going down, was super interesting.
But one of the things they were really obsessed about was how can we get basically being very aggressive with how they work with memory so they can get the best memory usage out of each server that they work with.
Like, for example, they went so far as disabling garbage collection in their Python apps, period.
Which is just crazy.
That sounds scary.
Yeah, they have a really interesting blog post they wrote up about it that they were able to get much better memory sharing across processes if they did that, like dramatically better.
It made it...
Right.
It probably makes for a cleaner use of memory, right?
Memory is not coming and going.
Exactly.
And apparently the cycles that were leaked were not sufficiently bad that there was a...
Surprisingly, it worked.
So the point is they're really, really focused on this.
And when you scale the monolith over and over and over, maybe it takes 200 megs per worker process.
Right.
Yeah.
If you want 10 of them, that's a gig.
But you could get these other ones much, much smaller and only scale the parts that are really hot, right?
Correct.
It's also a big savings, right?
And if you need to buy hosting for 100 instances of a monolith, that's going to be very expensive.
That's going to be a lot of cloud instances.
Yeah.
Now, if you're using microservices, you're scaling up, you know, very little things and only the ones that you need.
Yeah.
So you have a lot more knobs and you end up saving a lot of money.
Yeah.
And this was not the case with Instagram because they were already in this monolith space.
But had they been in microservices, they could have done their migration from Python 2 to Python 3 and Django 1.3 to modern.
They could have done that microservice by microservice.
And it probably would have been dramatically easier.
One at a time.
And if, so that's one, one of the other benefits you get.
Let's say, I don't know if this is true, probably not, but let's say one of the services that in this example, if they had microservices, if one of them was, could not be upgraded to Python 3 due to some late time dependency that hasn't been upgraded.
That's not a problem.
You can keep that one running Python 2.
It doesn't matter.
So you're not constrained to use the same platform in all your services.
If you find that some service can be benefited if you write it in Go or in Ruby or in Node.js, that's totally fine.
You can pick the best tool for each service.
Yeah, that's really cool that you can break it up.
And it also means, like say the data level, right?
Like you talked about relational versus NoSQL, like you could do MySQL on some pieces and MongoDB on others.
Absolutely.
And you don't have to say, well, this part's going to have to fit into Mongo or that part's going to have to fit into MySQL when it would more naturally live somewhere else.
Yeah.
So basically you can pick the best tools for each service and each service is completely independent from the others.
But basically you are encouraged to keep this coupling that's always bad under control by having these hard boundaries between services.
Hey everyone, this is Michael.
Let me tell you about Datadog.
They're sponsoring this episode.
Performance and bottlenecks don't exist just in your application code.
Modern applications are systems built upon systems.
And Datadog lets you view the system as a whole.
Let's say you have a Python web app running Flask.
It's built upon MongoDB and hosted and scaled out on a set of Ubuntu servers running Nginx and Microwisgi.
At Datadog and you can view and monitor and even get alerts across all of these systems.
Datadog has a great getting started tutorial that takes just a few moments.
And if you complete it, they'll send you a sweet Datadog t-shirt for free.
Don't hesitate.
Visit talkpython.fm/datadog and see what you've been missing.
That's talkpython.fm/datadog.
You know, there are some companies that basically have rules that say you're not allowed to create a web app that has more than 10,000 lines of code in it.
What you have to do is create a service and then maybe multiple services and then you can construct your app out of these services.
Right?
Almost like creating these guidelines that just naturally leads to microservices.
So we've competed against monoliths.
The other thing that I feel is like really strongly working in this space, trying to achieve the same thing has some benefits, some trade-off is serverless architecture.
AWS Lambda, Azure Functions, things like this, right?
Yes.
What do you think about those relative to this?
So glad that you asked because that's actually how we at Rackspace, in my team, that's how we deploy our microservices.
So we haven't discussed this, but one of the main components in a microservices platform is the load balancer.
You know, to achieve all these scalability and no downtime upgrades and another benefit that you get, you need to have all the services load balanced.
Even if you run one instance, it needs to be behind a load balancer.
So what you get when you go to a serverless platform like Lambda on AWS is that AWS manages the load balancing for you.
So all you need to do is, you don't even need to have a WSGI server.
All you need to do is write your microservice as a function and then upload the function with all its dependencies to AWS.
And then anytime, you know, the function gets called, AWS will somehow figure out how to run it.
It'll start a container, put the code in it, and then run it.
If you, in a burst, you make a hundred calls, then AWS is running the load balancer and it'll run a hundred containers for you.
You don't have to worry about it, which is really nice.
And then if you make an upgrade, you know, the moment you make the upgrade, any calls from the known will use the new code.
So you got immediate, no downtime upgrades as well.
Yeah, that's really neat.
Do you think that there's good situations to have like a combination?
It seems to me like there's certain things that would be really well suited for like a serverless Lambda type of thing.
And others maybe more stateful, like sort of a bigger microservice that would much better fit somewhere else.
So I'm thinking like if you wanted to say charge someone's credit card with Stripe, like to do that as a, like a single Lambda function that's really stateless, really straightforward.
Maybe that would make perfect sense.
Maybe something more complex, like for example, your message push stuff wouldn't necessarily be as appropriate there.
Right.
So here's one very important thing that Lambda does not support.
It does not support WebSocket services.
So server push, exactly what you just mentioned.
Basically for that, you need to have a permanent connection with the client.
So we have a WebSocket connection.
All the clients have permanent connection to the server.
The server needs to handle a lot of connections.
Now the Lambda services are, or functions I should say, really ephemeral, right?
They run and then they exit and then they don't exist anymore until you make another call.
So there's no way to have a permanent presence in Lambda.
So in that case, you will have to host that in a container or something like an instance, a cloud instance, for example.
Sure.
And do you get maybe better response time with, if you run it and say your own container that's more permanent?
Like, so there's probably a little startup to infrequently called Lambda functions or something, right?
In general, if you're looking for performance, you will not be using Lambda.
That's my experience.
It's, in general, slower.
So I can give you an example from work.
You know, we don't have, you know, nothing that's extremely complex.
But typically we see a REST API call that's hosted on Lambda and it goes, it gets to Lambda through another AWS service called API Gateway.
And we're seeing that nothing is, nothing takes less than half a second.
So 500 milliseconds for a simple call.
There's really, we found no way to bring it below that.
No matter how simple the actual endpoint is.
There's that much overhead just to get to your function, basically, get it up and running and so on.
Right. Yeah.
There's so many layers of that that go through AWS before, you know, your code gets to run that you really, you can optimize all you want.
You're still not going to make a difference.
Yeah.
Whereas in a Flask or other Python and Wizgy app, like 10 millisecond response time would be totally reasonable.
Correct. Right.
You can see much faster times and then you have the option to go async if you want, you know, something that's, that's very, very fast and can tolerate a lot of clients.
Yeah. So I guess the takeaway probably is like these serverless components of microservices, these like serverless building blocks are cool, but you can't just in general, it wouldn't make sense to just go and I'm only doing that for the most part.
Probably.
It makes sense in many cases, not every case.
And you have to keep performance in mind.
Typically, if you know, the kind of services that we do, these are all requested by a, by a rich client application, something like Angular or React.
And, you know, those are background requests.
So, yeah, if it takes half a second, that's fine.
Yeah.
Usually not, not a big deal.
We actually, we, we had one of the little services that we wanted to build at some point was,
a auto-completion.
Yeah.
It's not worth that.
And was that frustrating on Lambda?
Yes, exactly.
That you cannot, you cannot host that in Lambda.
That's the typing equivalent of hearing your own voice echo back in a call, right?
Yeah.
It's not good.
Yeah.
That didn't go the way.
Yeah.
That was.
So one of the challenges that I can certainly see, especially if you start throwing containers
into this as well as like, if I have a monolith, it knows how to find the user, the user interaction
bit and the credit card bit and so on.
It just, you know, imports and it works.
But if you break it across all these different servers, like how do you keep it connected without
the hardwiring every bit of deployment into the code?
There's a component that all microservices platform have that's called the service registry.
So basically there are, you know, each platform does it in a slightly different way.
But in general, the idea is that when you start a service or an instance of a service, the first
thing that the service does is talk to the service registry, which is in a known address,
right?
So everybody knows where to find the service registry.
That one, you basically hard code the domain or something in it.
Right.
It's hard coded.
It's usually in production deployments, it's highly available.
So you are not going to have, you know, a single point of contact.
Probably you can code a few, you know, addresses to talk to the service.
And if one of them is down, you try the next one.
So you want to make sure that, you know, this piece of code is always running.
Basically, the service starts and then it reports itself to the registry.
It says, hey, I'm here.
I'm at this address.
If you're running containers, the address is going to be basically Docker, for example,
it's going to come up with some port for you.
So you find out what port you're running on.
And then you tell the service, okay, I'm running on this address and this port.
So I'm ready to start taking jobs.
Yeah.
And that service registry can be very simple code, right?
It could almost just store like a dictionary in memory or something, right?
Right.
Essentially, it's a dictionary, right?
If you think about it.
The complication is that you want that to be super robust.
If the server where the registry is running goes down, then nothing can find, there's no way for microservices to communicate.
So it's very important that you hosted, you know, multiple locations and, you know, that there is a redundancy.
Do people sometimes use things like S3?
Just like, I'm going to write to this bucket.
And then, that's actually an interesting idea.
I haven't seen that.
It might get complicated with multiple accesses.
Yeah.
You need to implement some sort of a knocking mechanism.
I would think to keep the, you know, the, the file that has all the list of services robust and never get corrupted.
Yeah.
Maybe each file could have it.
Each service could have its own file.
I don't know.
Or, right.
You could write different files.
That, that actually could work.
Yeah.
I think that could work.
Interesting.
But basically, there's a really simple thing that just every, every server can go, I'm here.
The things I need, where are they?
Right?
Right.
So then, just to complete this, they, one of the, the simplest service registry options that, that, that, the one that I like, it's called etcd.
It's an open source project from core OS.
And, basically, yeah, you, you send a, a request.
You can even do it in a batch script with core.
Okay.
Just, it's a key value database, basically.
That's very fast.
So then etcd, in this example, will have the list of all the services that are running.
On the other end, we have the load balancer.
And the load balancer will go periodically check the contents of this service registry and refresh its own configuration based on that.
So, a service starts, it writes itself to the service registry.
Then on the other side, the load balancer says, oh, there's a new service.
I'm going to add it.
Yeah.
Oh, that's, that's cool.
I didn't think of using the service registry to drive the load balancer, but that's cool.
Yeah, that, that, that's very nice.
There are, there's actually, I know of one load balancer that, that has this functionality embedded.
It's called traffic.
If you go with a more traditional one, like Nginx or HAProxy, which are the ones that I've used for a lot of time.
With those, you, you need to add a piece to the side that does the, the, the, watching the service registry and then updating the configuration and restarting the load balancer.
Right, right.
And the one that I know about, which is actually written by, by Kelsey, it's called ConfD.
That's the one that I, I showed in the class.
Okay.
Yeah.
That's cool.
And Nginx resorts pretty quickly.
So it's, it's pretty good.
Right.
Nginx is pretty good about reloading itself cleanly.
When, when you update the configuration, HAProxy is getting there.
It's getting better.
It's a little bit clunky, but basically it starts a new process when, when you want to update the configuration.
And process by which all the connections are passed from the old process to the new process has been for many years has been problematic.
It costs some downtime.
It's much better these days.
Okay.
That's great.
Another challenge I can imagine is if I just start using logging in my monolith app, it will all go to the same file.
It will go always in order unless I'm doing threading.
It's a piece of cake, right?
It's a piece of cake.
If I have 20 little microservices, how do I like trace through the steps?
Right.
You need help.
You really can't manage 20 log files.
Sometimes it gets insane.
So basically you use, you find another piece, but basically a log consolidator.
And there's one that's pretty good for, I mean, if you're doing Docker containers, that's called log spout.
Log spout.
Okay.
So basically what this does is it grabs all the logs from all the containers in a host, the host where log spout is running.
And then basically it outputs a stream that you can watch over.
For example, you can connect with a web browser.
That's WebSocket, for example.
So you can connect over WebSocket and then watch the stream of logs in real time.
Or you can connect log spout to something more useful in a production environment, which will be, for example, an elk stack.
So Elasticsearch, Logstash, and Kibana, which is this trio of open source apps that basically create a very powerful log storage and search solution.
Yeah.
Okay.
So basically you put in something that brings it back into one place.
Yes.
You usually want to have everything in one stream and then you can filter if you want.
But imagine if you have five instances of the same microservice, you may, even though it's five different ones, you may want to see the entire thing.
Because if a client is sending requests to this service, requests are going to randomly arrive to any of those five.
Right.
It's hitting the load balancer.
Yeah.
Right.
So you probably want to see the sequence regardless of which of the instances get a specific request.
Yeah.
Do you do or do people do things where they somehow identify a request?
So like flag it with some kind of unique ID at the top and then flow that through so that you can say this, this is the steps that this request went through.
Yeah.
That's pretty common.
Some, some platforms offer that.
I implemented by hand in some cases myself.
But basically, yes, the entry point request.
So the first, the service that receives a request from the outside world assigns an ID to that request.
And then in any communications that service has with other microservices, it will pass that ID.
So you always preserve and log the initial ID and then, and then you get a trace of all the services that worked on a single client request.
Right.
And there's infrastructure to actually do the communication between microservices.
Is that typically requests?
There are different ways.
So the easiest will be to use HTTP as an interface, as a REST API for each service.
And then, yeah, you use a Python requests.
Some people prefer something that's less chatty.
So HTTP, as you know, you know, you have all these requests, the headers, you know, all that stuff.
When you are talking to a client, that makes sense.
Right.
And besides, that's the only way the browser can, or an HTTP client can talk to the server.
But when you're talking among services, you may say, well, okay, I want something quicker.
So some people implement RPC schemes where a service can say to the other server, hey, I need to do, I need you to do this.
And it's, for example, passing messages through a message queue, which could be a Redis queue or SQS if you're on AWS.
Do people set up socket servers, maybe?
You could do a socket server too.
Yeah.
If you're looking for really low latency, low traffic.
The main idea, what I would consider a good design point, you know, when thinking about how microservices communicate, you want to leave the door open to having different, you know, tech stacks on different services.
You don't want to go with something that, let me give you an example.
I would probably not use a Celery worker for this, right?
Because that will basically restrict me to use Python.
Right.
And you probably wouldn't ship data across the wire as pickle.
Right.
Yeah.
As versioning issues, not just, not just Python.
Yeah.
But even within Python, that could be a big problem too.
Right.
Yeah.
So, you know, the way Celery works, I think it's not, you know, it's not friendly to the microservices
idea.
So the idea is that microservices try to push.
So yeah, I would go HTTP.
Maybe messages are JSON formatted over a queue.
Mm-hmm.
All things that you are sure that any technology can easily communicate over.
Yeah.
It makes a lot of sense.
I imagine there's a lot of JSON happening back in the data center.
Right.
Yes.
Absolutely.
Yeah.
Nice.
So that maybe brings us to an interesting place to talk about the tools.
We talked about requests and some of the other tools that work and don't work, but what
are the other Python tools that you might involve here?
Well, of course you need a framework, right?
And we discussed this.
I like, or surprise, I like to use Flask, but really you can use any web framework that
you like, right?
As long as it knows how to communicate with the other services.
As far as other Python tools, there are many packages that talk to service registries, for
example.
So if you want your Flask-based or Python-based microservice to be able to talk to the service
registry, there's packages for certainly for etcd.
If you use another one like console, for example, this one from HashiCorp, there's packages.
So you're going to find a lot of support for these tasks that you need to do that are sort
of specific to microservices in the Python ecosystem.
Sure.
Besides that, you're going to be doing things, and this is something that I really like, in
the same way like you work with a monolith, but you're going to be working with a much much
much smaller code bases.
code bases.
So you're going to be doing unit tests the usual way, but you're going to be testing each
service separately.
And then you're going to have integration tests if you need, and you probably do.
But yeah, nothing really changes.
It's just that the scale goes, you know, you're doing your work on a much smaller scale.
You're working with smaller applications.
Yeah.
It sounds to me like they're easier to work on and maintain and deploy, but maybe more difficult
to initially set up the infrastructure that wires them together.
Yes.
You have more servers to set up.
You've got the load balancers.
You've got the service registry.
Like these things you have to add.
But once that is in place, it kind of sounds like life gets easier.
So there's like a bar to cross, but once you cross it, you're good.
Yeah.
Right.
I agree with that.
Yes.
It's difficult to set up the platform.
And of course you can go, if you use Kubernetes, for example, or AWS Lambda, you know, a lot
of all those pieces are done for you.
You don't have to worry about load balancers and service registries, right?
They do it for you.
This portion of Talk Pythonry is brought to you by us.
Us.
As many of you know, I have a growing set of courses to help you go from Python beginner
to novice to Python expert.
And there are many more courses in the works.
So please consider Talk Python training for you and your team's training needs.
If you're just getting started, I've built a course to teach you Python the way professional
developers learn by building applications.
Check out my Python jumpstart by building 10 apps at talkpython.fm/course.
Are you looking to start adding services to your app?
Try my brand new consuming HTTP services in Python.
You'll learn to work with the restful HTTP services as well as SOAP, JSON and XML data formats.
Do you want to launch an online business?
Well, Matt McKay and I built an entrepreneur's playbook with Python for entrepreneurs.
This 16 hour course will teach you everything you need to launch your web based business with
Python.
And finally, there's a couple of new course announcements coming really soon.
So if you don't already have an account, be sure to create one at training.talkpython.fm
to get notified.
And for all of you who have bought my courses, thank you so much.
It really, really helps support the show.
In this class that I gave at PyCon, I didn't want to just tell, okay, install Kubernetes and
you're done.
I wanted to teach what microservices are.
So I built my own platform, which was a lot of fun.
And I thank the PSF for approving my tutorial ideas and let me work on this.
It was a lot of fun.
And I wanted to demonstrate that really it's not as hard as it sounds.
You can go pick, you know, the best tools for each of the different tasks, you know, in
a very flask, you know, fashion, right?
Where everything is done, you know, you pick the best tool for each task.
Yeah, it sounds like people who like micro frameworks for their web frameworks might like this as
well, right?
Because you kind of get a pick and...
Yes, they're going to find that there's a lot of affinity, right?
So I built a platform using Bash.
So it's all Bash scripts.
You can do a...
You've seen Kelsey Hightower do a super cool voice operated demo of a no downtime upgrade,
right?
So take away the voice thing.
I didn't do that.
But, you know, during class I showed how with a Bash script you can deploy your upgrades
without the service ever going down.
You know, your application keeps running while you do the deployment.
Sure.
So, yeah.
Yeah.
So is there a roadmap or some guidance on how to take maybe a 50,000 line monolith web app
and turn it into a number of services?
Well, that's really difficult.
I think that the best advice I can give you is you probably cannot do it all in one go.
You're going to have to find a strategy to do, you know, phased migration to microservices.
You need to have a very good unit tests in place because the task of...
Basically, what you're going to do in most cases is take the monolith and put it inside a microservices platform as a big piece, right?
And then over time, you're going to start taking little pieces out of it and write microservices, right?
So the task of writing a microservice when you have, you know, the monolith is basically involves a lot of copying and pasting, right?
You have to move endpoints that are in the monolith to a standalone application.
Sure.
And that's pretty easy in some aspect, but breaking the tight coupling and the dependencies of code that you're moving around, that sounds to me like it could be pretty challenging.
Yes.
It's difficult.
It's actually hard.
You know, all that...
Basically, when you work on a monolith, you accumulate technical debt.
That's pretty common.
You're going to find that many times that technical debt is going to inform your decisions.
You're going to take less than ideal decisions when you design your microservices to keep things the same way.
I can give you an example.
In this project that I showed during the PyCon class, I was actually migrating this chat application to microservices.
And I started and I migrated the UI first.
That was very easy.
And then I migrated the users.
And then I went to migrate tokens.
And I realized that I could do a much better job with tokens.
The tokens in the old application were sort of inefficient, were random strings.
You know, when you're working with microservices, you want tokens that can be verified without calling the token service.
And when you need that, you usually use JSON web tokens, which you can verify with cryptography.
So, I had to decide, I mean, do I keep this and make it inefficient?
Or do I say, okay, I'm going to draw a line in the sand and I'm going to change the token format, but then everything is going to break.
I'm going to have to migrate all the services, you know, to the new token style.
Right.
And those decisions, you know, on a real application, they're going to be, you know, much harder to make.
Yeah, I can imagine that.
But, so, one thing I was thinking of while you were speaking of like how you might break this apart, it seems like you could almost do the partition in the web tier and then the data.
Like, so, for example, if you have a database that obviously the monolith talks to the database, all of it to the same, through the same connections.
If you could break this out into services, they theoretically could go back and just continue talking to the same database and you could kind of get the service decomposition piece working.
And then you could say, okay, now how do we move this into the application database that's dedicated to each one of the services.
So, that could be a valid approach.
So, when you do it that way, if you're sharing the database, then the zero downtime upgrades are still difficult.
Yeah, I'm just thinking as a transitional thing.
So, it's going to be a transition, right?
Yeah.
You want to go all the way eventually.
But, yeah, definitely.
Okay.
Yeah, you have to figure it out.
It depends on the application what's the best route.
But, yeah, it's difficult.
What I've seen some people do is they say, okay, I'm going to migrate to microservices, but only from now on.
I'm not going to change what I have.
So, basically, they grandfather this big piece.
You know, they think of it as a microservice, even though it's not.
It's a big microservice.
Right.
It's a big one.
But then, you know, from then on, any new functionality, they start writing in microservices.
And that's actually a very valid approach.
In many cases, it's the only viable way, right?
Sure.
Sure, sure.
How does software development change for a team when they're transitioning to microservices?
How does their world get different?
Well, they don't have to all work in the same code, right?
So, that's a big plus.
Fewer merge conflicts.
Right.
You basically merge conflicts.
For me, I don't remember when I had a merge conflict last.
Right?
Yeah, you don't usually see that.
So, usually, you're going to find that your team, the members will get specialized in specific services, right?
For example, at Rackspace, I've been doing a lot of authentication microservices, right?
So, you know, when there's a new need for authentication, I do it usually.
Some people may not like that, right?
May prefer to be generalists.
So, yeah, it depends.
But you find that, you know, some people is more, basically, has affinity to certain parts of the system, certain microservices.
Sure.
And now they can focus more on it now because it's more explicit.
Right, yeah.
And they can do a much better job at, you know, that specific task because they don't have all the baggage of the remaining, the rest of the system.
That's basically that needs to be, you know, that you have to make sure that you don't break.
Right.
So, one thing that seems like it might make sense would be to rotate the team around through the different services, potentially.
If you want to make sure, like, there's many people that kind of know the whole thing.
Like, you can say, okay, this month you're on this service, that month you're on.
Yeah, that's actually a good idea.
You can find a way for the person that's experienced with microservice to sort of mentor a new member, you know, and basically code review the changes that the new person makes, for example.
Yeah.
There are a lot of different ways to make sure that everybody gets a little bit of everything, sure.
Yeah.
So, yeah, overall, I find that if you like to code, right, I mean, we can talk about the ops side, right?
If you like to code, then you're going to be coding more and, you know, fixing bugs a lot less.
Yeah.
You're going to find that you're going to be working on small code bases and that leads to less mistakes and errors.
Yeah, that sounds great.
And I have to write unit tests that a lot of people don't because it's too complicated.
And now you're back to a simple application that's very easy to unit test.
Yeah.
To be more careful on the boundaries, though, because they all talk to each other, right?
And then you need, right, this I would probably put an experienced person.
You need someone that overviews what are the interfaces that all the microservices expose to the rest of the, you know, to the other microservices and sometimes to clients.
Right.
You need to make sure that, especially with the public endpoints that are consistent.
So you need one person that's experienced, at least one person that's experienced in API design to make sure that you get good APIs.
Yeah, of course.
That's the other thing.
It's very difficult to change them once they're out there.
And this is if you want to have no downtime deployments, you cannot really introduce breaking changes.
So you cannot remove an API.
You cannot remove a column in the database.
You know, there are some rules that you need to follow.
Right.
So when you design databases and when you design APIs, you need to have people review that very well, make sure that you like what you are designing, because you're going to have to be with those decisions for a long time.
Yeah, that's a good point.
Some of the HTTP frameworks for building services have even built in versioning into them.
I'm thinking of like Hug and some of these things.
But obviously, you can add it into your own apps pretty easily.
Just set up a second endpoint rather than calling the same one.
Is there something that you have to do that?
You have to do that.
Basically, you're forced.
So imagine you have five instances running of this one microservice.
And now you want to introduce a breaking change in one endpoint.
And of course, you don't want to go down for the upgrade.
So you cannot stop the five instances at the same time.
You're going to have to do a rolling update upgrade.
Right.
So, you know, during a window of time, you're going to have a bunch of instances on the old API and a bunch on the new.
And then the rest of the system knows nothing of this.
And they're going to start sending requests, probably assuming that the old API is in place.
Yeah, until you upgrade that part, which then I'll go to the new one.
But it'll be sort of moving around through these services as that process happens.
Right.
So you need to create a new endpoint for the breaking change.
Keep the old one working.
And then once you're sure that the whole system is upgraded, it's on the new one.
Only then you can go ahead and basically deprecate or remove the endpoint that you don't want to use anymore.
Right.
That's probably important as well to like eliminate the cruft building up as well.
Yeah.
I think that, you know, this platform encourages you to be clean and to keep things clean, to think about these important decisions, you know, very carefully.
Yeah.
Excellent.
So we talked about how AWS Lambda has this sort of built in latency.
And when you think about performance, a lot of times there's really two sides to it. Right.
One is if I go and I hit that endpoint, how quick does it get back to me?
And then the other one is if a million people hit it, how much does it degrade from that one person hit it? Right.
Like how scalable is it and how like single request high performance is it?
I can certainly see how this generates better scalability. Like you can very carefully tune like the individual parts of your app and scale those up.
But it also seems like it might add some latency like for an individual request. So how much slower, like what's the typical changes? Like would I add 10 milliseconds to a request? Would I add a hundred milliseconds? What?
Yeah, that's a good question. So even if we take serverless out of the equation, so.
Yeah, because it's really bad. Yeah.
Right. So the performance is not the same as in a monolith, just by the mere fact that in many cases, the client is going to send the request and the service that receives that request cannot carry out the request alone. Right.
So you have to talk to a bunch of other microservices. Right. So there are a lot of communications among the microservices that of course will take time as well. Right.
So, so, so, so latencies increase no matter what. Right. So microservices is not, if you're looking for performance, it's not really.
It doesn't necessarily have to be microservices versus monolith. It could be more coarse grained microservices, more fine grained ones potentially. Right.
Right. The times that I've seen, they're not really, you know, that terrible. I mean, we are talking like tens of milliseconds.
Yeah. You know, you're making requests over the internet, maybe across the country, it might be a hundred millisecond ping time. So if it's a hundred, 110, who cares?
It could be in the noise. Right. Yeah. Absolutely. In many cases, it's going to be in the noise. Yeah.
So, you know, compared to the, all the benefits that we already discussed, I think in my view, it's a no brainer. It makes a lot of sense in many cases, but yeah, it's, you know, it's all this complication of, you know, services talking to each other.
And you might find that in some cases you need to go async. So you can totally have an asynchronous microservice. So you tell it, you need something and the service says, yeah, okay, I'll do it on my time.
Right.
But keep going. Don't mind me. Right. That's totally fine.
Yeah. For example, send this email. You don't have to wait for the email to be acknowledged to be sent. Right.
That's a great example. Right. Yeah. Yeah. It can definitely, email can be pretty slow actually, given all the stuff that it does.
Email, yeah, you count it in seconds.
Yes, exactly.
Microseconds.
You got to mail a group, right? Like, I want to send this mail to 2000 people in my class on my platform. Like, okay, that needs to be a thing. Just let me tell you, I've learned that.
Yeah. And that's actually, that's a good example in which you may, in many cases, you're not interested if the email bounces.
Yeah. What are you going to do about it anyway? Right. Right. Exactly. If it bounces, there's nothing to do. If you are, you can have that service that's asynchronous record the addresses that are bouncing in a database that's going to be owned by that service. And then later on, you know, in a cron job or whatever, you can clean up the addresses that are bad.
So some other service can send a request to the email service and ask, you know, what are the bad addresses that you know? And that that will be another endpoint. You'll return the addresses and then they can be cleaned up or, you know, the accounts can be canceled or whatever.
Yeah. It sounds, it sounds really promising.
It's a great way to think about problems, right? It's, it's all, you know, little pieces. So it's a lot easier to think about solutions there. And, and, you know, at the beginning, it's hard to start thinking this way.
But, but then you get used to it and all the problems become easier.
Yeah. I can definitely see how that, that would happen. It might be difficult to think, how am I going to build this huge app? But if I can build, well, could you build 10 small apps and then have them help each other out? Right? Right. Exactly. Yeah. Very cool.
All right, Miguel. I think we're going to leave it there. That's like, I think a great, great conversation on microservices. So let me ask you the two questions. So if you're going to work on your microservices and Python, what,
editor do you use? So it's getting complicated. It's complicated. Yeah. That's the correct answer for me. I usually iterate over a few editors. So Vim is my go-to editor. Many times I need to edit files on remote hosts. So Vim works anywhere. So that, that, that's, that's the one that I use most of the time. Sometimes I need to debug. And for that, sometimes I use, you know, an IDE. And the two that I've been using, I can't even
decide on one. I, the two that I'm using is PyCharm and lately Visual Studio Code, which, which is surprisingly good.
Yeah. The, the Python plugin there is, is doing really quite a bit. Python plugin is, it's not an official plugin, but yeah, this person that I wrote it, he, he did an awesome job. It's a, it's very, very good.
Yeah. He did a great job. I actually had him on an episode, maybe 20, 20 shows back or something. It's, it's very cool how we took like 10 different
open source projects and like brought them all together and turned it into the plugin for Visual Studio Code. It was cool.
Well, he did a great job. It's super powerful. And in particular, I like the way that you, you, you set your configuration for a project, which basically it opens up a text file and you write JSON.
Yeah. It is quite interesting for sure.
That's a, you know, contrasting to PyCharm where you have to enter a dialog, you know, a window and find the setting that you want.
Yeah. Well, it also makes it very source friendly, like a source control friendly.
Source control friendly. And, you can copy configs from one project to the next. It all becomes much, much easier.
Yeah. Great. All right. And PyPI package.
So one package that I'm, I'm sort of ashamed. I, I didn't know. And I, I, I learned about it from my colleagues at Rackspace. It's called Arrow.
And so, so this is a package that it's a, drop-in replacement for, for the daytime package in the Python library, but it implements all the holes in support that, the daytime has.
For example.
Yeah, definitely. I would definitely second that one. Arrow is awesome.
Yeah. I, I, I only knew it from, from a few months since I've been working with this team and I, yeah, it's, I, I use it all the time now.
So for example, it has, you know, daytime starts with, with this naive time zone approach where there's no time zone.
So by default, Arrow will, will use UTC, which is what you always want anyway. Right.
So you always work with UTC.
Especially if you're working on servers.
Right. Yeah. You, you want to have, common units. So, so that, that's the one that everybody uses.
And then, support to convert to and from the ISO 8601, which is, daytime can output ISO 8601, but cannot import from it.
So if, something very common. So another thing that I, I tend to work on is, is the, the billing microservices in my team.
And you, you want to know, you have a date and you want to know the first of the month and the, the last of the month.
It's like in one line you can get it. You don't have to do strange aerobics or acrobatics to, to get the first and last of, of a given month. So yes.
Yeah. People should definitely check that out. All right. So people heard this conversation. They're probably excited about microservices. How do they get started? What do they do?
So what I recommend, if you have, three hours to waste, you can check out the, the YouTube video of my tutorial.
I think I've made it very approachable. If, if your experience developing either Django, it doesn't need to be Flask.
So any web, you know, monolithic web applications, I think you're going to pick up the tutorial really well.
So the code that comes with the tutorial, which is on GitHub includes a, vagrant setup.
So you can deploy the system that I show in the class to, to your machine, to your laptop on a vagrant VM, and then you can play with it.
And even at the end of the class, I, I listed a list of things that, that will be great ideas to, if you want to practice.
So you can take the project that I built and then extend it in many different ways.
Uh, that would be my recommendation. you can look into Kubernetes, which is, something that you can also deploy to your laptop.
If you want to use it for testing. I include it in, in, in the, in the GitHub repositories for this class, the, the scripting, required to deploy.
The same application to Kubernetes, if you're into that. And then the, the other valid option would be to look into AWS Lambda and API gateway.
Have you seen Zappo?
You mean Zappa? Yes.
Zappa. That's Zappa. Yeah. Which is a framework that just like uses Lambda as a backend.
Yes. I I've seen it. I, I, I even wrote a clone of it, which is called Slam, which is in my GitHub, account as well.
But yes, the idea is that you take a Whiskey application, which is really a function. If you think about it, Whiskey,
you know, it's a callable, not only a function, but a callable. The API is quite simple. You can think
about it as a, as a function, right? And, and then Lambda requires a function. So there's a really
match. The only problem is that the way Lambda expects the function is not the, you know, in the
way Whiskey applications are formatted. So then Zappa comes in or my slam also. And basically the,
it's an adapter that sits in between Lambda and your application and makes the conversion between the
two formats. I see. All right. Well, that's really cool. That that's a really easy way to deploy your,
your, any Python web application that that's, you know, Whiskey. So Django,
Plask, Pyramid, Bottle, all those, you can get a deploy to AWS. Yeah. All right. Well, very,
very cool. So I definitely recommend people check out your tutorial, which I'll put in the show notes.
And I'm also going to put Kelsey Hightower's talk. Yeah. Those go well together. I think
that's, that's actually a good thing to watch first. If you like that, then you can learn how
those things work by watching the tutorial. Yeah. Kelsey's is high level and flashy and
interesting. And then yours is the detail. Yeah. Right. I wish I could be as good a speaker as he is.
Yeah. That was really great. Yeah. All right. Well, Miguel, thank you so much for being on,
on the show once again. And it's great to chat with you. Thank you for inviting me. You bet. Bye.
This has been another episode of talk Python to me. Our guest was Miguel Grinberg. And this episode
has been brought to you by Datadog and Talk Python training. Datadog gives you visibility into the
whole system running your code. Visit talkpython.fm/Datadog and see what you've been missing.
Don't even throw in a free t-shirt for doing the tutorial. Are you or a colleague trying to learn
Python? Have you tried books and videos that just left you bored by covering topics point by point?
Well, check out my online course, Python Jumpstart by building 10 apps at talkpython.fm/course
to experience a more engaging way to learn Python. And if you're looking for something a little more
advanced, try my WritePythonic code course at talkpython.fm/Pythonic. Be sure to subscribe
to the show. Open your favorite podcatcher and search for Python. We should be right at the top.
You can also find the iTunes feed at /itunes, Google Play feed at /play and direct RSS feed
at /rss on talkpython.fm. Our theme music is Developers, Developers, Developers by Corey Smith,
who goes by Smix. Corey just recently started selling his tracks on iTunes. So I recommend you
check it out at talkpython.fm/music. You can browse his tracks he has for sale on iTunes and listen
to the full length version of the theme song. This is your host, Michael Kennedy. Thanks so much for
listening. I really appreciate it. Smix, let's get out of here.
Thank you.
Developers, developers, developers, developers, developers, developers, developers.
.
you