Python is said to be a language that comes with batteries included.
It has many meanings depending on the level you're focusing on.
The lowest, it's a very rich and expressive language.
Most commonly, this phrase means Python has a very powerful and comprehensive standard library.
Itertools or elementary, anyone?
But more holistically, most of the batteries are external, interchangeable ones from PyPI.
Right now on PyPI.io, there are over 97,000 packages listed there.
That's fantastic.
However, it does lead to a discoverability problem.
What options do you have for admin backends on web apps?
How about cryptography or raw socket programming and so on?
This week, Matt McKay is back with me.
Remember him talking about deployments on episode 26 last year?
This time, we're talking about Awesome Python,
a website and open source project attempting to categorize and sort of rank these options.
This is Talk Python to Me, episode 96, recorded January 16th, 2016.
Developers, developers, developers, developers, developers.
I'm a developer in many senses of the word
because I make these applications, but I also use these verbs to make this music.
I construct it line by line, just like when I'm coding another software design.
In both cases, it's about design patterns.
Anyone can get the job done.
It's the execution that matters.
I have many interests.
Sometimes you can fly to Python.
Welcome to Talk Python to Me, a weekly podcast on Python,
the language, the libraries, the ecosystem, and the personalities.
This is your host, Michael Kennedy.
Follow me on Twitter where I'm @mkennedy.
Keep up with the show and listen to past episodes at talkpython.fm
and follow the show on Twitter via at Talk Python.
This episode has been sponsored by AnacondaCon from Continuum Analytics and by Hired.
Thank you both for supporting the podcast.
Hey, Matt.
Welcome back to Talk Python.
Thanks, Michael.
It's good to be back.
It's great to have you back.
You just pointed out a little bit ago that it's been 70 episodes
since you were on episode 26 to talk about deployments with Python.
I can't believe it's been so long.
It's hard to believe.
It's been like a year and a half, almost two years.
Yeah, yeah.
Well, that was a great episode.
It was really popular.
And, you know, the world has changed a lot since then.
So that's a lot of cool things we're going to be talking about today.
We're going to focus on stuff that we found, mostly you have found,
that's awesome in the Python community.
But before we get into those, let's, you know, 70 episodes ago.
I'm not sure everybody can remember back 70 episodes.
What do you do day to day?
Sure.
So I'm a developer evangelist at Twilio.
And Twilio makes it easy for developers to add communications.
So if you want to send a text message or you want to make a phone call with Python
or any programming language or add video chat to an application,
you can do that with Twilio.
What I do as a developer evangelist, I am, first off, a software developer.
I write a lot of code.
Most of my code is open source.
So it'll be examples for how to use Twilio or even just how to create something as a developer.
It doesn't have to use Twilio.
And then I travel and speak at tech conferences all on software development.
No marketing fluff.
Just really just being a software developer, I get to write code, explain code, and teach other people,
which is just absolutely awesome.
Yeah, that's great.
I always, I've always thought that these developer evangelist jobs were just fantastic, right?
Like it's part research and play.
It's part telling people about stuff that's cool.
And you have a lot of freedom to kind of learn and explore, right?
Yeah.
I mean, the big thing for me is just getting the time to create projects.
Like when I have an idea, I can just go sit down, write some code, implement it,
and then show it to other people.
It's really, it's one of my favorite parts about it.
How much does that feedback like tweak or inform the APIs that Twilio uses?
Oh, a lot.
Coming back from the evangelist.
Yeah.
I mean, one of the best parts about the gig is I get to use all of our APIs before they come out.
So before anybody gets their hands on them, myself and some of my colleagues are like testing out the APIs,
giving feedback to the API teams.
And that helps to improve the results that ultimately get into the hands of developers.
Oh, that's great.
Now, if I go out and search for something on Python, there's a very good chance that something that you are responsible for that's not directly part of Twilio will be like the number one Google result for my answer.
You do full stack Python, right?
Yeah.
So I'm the author of full stack Python, which is a open source project.
I started over four years ago.
And the whole idea behind full stack Python was I'm a Python developer.
And I had this before I joined Twilio.
I had a bunch of junior developers that were joining my team and they kept sending me emails.
Hey, what's a what's a whiskey server?
Like, what is that?
And I kept sending all these emails and I was like, this would probably be relevant to much more than just our team.
So I started putting together a project with pages that had explanations on every single Python topic.
And it's just expanded out from there.
I pretty much work on it every day.
And it's it's read by over 80,000 developers a month at this point.
That's fantastic.
And you have a book that goes with it as well, right?
Yeah, that's what I was on for in episode 26 back 70 episodes ago.
It's about it's a step by step guide to deploying Python applications.
So if you've never done it before, you've built your first web application.
This is every single step to get a virtual private server and to stand everything up that you need DNS, the whiskey server, reverse proxy, all those things that trip developers up and certainly trip me up several years ago when I was trying to deploy my own applications.
Yeah, it feels to me like that kind of that kind of information, those kinds of things are the things that you kind of it's a lot of work and a lot of research to get it all put all the pieces combined just right.
So your web apps work great, but then you write it down, you put it in a script and you forget about it.
So having like a book as a reference to learn it the first time and then keep it around is really cool.
Yeah, exactly.
So and as we talked about on that episode, first, I teach how to do it manually and then we automate it with Ansible.
So it's a pretty good way to kind of understand the nitty gritty and then automate it.
And you know what, if you want to forget about it at that point and just use the scripts, you can, but at least you learned it the first time.
Yeah, absolutely.
You'll understand that when you come back, right?
You'll know what you're doing if you need to need to come back and tweak the automation.
That's cool.
Now we're going to talk about Awesome Python, the project, the GitHub project.
But before we get into that, how about we do a quick update on our Python for Entrepreneurs course that you and I are writing together?
Yeah, let's do it.
So that course is almost done.
I have to say, this one is, how shall I put it, longer than I expected.
A little longer to write than I expected.
But I haven't even told you, I've finished almost all the chapters I have to record.
They're off to editors.
I have basically one and a half chapters left to record and I'll be done.
And then that just leaves a few more to go and that class will be all wrapped up.
Yeah, and for me, I'm just finishing up some of the second Git chapter and then we're going to get into deployments and SEO and everything that goes along with actually running the application that you build.
So I'm really excited.
I am so excited to get this thing out to people.
I know people have been watching it in early access mode, which you get from just buying, pre-ordering the course.
But I'm excited for it to be like really wrapped up and done.
Yeah, I really hope this turns out to be a great resource for the Python community around sort of this whole lifecycle of idea to web app to deployment to business.
It'll be fun.
Okay, so we'll come back and let people know when that's actually finished, but it'll be soon.
I promise those of you who are out there waiting.
So let's talk about some stuff that's awesome.
Sure.
So the awesome Python repository has the great slogan, life is short, you need Python.
And it is one of many awesome lists.
So the awesome list popped up on GitHub where people just started throwing together lists of tools and resources that they were using.
And so this repository came up in the Python community.
And so this is a GitHub Vinta slash awesome Python.
It was created by Vinta Chen about a little over two years ago.
And this is kind of the main one for the Python ecosystem that goes over all of the different open source projects and categorizes them, whether they're a task queue or web framework has some sort of basic categorization for each of the projects.
Yeah, I think this is really handy when you don't know what it is you need for something.
And so I don't know how many projects are listed here.
I probably could have gone through GitHub source code and like written some script or something to figure it out.
But it seems to me like it's close to like 500, 400 projects.
And they're all categorized.
Like you said, here's admin backends.
Here's authentication.
Here's caching.
And maybe under caching, there's like 10 choices on how you can do caching and plug that into your apps.
So if you don't know what you need or what even a good choice for caching is in Python, you're just like, hey, I know I need caching, but that's as much as I know, you know, drop into awesome-python.com.
And then you can go to the caching section and, you know, pick.
Yeah, so I think for me as a fairly senior developer, what I think these lists are most useful for is if you've already been programming in one language, say you're a Java developer.
Like I actually came from Java and then started in Python several years ago.
And I didn't know what the equivalent to a web application server was.
Well, that's a WSGI server.
And with the awesome lists, I can see at a glance, like what should I be using for caching in Python?
So if you were already familiar with all the conceptual ideas, like a task queue, like a web framework, then you can see the implementations in a different programming language.
So you could go to the awesome Ruby repository and see the equivalents to the Python projects that are out there in case you just happen to want to learn Ruby or you're, you know, on the job and you have to suddenly write some Ruby code.
Yeah, I think exactly.
If you know I need caching and you want to say these are the five things I should probably research and you can compare them pretty quickly because, you know, you'll open up the GitHub page.
You can tell, does this have five stars or 5,000 stars, right?
How active is it?
What are the deployments?
Things like that.
It's really good for that.
If you totally don't even know what you need to look for, you know, maybe something like full stack Python makes more sense, right?
You want to talk about how you see those differently?
Yeah, so there is a there's a bit of difference between an awesome list and what I'm trying to create with full stack Python, which is the awesome list is really a laundry list of all of the open source code that is out there that might be applicable to your your project.
But again, going back to task use, for instance, if you're a junior developer or you're in a boot camp or you're just starting to program and you don't know what a task queue is, well, seeing a list of implementations really is not going to help you.
It's not going to sell re RQ and these different implementations.
You don't know what they're going to do for you.
So that's why I have like a task queue page that explains the conceptual idea and then explains the differences between the implementations.
So I think what's missing from an awesome list is the context.
But if you're a more senior developer, you've already been working in one ecosystem and you're trying to switch over to another one or you're just trying to see what other implementations are out there compared to the the implementations you've been using that.
I mean, it's great for that.
Doesn't it just kind of gets past the the verbiage and gives you all the implementations that you need?
Yeah, I find having been a programmer for a long time.
I find these lists really, really helpful for discovering things that, you know, maybe they're not at the top of Google, but actually turn out to be just the thing that I need.
So but yeah, it's when you're totally new, maybe maybe it's not the best place to start.
But still, even if you're new, go check out the project.
It's you'll find it interesting, I think.
So you and I chose some things out of this awesome Python list that we thought would be fun to highlight as part of the show.
Yeah, my criteria for this.
So we picked 10 libraries at random that are listed in the awesome Python.
And by the way, Talk Python to Me is not one of the 10 we chose, but it is listed on an awesome Python as it should be.
Thank you.
And so is Full Stack Python.
And so the criteria that we used were projects that were not completely obvious to the community, but that are almost everywhere, or that have like a ton of potential to really impact the Python ecosystem.
So for example, we didn't choose Django because Django is talked about widely and used widely in the Python ecosystem.
But we did pick a CMS that is powered by Django under the cover.
So the CMS runs on top of Django, and we'll get to that in a few minutes.
So that's sort of the criteria we used to pick 10 projects at random that we wanted to highlight because they're important to the Python ecosystem, or they have potential to become really important in the future.
I think that's really interesting and really great because it's while we could do a show where we just go and say, let's just go find the 10 most downloaded packages on PyPI.
That would actually be pretty interesting.
But I think here we're going to hopefully expose people to things that are like, oh, I didn't even know that existed.
That's going to be awesome.
I'm going to go check it out.
So let's see if we can deliver on that, huh?
Exactly.
All right.
You want to dive in?
Yeah, let's get started.
So the first thing that I'm really happy to see that's on our list is something that helps us move from legacy Python to Python, or as some people say, from Python 2 to Python 3.
Sure.
Yeah.
So this is the library 6.
So 6 is a Python 2 to 3 compatibility library.
And this is one that you're not necessarily going to build an entire application around, but it is incredibly useful for developers who have built a Python 2 library.
And now they want to support both Python 2 and 3.
So you can think about it as a bridge.
6 is a bridge from Python 2 to Python 3 and allows you to support both ecosystems at once.
So it's really providing utility functions to smooth over some of the differences between the Python versions.
And the goal is really to have that Python code that you're writing for your own library be compatible with both Python 2 and Python 3.
Right.
So where there's some kind of difference versus, you know, like raw input versus input or bytes versus stir, things like that.
You can use this library and it will know which version of Python it's running on and it will, you know, switch between them.
Right.
Yep.
Exactly.
So if you're using Python 2.6 or above Python 2.6, 2.7, and then you want to support Python 3, for example, the syntax compatibility.
So we've got print functions that have changed.
We have other tech, the way that Unicode is handled.
All those things need to be either smoothed over in your own code or you can just use 6 as a dependency.
And it's only one Python file, which is awesome.
So you can just copy this into your project.
And then this helps you to smooth over those differences in that would otherwise come up because you're trying to support both 2 and 3.
Right.
And hopefully you can do this without having if Python 3, this code, if Python 2, that code.
But right, just leveraging this library and presumably somewhere deep inside of it, it's doing those types of things for you.
So you mentioned some of the areas that you can help with.
What else does it cover?
Where am I to use this?
For example, if you have unit test assertions in your code or you're renaming modules and attributes.
So there's actually a huge laundry list of areas that you may not be that intuitive if you're a Python developer, that if you're building a library for others to use, sort of it just crops up as you're looking to port from 2 to 3.
And that's really where this would come in.
Yeah.
Okay, cool.
Yeah, I remember there's like a module, something like copy underscore reg in Python 2 and it's copy reg without the underscore in Python 3 or vice versa.
You know, things like that, right?
Like, why are these so just barely different?
This is crazy.
Anyway.
Okay.
So, yeah.
So if you've got some Python 2 code that you want to bring forward or maybe you can even use it to help somewhat bring a Python 3 libraries to support Python 2 if you're writing some kind of package, right?
Yeah.
And the reason why I picked this one to highlight was it's really everywhere.
Many of the really popular, most commonly used Python libraries rely on 6 library in order to smooth over the differences in Python 2 and 3.
And so it's sort of the guy behind the scenes who's helping everybody out, but no one really talks about as like the main person.
It's the guy behind the curtain.
Yeah.
Okay.
Awesome.
Yeah.
So check it out, 6.
And it's named 6 because 2 times 3.
Yeah.
There you go.
Nice.
Let me take just a moment and tell you about an offer from one of our sponsors, Continuum Analytics.
They're hosting AnacondaCon 2017.
It's the first conference for open data science leaders around the world.
This is the definitive gathering place for the Anaconda crew.
Whether you're a new or a long-standing member of the data science community focused on business or technology, AnacondaCon will help you conquer your biggest data science challenges.
Over 20 industry experts will discuss the open data science movement and share their best practices for leveraging Anaconda.
You'll hear how they're discovering cures for rare genetic diseases, performing city planning, and refining tax policy analysis with Python.
Team Anaconda wants to help empower your whole team, so they're offering a two-for-one ticket price to AnacondaCon 17 from now through January 31, 2017.
Register today at talkpython.fm/acon to take advantage of this excellent offer.
That's talkpython.fm/acon.
Thank Continuum for supporting Talk Python.
They help make the show possible.
Now back to the interview with Matt.
There are a lot of data scientists and actual straight scientists that listen to this show, and they might be happy to hear about your number two item here.
Yeah.
So I picked Boca, or you can also pronounce it Bo-kay.
In the official FAQ for the project, it says you can pronounce it either way.
I'm not sure if there's a more right version, but we'll call it Boca for now.
And so what this allows you to do is write Python code and create data visualizations.
And so I have tried to create data visualizations in many different JavaScript libraries before.
The most popular right now, it seems like, is D3JS.
And I love those libraries.
But when I'm in Python land, sometimes I just, I can't think about switching over into the JavaScript world.
And so a Boca allows you to just write the Python code, and then it will build the visualizations for you.
Now, it's not relying on D3JS, but it draws visualizations that are very similar to that.
And so you're not just getting like a cut rate data visualization.
This is actually building something that is like, you can include it in your web application, and it is nice and professional.
Yeah, this is really, really a cool library.
And if you're already doing some kind of scientific computation or numerical computation in Python,
you don't want to like drop out and just, you know, drop a bunch of data into a JavaScript file or to an HTML file
and then write a bunch of JavaScript D3 to actually turn that back into something visual.
You can basically do all the visualizations in Python and then just plug it in.
And what's really cool is it's not just like generating like a picture, say, with Matplotlib.
Like, so with Matplotlib, you can say, here, I'd like to have this picture, boom, save it to an image,
then put the image on a page.
But these things, these bokeh graphs and visualizations that go up there, they're totally interactive, right?
Yeah, exactly.
So it's not just, as you said, it's not just an image that is generated.
It is the actual visualization that can be, that is fully interactive as well.
And so you can control how much interactivity and what data and what axes you want to, you want the user to be able to explore.
But you can do that all in, in Python itself.
And I actually, I found out about this project a couple of years ago at PyCon 2015.
Sarah Bird gave an awesome talk where, and you can actually go, go find it on YouTube for, if you search Sarah Bird PyCon 2015.
And she drew an entire map with just bokeh.
And I was like, whoa, you just, you just wrote, she live coded some Python code.
And I was like, whoa, you've got a map on there with all your data.
That would have taken me a week in D3JS.
I was just so impressed with it.
Yeah, it's really, really cool.
Okay.
So, you know, another thing that's really popular in Python these days, it seems, is static site generators, right?
Yep.
That well, and, and, and Pelican, which is a static site generator, which is not the project we pick, but Pelican, I use every single day to create full stack Python.
So I picked a separate one, and this one is either pronounced make docs or mk docs.
I actually just gave a talk on static site generators at San Francisco Python, and I tweeted at Tom Christie, who's the, the creator of not only Django rest framework, but this project as well, make docs.
And I said, hey, how do I, I'm going to check myself before I wreck myself.
How do I pronounce this project?
And he said, well, you know, I, I say mk docs with a Scottish accent.
And I was like, mk docs.
Okay.
I don't, you know, I'm not, I don't really have a Scottish accent.
So, okay, sure.
And then he said, but, you know, I don't really know.
Let me, let me talk to one of their core committers.
And he said, mk docs, that's how he pronounced it, but they don't really have a particular opinion about it.
So, you know, whatever's easiest for you.
If you've got a great Scottish accent, I say go for that route, but, but I, I certainly do not.
So what make docs is, is a static site generator for really for project documentation.
Let me ask you a quick question or clarification just for everyone listening, because many people would know, but not everyone.
What's a static site?
Yeah.
So a static site is kind of taking you back to the origins of the web.
It's just HTML flat files, HTML, and it has CSS and usually JavaScript, but there's no database backend.
There's no dynamic content on the backend.
So it's really just files that can be served up by a web server.
And the reason why it's called a generator.
So that, that would be the static site.
The generator part of it is you write your content either in markdown or restructured text or some sort of markup format.
And then you have templates.
So for Pelican, for example, Pelican has Jinja templates that you can use.
So Jinja is a template engine and make docs has its own, actually make docs makes it even easier.
You don't even have to write the templates.
If you don't want to, you just write the content.
And what that does is it combines the content that you've written with some sort of template engine and then produces HTML files or some sort of output.
It could be XML, whatever you want, but most commonly you're producing a static site, which would be an HTML.
Okay.
Yeah, that's really cool.
And there's a lot of benefits to that.
It's super easy to scale a site like that, right?
You don't have to worry about the databases and concurrence execution.
You're just serving static files.
And it's also easy to put that on to like a CDN sort of thing, like Fastly or something, right?
Yeah.
So the way that I deploy a full stack Python is it's just, you know, it's just a bunch of HTML files that get generated with some CSS and JavaScript.
And then I just have a content delivery network.
So I have the files hosted.
Sometimes I use GitHub pages.
I've also switched it over to Amazon S3.
In front of that is a CDN, and I use Cloudflare.
And what that does is it just serves up all those files.
There's no database calls or anything like that.
It's just really fast, immediately serving up those files to whatever web browser is requesting them.
Nice.
And it serves them up.
It moves them around the globe and serves them up from somewhere by place, which is pretty awesome.
So people on the other side of the earth don't suffer because you decided to put your server in New York or whatever.
Globally distributed, exactly.
Yeah, yeah.
That's awesome.
Okay.
So what's cool about MakeDocs?
Why is it a great one?
How does it work?
I think the thing about static sites is they've traditionally been made around blogs.
But project documentation is one of the best examples that I think, if you want to use the word, use cases that I've seen.
So the idea behind MakeDocs is it's already being used to power a really popular project, a Python project, which is Django REST framework.
And that allows MakeDocs to have a real project using it.
And then they open sourced it so that anybody can use it.
And the idea is generate beautiful project documentation.
Now, for many developers, documentation is sort of not the first thing on their mind.
But in the Python community, I think we have really great documentation for most of our projects.
It's kind of something that is a badge of pride.
And so that's where I think using MakeDocs and just writing your content, you don't have to worry about the styling if you don't want to.
It's just going to create a beautiful set of project documentation for you.
Yeah, that's excellent.
And I think, you know, you could even check in the docs into, like, your GitHub repository.
And if somebody clones your repo, they can just go, like, right there and just have offline docs as static files.
That's really cool.
Very nice.
Okay, MakeDocs.
You know, speaking of Markdown, I think having Markdown wherever you accept user input is really nice in your web apps.
You know, Stack Overflow definitely popularized this.
Markdown comes from John Grubner a long ago, I don't know, like 2008 or something like this.
It's not a brand new idea.
But certainly using Markdown to accept input from users is much safer than accepting pure, you know, rich formatted HTML, right?
Yeah.
So the thing about Markdown is it's a certain format.
And there's, well, for better or worse, without getting too into the weeds, there are different variations on that format.
But there is sort of a standard Markdown format.
And there's a Python project that parses that out.
So then you can work with it in your Python code.
And so the Python Markdown project is kind of everywhere.
Whenever you're talking about a content management system or a static site generator like MakeDocs or Pelican, which I use.
Really, when you do pip install Markdown, what happens is that a Markdown parser is installed and allows the whatever libraries you're depending on to then parse out that Markdown and work with it in their own code.
Yeah, that's really cool.
So, for example, on like Python Bytes, the new podcast that I started, so Python Bytes.fm, on there, there's a back-end CMS type thing that I created for it.
And there's a place we put the show notes.
And on Talk Python, that was just an HTML fragment.
But every now and then I would mess up that HTML fragment.
And it would screw up the site.
Or, you know, I just have to like format HTML in the browser and write it by hand.
And it's not so fun.
So when I did Python Bytes, I'm like, that's it.
This part is going to be input as Markdown, output as, you know, HTML.
And so I use something similar to Python Markdown for that, which is really cool.
I actually use this thing called Python-Markdown2.
I don't think they're related to these two things.
It's nice because it has extensions.
You can add on like variations.
Like I want to add like GitHub table support type things and stuff like that.
Yeah.
Again, this is just another one of those libraries that's everywhere.
And whether they're using Python Markdown, Python Markdown2, or there's many other parses that are out there,
they're kind of the guy, the gal behind the scenes that is doing a lot of the heavy lifting
and allows other projects to take advantage of that work that's done.
Nice.
So while we're on the topic of, hey, I want to put some stuff into some kind of back end on my site and have it show up,
one thing you do is write a site from scratch and fill in a few pieces like show notes and episode names,
or you could grab a whole CMS, right?
Yeah.
So Django can often be used as a CMS, which is a content management system.
I would say like WordPress is probably the most popular of these, right?
If you look at like overall deployments or maybe Squarespace, but plug it into Django is really sweet.
So you found a really nice one.
What happens is if you start building a site on Django, and particularly if people that are not developers are using it,
you often find that you run into limitations.
And that's not, I wouldn't say that this is necessarily something that Django has to fix.
This is where a content management system comes in.
So let me give you a specific example.
If you have people that are not developers that are punching their content into this content management system, like Django and the Django admin panel, for example,
and then you want to roll back, you say, oh, shoot, I deleted, I deleted a page.
You know, let's roll back.
Well, Django doesn't have that type of capability.
And that's where content management systems are on top of that.
So the one that we picked here is Wagtail.
And Wagtail is actually a newer content management system than some of the other ones like Mezzanine or Django CMS.
But Wagtail is, I've been really impressed with Wagtail.
I've used it a bit, both as a non-developer user to punch content in and a little bit on the back end as well.
And it's been really powerful so far.
Yeah, that's really great.
And they've got a great example.
They have like a demo video for it as well for Wagtail.
And it was used at the Royal College of Art.
It's in the UK somewhere.
And it basically powers that website with thousands of web pages and many, many users and all sorts of really cool workflows.
It's actually quite visual and pretty.
One of the things that comes to mind when I think of CMSs is they seem synonymous with slow and clunky to me.
There's a tradeoff with CMSs, which is that they're building all this extra functionality on top of an existing web framework.
And they need to be used by people that are not developers.
So you don't want to get into a situation where the CMS is preventing you from doing things with your content that you want.
So there's a tradeoff.
Sometimes CMSs are too heavy-handed.
Sometimes they're just really confusing to use.
But I found that Wagtail, which we're actually using at Twilio, is actually one of our favorite CMSs.
And we did a large evaluation with a lot of different programming languages.
And this is the one that came on top for us.
It's not necessarily the best for everybody, but it's the one that works really well for us.
I can say the Royal College of Art website is quite zippy, even though I'm like nine hours time zone away from it.
So mostly any sort of slowness would be ping time.
So I feel like this is a CMS that doesn't have that feeling, right?
It's really nice.
And it's awesome because you're using it at Twilio, too.
Well, the thing about Wagtail is that they've done a great job with making it really polished.
And I think that's one of the reasons why we were so impressed with it.
And so this is what's powering the Twilio docs.
We actually look at our documentation, the documentation for the Twilio APIs as a platform itself, because many different teams around Twilio are using the docs.
They're writing docs.
They're updating the docs when things need to be fixed.
And so we needed some sort of powerful platform that allowed many different teams to interact with it.
So after our evaluation, we've used Wagtail as our CMS for the Twilio docs.
Yeah, it's really cool.
I'm looking forward to finding a place to use it because it looks like it's fun to use.
The back end looks real friendly and modern, not like some crummy sort of app.
It's nice.
It's a really nice and polished project, really.
I think a little bit newer and up and coming as a CMS.
Awesome.
And it's cool that it was on the awesome list.
Okay, so another thing that I think is really interesting is design patterns.
I've always loved design patterns.
You know, I've read, I remember reading the Gang of Four book about C++ and small talk design patterns like way long ago.
And then reading it again so I could actually understand it like a few years later.
But one of the things that I think is interesting is if you take those patterns and you apply them to Python, sometimes they look very different and sometimes they look very similar.
Like, for example, the decorator pattern.
You know, that has one form of C++, but it's almost a language feature in Python.
Yeah.
That's where the, on the awesome list is the Python patterns repository.
So there's a link to Python patterns.
And this is the implementations of various patterns, whether it's like behavioral patterns or structural patterns, different conceptual ideas.
And they're implemented in Python code.
And they're implemented in Python code, which allows someone, for example, when I was going from Java and I had constantly use the singleton pattern or the factory pattern to then see what that looks like in Python.
I think it's really great for people who are coming from another language.
They already have these ideas in their mind, right?
They can drop in really quick and you just pull up like, you know, 20 lines of code and you're like, oh, this is how you do it in Python.
I never would have thought to do it this way.
This is so much better than what I was trying to do a minute ago.
And they have some interesting ones in here.
They have the standard gang of four ones like state, strategy, iterator, visitor, and so on.
But then they also have some things like the Borg design pattern and the Blackboard design pattern.
So I actually learned a few things.
And I don't really know yet where I'd use the Borg, but I'm pretty sure I like the name of it.
This repository is great for learning.
I would caution anyone that is coming from another programming language to suddenly drop different patterns into their code when those patterns may not be idiomatic to the way that you would write Python code.
Because Python code has its own way of a different way that you would write Python code, for example, making everything explicit rather than implicit like you would in the Ruby ecosystem.
So I think these are awesome for just learning about how different patterns could be implemented.
But I would caution people to make sure that this is actually how you would write Python code as opposed to just dropping them into your code without thinking about them.
Yeah, I kind of have like an Einstein type of philosophy about design patterns.
Like design patterns are awesome.
And you should use them in your code.
But just enough, right?
Things should be as simple as they can, but not simpler, right?
Like you should have patterns that solve problems.
But I feel like people, when they learn design patterns at first, they're like, oh, I'm going to put patterns everywhere.
I can't wait to like turn everything into a state pattern.
And then the visitors go in here like, oh, no.
Oh, no, that's going to be unmaintainable, right?
And so patterns, I don't know, to me, they're like salt, right?
A little bit here and there.
Yep.
And if you find yourself with a specific problem in your code, knowing what those patterns are and how they can be implemented in your programming language can be really powerful.
Yeah, absolutely.
You know, another thing that I'd like to lean on to some kind of framework for is authentication, especially things like OAuth.
Those are super, super complicated to figure out yourself.
But luckily on our list here, we have some OAuth stuff, right?
Yeah.
So the project we picked here is Authomatic.
And the tagline here is simple but powerful framework authentication and authorization client.
I think there's a spectrum with authentication libraries.
On one hand, you've got sort of these maybe heavy handed.
They tell you exactly how to implement authentication models.
So for example, OAuth.
Let's say you want to have a Twitter-based login or a GitHub-based login.
Some of the authentication authorization frameworks will tell you, hey, in these three steps, add this to your code, and then you'll have OAuth login.
But on the other hand, these libraries have to work with various frameworks.
So the way that you would implement something in Django versus Flask versus Pyramid versus Turbogears can be different.
And that's where I think taking a look at all the different libraries can be helpful.
So Authomatic seemed like a really great one because it had examples with different frameworks, which I really liked.
Yeah.
You can definitely go in there and it's like these are the few things you drop in for Django.
Here's the few things you do for Flask.
Here's what you do for Pyramid.
Even one for Google App Engine.
And I really found that some of the other ones that seemed comprehensive had almost no examples.
And you're like, but there's all these moving parts.
How do I put this together in this particular web framework?
Like, why is there no example here?
Yeah, I looked at a number of them.
And maybe we should put a little word of caution in that this hasn't been updated for a little while.
But that's kind of true with some of the other options out there as well.
So they kind of seem more steady state or they're in some sort of flux or something.
So look through the whole list.
Here's just one example to sort of point out.
Like, there's a bunch of great things about OAuth and social auth and things like that.
Yep.
And this makes it much easier for you to implement them rather than having to rewrite everything in your own code.
Yeah, for sure.
This portion of Talk Python is brought to you by Hired.
Hired is the platform for top Python developer jobs.
Create your profile and instantly get access to 3,500 companies who will work to compete with you.
Take it from one of Hired's users who recently got a job and said,
I had my first offer on Thursday after going live on Monday and I ended up getting eight offers in total.
I've worked with recruiters in the past, but they've always been pretty hit and miss.
I tried LinkedIn, but I found Hired to be the best.
I really like knowing the salary up front.
Privacy was also a huge seller for me.
Sounds awesome, doesn't it?
Well, wait until you hear about the sign-in bonus.
Everyone who accepts a job from Hired gets $1,000 signing bonus.
And as Talk Python listeners, it gets way sweeter.
Use the link Hired.com slash Talk Python to me and Hired will double the signing bonus to $2,000.
Opportunity's knocking.
Visit Hired.com slash Talk Python to me and answer the door.
You talked about static sites earlier.
We mentioned the deployment and the performance story and all that, and that's great.
But sometimes you just need a data-driven web app, right?
Like, for example, it's super hard to go to, like, a static site and add voting or comments or things like this, right?
So maybe the thing that you're building, you really do need to have some kind of data-driven thing here.
But if it's not fast enough, what are you going to do?
Well, you've got to have caching.
So, you know, caching is one of these really hard problems in computer science.
So taking advantage of a library that allows you to implement caching can be really helpful.
And I know this one you picked, Beaker.
Why don't you tell us a little bit about Beaker?
Yeah, yeah.
Beaker looks really awesome.
So Beaker is a way in which you can cache binary representations, pickled versions of Python object graphs.
So you just take the thing you want to save and you just stash it in Beaker.
And it can go into all different places, right?
It can go into databases.
It can go into memcached.
It can go into just be stuck in memory, you know, until your process reboots.
You can go straight to the file.
And it also has a SQLAlchemy plug-in.
So you point SQLAlchemy at the database that you care about, right?
SQLAlchemy supports all sorts of different databases.
And then it just talks to SQLAlchemy.
So pretty much any type of database you want to use as a back-end for your cache, if you don't want to put it in memory, you can do it here.
So there's all sorts of cool stuff.
Like you can take it and plug it in as middleware in WSGI servers.
So it automatically has little plug-ins for pylons, pyramid, turbo gears, things like this.
Yeah.
I think this could be really helpful for people who need that one or two slow pages to go much faster.
Yeah.
And if you can't build your site as a static site, certainly you're probably going to investigate caching at some point in order to improve the performance.
So Beaker seems like a really capable one to use.
Yeah.
It definitely does.
So some of the features they list are like it's fast, obviously.
That's a requirement for a cache, right?
But it has multiple reader, reader, writer, block performance.
So it avoids duplicate creation but allows for high parallelism for reading.
All the back-ends, like I said, you can extend it and write your own back-end if you want to plug into TinyDB or, you know, whatever.
All that kind of stuff is really cool.
Yeah.
Sounds like a good one to check out.
So another one that you picked, which I think is really smart because people always say, hey, don't roll your own crypto.
And you don't want to write your own password storage hashing library unless you're just trying to learn how it's done.
But you don't necessarily want to write something for production.
But you picked Passlib, which you've used in the past.
Why don't you tell us about Passlib?
Yeah, I use Passlib all over the place.
I use it on the training website, for example, for accounts there.
And I don't want people to end up on, you know, some sort of news story.
Oh, a development server with production data was compromised.
And here's, you know, a quarter million accounts.
And too bad they used poor, like an old outdated hashing algorithm or something like that.
And it turns out that everybody's going to have to reset all their passwords on the internet everywhere.
So Passlib, I hope, will help people avoid that fate.
The idea with Passlib is it takes all the best practices around password storage and verification and validation and puts them into two functions, encrypt and verify.
So if you can't remember to call encrypt and verify, you know, you probably have no business writing authentication code, right?
But the idea is they provide a whole bunch of different caching algorithms or, sorry, hashing algorithms.
And they only support ones that have no known vulnerabilities, are public, have public reference implementations, can be made to be computationally expensive, you know, have a lot of scrutiny and things like that.
So Argon2, Bcrypt, SHA-512crypt, things like that.
So you're already starting from, thank goodness they didn't use MD5 for the hashing, right?
Which is one of the problems.
The other problem is with the advent of graphics, the GPU processing, people can use the GPUs to do advent of computational GPU processing, not for graphics.
They can use the power of those graphics cards to just try so many variations on these hashes because the graphics cards can do them really quickly.
And so you want to make the stored passwords computationally difficult to guess at.
So instead of just hashing with a strong algorithm, Passlib will take it and it'll hash it like 200,000 times.
So it'll take the input, stuff it back around, hash that thing, and then just, you know, fold it over 200,000 times.
And it also adds random salt to each thing it stores.
And, of course, folds that all through.
So it's just super hard to guess what the possibility is.
But it's easy for you.
You get back some kind of password hash.
You call verify with the plain text password and either it matches or it doesn't.
Yeah.
Well, it seems like it provides a really good tradeoff between, you know, developers want to spend a least amount of time working on security.
And so they have more time to work on features.
And it seems like this is making a good tradeoff where it's, hey, this is really simple.
Here's what you do.
Call these two functions and we'll take care of this bit for you.
So that seems like, as far as a library, that's probably some of the best you can ask for.
Yeah, it's really cool.
And you can actually evolve over time.
So, for example, part of the thing that gets stored in the database has the number of rounds or the number of folding iterations you've done.
And so as computers get faster, you can actually crank that number up over time without breaking the old passwords.
And then when the user comes back, you just regenerate with a newer, more computationally expensive variant.
Nice.
Yep.
Also supports two-factor authentication, but I know nothing about this.
You could use Authy for that or implement with SMS via Twilio.
That's my little Twilio commercial for this one episode.
Oh, awesome.
Very cool.
Okay.
So another thing that's cool, well, something that's just awesome about Python, the whole ecosystem, is really pip and the deployment and packaging story, right?
That I can just go grab all these awesome packages and, you know, 95,000 others.
Yeah.
So the fact that we have PyPI as the central repository for all of our dependencies and it's really easy to upload your own libraries to the central repository.
As a Python developer, it makes things so much easier for me.
When I was a Java developer and I used to use Maven, Maven was incredibly complicated.
You have this huge, you have these massive XML files.
It worked, but it was just, it was really complicated.
And then I came over to Python and you've got a requirements.txt file.
And requirements.txt, when I read it, I thought, this is so beautiful.
I literally thought it was beautiful.
There, you have the name of the library that you depend upon.
You have, and then you optionally have the version number that you want to use for that library.
Now I say optionally, but really it should be required.
Before I joined Twilio, I did a lot of Python consulting and it was my nightmare when I would get a project and either would not have a requirements.txt file.
You know, this is something that had been running in production for a few years and they said, yeah, we threw all this stuff together.
You know, we use Django.
And I thought, okay, Django, that's great.
Which version?
And they're like, well, we don't, we don't know.
And so, so we've got, we luckily have now some tools that help out with figuring out what those dependencies are.
So pip-tools is a set of tools, not only to keep your, your pinned dependencies fresh, but also to kind of infer what some of the requirements are that you may, you may have.
What's great about this is if you don't necessarily know what the exact version numbers are of all of the, of all of your dependencies, you can use the, some of the pip-tools in order to infer some of those things.
So if you, for example, like Django, which version of Django is at 1.4 is a 1.10.
This will help you to figure that out.
Yeah, absolutely.
Yeah.
And pip-tools will actually generate the full dependency graph for you.
So one thing you might say in your requirements file is like, I require flask and maybe even say I require flask 0.10.1 or whatever it happens to be.
Right.
But flask itself requires it's dangerous and Jinja to and markup safe and these types of things.
So you can go to pip-tools and you say, run pip compile, and it will actually generate a requirements.txt.
That has the sort of closure of all of your dependencies, the dependencies and their dependencies and so on.
So you can see and pin every single dependence you have.
And it also adds a little comment like these dependencies were added because of flask.
This one was added because of Jinja, which was added because of flask, things like that.
I think that was really cool.
Yeah.
Yeah, definitely.
And just incredibly handy.
If you get dropped into a project and you have no idea what the exact version numbers and requirements and dependencies of dependencies are.
Yeah, absolutely.
And the thing that I use actually in production for this kind of stuff, it's not the same, but I use pyup.io.
I met those guys through the show a while ago and I'm like, wow, this is such a cool project.
Pyup.io.
And you basically, it's like an online system.
You point it at your GitHub repo, public or private, and it will watch this file.
And if there's changes or releases or especially security releases, it will like send you a pull request saying there's a security vulnerability and it's dangerous or something.
Here's the updated requirements.txt to fix it in your app.
Wow.
That is incredible.
That's super useful.
Yeah.
It's the notification.
I mean, these things are changing all the time, but it's like the push notification to you that your app now has like an important update to its dependency, which I thought was really cool.
Awesome.
Okay.
Well, those were a great bunch of libraries.
Hopefully you guys learned a lot about them.
Matt, it was fun to explore these with you.
I didn't know about all of them before we started talking.
Well, I don't think anybody knows all of the Python projects that are out there.
You wouldn't get any coding done if you knew what all the projects were.
So that's what I think is great about the awesome lists is you get to see at a glance just different libraries that you may never have known about.
Yeah.
I totally agree.
And that's why I think it's so interesting to bring it up for people is because you can drop in there and really quickly learn about all these tools that you didn't know about a few minutes ago.
Very cool.
All right.
So as always, before I let you get out of here, let me ask you the two questions.
Okay.
So I don't even actually remember your answer from a year and a half ago.
So when you write some Python code or any code, really, what editor do you use?
So I use Vim.
That hasn't changed.
It's been Vim for a fairly long amount of time.
Now, to be fair, I use Vim for Python code.
Most things I do.
I use Xcode for my Swift code and I use IntelliJ, the ultimate IntelliJ edition for Java code.
So I've been doing some more Java coding lately.
So I kind of bounce back and forth.
But I almost always use the Vim mode in whatever editor that I'm in.
Yeah.
That's probably a requirement for you, right?
That it has a Vim mode.
Yes.
Nice.
All right.
And we kind of touched on a bunch here, but you want to pull out a favorite PyPI package?
Oh, man.
I'm going to, I got to, you know, I got to give a shout out to Pelican.
I just, as a static site generator, I think when you use a tool, and I may have said this
on episode 26, but when you use a tool for six, seven years and you still love it, I think
that's a really good, really good sign.
So Pelican is that tool for me.
And it's a great static site generator that powers full stack Python.
Yeah, that's awesome.
I think that is a really good sign.
If you love it after you've lived with it for a while, then it must be something special.
Very cool.
So I feel kind of like as a final call to action, I noticed there were some projects not listed
there that I thought were awesome.
You look in various, some of the categories, you're like, oh, why is this not listed here?
So maybe if you're out there and you own a package or you're really passionate about it
and you don't see it there, you think it's awesome, maybe go send those guys a message
and let them know they should add your package.
All right.
Well, Matt, it's been fun to talk to you.
Thanks for sharing these packages with everyone.
Yeah.
Thanks, Michael.
It's been a great time.
Yeah.
Great to catch up with you.
See you later.
This has been another episode of Talk Python to Me.
Today's guest has been Matthew Mackay.
And this episode has been sponsored by AnacondaCon from Continuum Analytics and by Hired.
Thank you both for supporting the podcast.
Whether you want to hear the keynote by Ryan Curran from Forrester Research, meet the guys
behind Anaconda, or just mingle with high-end data scientists, you need to find your way to
Austin, Texas for AnacondaCon this February.
Start at talkpython.fm/acon.
Hired wants to help you find your next big thing.
Visit Hired.com slash Talk Python to me to get five or more offers with salary and equity
presented right up front and a special listener signing bonus of $2,000.
Are you or a colleague trying to learn Python?
Have you tried books and videos that just left you bored by covering topics point by point?
Well, check out my online course, Python Jumpstart by Building 10 Apps at talkpython.fm/course
to experience a more engaging way to learn Python.
And if you're looking for something a little more advanced, try my Write Pythonic Code course
at talkpython.fm/Pythonic.
Be sure to subscribe to the show.
Open your favorite podcatcher and search for Python.
We should be right at the top.
You can also find the iTunes feed at /itunes, Google Play feed at /play, and
direct RSS feed at /rss on talkpython.fm.
Our theme music is Developers, Developers, Developers by Corey Smith, who goes by Smix.
Corey just recently started selling his tracks on iTunes, so I recommend you check it out at
Talk Python.fm slash music.
You can browse his tracks he has for sale on iTunes and listen to the full-length version
of the theme song.
This is your host, Michael Kennedy.
Thanks so much for listening.
I really appreciate it.
Smix, let's get out of here.
Outro Music.
Thank you.