What is the fastest, most scalable web platform out there?
Is it Pyramid running on top of MongoDB with the Redis cache?
Maybe Flask and Postgres as a service with some high-performance cluster?
Or some funky Go API framework?
No, it's static files.
But today, static files does not mean that you write a bunch of HTML.
With static site frameworks and generators like Lecter and Pelican,
you can use data to drive the creation of static sites
and then host them wherever makes the most sense for you.
On this episode, you'll meet Joseph Nix, who works on Lecter,
a Python-based static site generator.
This is Talk Python to Me, episode 160, recorded April 19, 2018.
Welcome to Talk Python to Me, a weekly podcast on Python,
the language, the libraries, the ecosystem, and the personalities.
This is your host, Michael Kennedy.
Follow me on Twitter, where I'm @mkennedy.
Keep up with the show and listen to past episodes at talkpython.fm,
and follow the show on Twitter via at Talk Python.
This episode is sponsored by Linode and Rollbar.
Please check out what they're offering during their segments.
It really helps support the show.
Hey, Joseph. Welcome to Talk Python.
Hi, thanks for having me.
Yeah, it's great to have you here.
I'm really excited to talk about Lecter and, more broadly,
these static site generators because they're such an interesting way
to build web applications that I think are really quite the opposite of
full-stack, data-driven, scale-out, this and that, right?
Yeah, definitely.
But they definitely have a huge place, a good need, I think.
Yeah, and I think Lecter lives in a pretty interesting space in that realm as well.
Before we get into it, though, let's start with your story.
How did you get into programming in Python?
My story is kind of fun, I think.
So I didn't start out wanting to be a programmer at all.
I grew up interested in physics and math.
That was all that I was about.
I wanted to be an astrophysicist since I was a little, little kid.
So I went to school at UAH in Rocket City, Alabama.
That's where they built the Saturn V.
And I wasn't taking any programming classes except for one that was mandatory.
It was a C++ class that I struggled through.
But I met some cool friends.
And for fun, we would spitball ideas.
We'd talk about cosmology and things like this.
And we ended up, again for fun, inventing a kind of rocket engine.
So I didn't see that coming.
It's a small thing, at least physically.
It's a kind of electric propulsion device, which I could talk about forever, basically.
But it's pretty cool.
And we researched it on paper.
And we decided to form a company to protect the intellectual property with aspirations of actually doing physical R&D and doing something with it.
Right.
So we did that.
And then we realized, well, okay, now we've got this company.
And we want to do rocket research.
And we have no money at all because we're poor college students.
Right.
There's no hiring engineers or things like that, right?
No.
We considered various ways to remedy that.
But what we decided was we looked at everybody else that was doing something similar.
There's all sorts of space entrepreneurism going around at the time and still now.
And almost all of these that we could find were founded or funded by people who made their money in software.
Right.
I mean, we've got SpaceX.
We've got the Blue Origin.
Yeah, Blue Origin.
Yeah, yeah, yeah.
Exactly.
And some probably other, a bunch of others we, I don't know about.
Virgin Galactic.
Yeah, yeah, exactly.
Richard Branson.
I mean, it's not entirely software, but a lot of it.
But absolutely.
People, all these software executives and founders, it seems like they maybe even had it in their mind a long time ago that this is what they wanted to do.
They really just wanted to go to space.
And they thought of some novel software applications that are useful in and of themselves and interesting in and of themselves, but will help them get to space as well.
That's really interesting.
So how did that get you into programming?
So we decided we needed to teach ourselves software so we can make some money to do this rocket research.
So that's exactly what we did.
So kind of from the very beginning for me, I was talking with my friends and we decided what we wanted to do.
We looked at the various programming languages out there from a completely fresh perspective because none of us were programmers really.
We looked at PHP, we looked at Fortran, we looked at C, Python, and we kind of felt that Python was maybe the way to go.
It seemed like it was becoming popular.
We read about why.
We experimented with a few languages and we liked Python.
So we dove in and it stuck.
It was very easy to pick up.
That's definitely one of its superpowers.
And also so many of the languages that are easy to pick up there that way because they give up sort of the professional high-end proper architecture.
And Python seems to live in this space where like you can start without all the structure and objects and separation and packages.
But it can grow into that as you need it, right?
So I think that's part of the secret sauce.
Yeah, definitely.
When we first started out learning Python, I was still having a hard time wrapping my head around OO.
You know, just doing very basic imperative and functional things to get our feet wet.
But it grew.
And I've been doing Python now for a decade and I feel pretty comfortable.
We'll get into the lecture and all this stuff.
I just want to ask you, given that story, did you watch Falcon Heavy launch?
Yes, I did.
Oh, my goodness.
It made me cry.
I thought that would be kind of a major thing.
But then like you watch it and it's just like, oh, my gosh, like people are still doing amazing stuff in space, even though the government seems to have given up on it.
Yes.
And that was really frustrating, too, at the time.
So we started learning Python back in like the mid 2000s, which was right before the economic crash in 2008.
And people, which is when we were starting to get good at Python, we were trying to be consultants, but it's hard to find a job.
People weren't really hiring then.
And all these spaceflight companies were struggling, too, because, again, everything was kind of suppressed a little bit.
And they were trying to get off the ground running with something aggressive in the first place.
But they've pushed through and they've done great things.
And I'm super excited to see where all where all of them go.
Yeah.
Watching those two outer rockets land and I guess the third one try to land was just it was like, oh, my gosh, we are in science fiction land.
But it is like happening.
Right.
It's just when I saw that, I'm like, that is something you would see in a movie.
That is not something you would see in reality.
No, it was just unbelievable.
I think they had a video where they tried to show what this will look like before it actually happened.
And they had the two rockets land separated in time.
And I don't know if they were planning on them actually landing just identically together.
My guess is they wanted that, but they weren't going to bank on it.
So they didn't want to get everybody's hopes up.
Boy, it just looked ridiculous.
Yeah.
I'm so looking forward to the next five years.
What's happening around there.
All right.
We could we could totally turn this into a space show.
But let's stay focused on the stuff that came after your project.
Right.
Just to sort of close it out.
Like, are you is that company still exists?
Has anything happened of it?
Yes, definitely.
We have grown in skill and we're now a stable company.
Our company is named Terminal Labs.
We're a small Python consulting shop.
We work contracting for other companies to do whatever it is they need.
In the past, we've done a lot of full stack web development.
We've worked with Django and Flask, some really big Django projects.
And we've also loved it.
Any chance we can get to do any science.
So data science and machine learning are really exciting.
And we're getting more and more into that all the time.
We've got a pretty strong DevOps background.
So all this is keeping us employed and raising our skill level further so that we can eventually
make a rocket.
Awesome.
All right.
So that sort of brings us to what you're doing today.
And that's really where you're still focused is sort of continuing to grow the
consultancy and stay and play and basically play in the science space when possible.
Yes, absolutely.
And making connections.
And yeah, our goal is to get to space and how exactly we're going to transition there.
Time will tell.
We've got some ideas.
But this has been a pretty good course so far.
And I think it'll work.
Very cool.
Glad to see that that's still going.
So let's talk about the project that you've been involved with sort of most broadly at first
with static site generators.
So many people who listen to the podcast are web developers, but a lot of people are scientists
or data scientists or even students.
So maybe just like could you compare and contrast a static site generator experience to like, let's
say, like a traditional web app.
You talked about Flask and Django, either of those.
Yeah, absolutely.
So with a traditional web app, you have a server.
There is a server on the cloud that you that a user will hit and make a query to.
And then that server does some logic and often connects with a database to dynamically generate
something to return to your browser.
And a static site generator doesn't need any of that.
And often I'd recommend against it.
Like you could put it behind Flask, but there's really no need.
The build products of a static site generator are basic HTML.
The tradeoff, like, you know, it seems at first like so the tradeoff is I could just create a bunch of
JS, CSS and HTML files and just write them.
But that becomes super tedious super quickly, right?
Like if even if you just want the same look and feel across multiple pages, right?
Like if you've got 20 pages like that and somebody comes in, either boss, client, whatever it says,
I would love for that background over here to be blue and the font to like reorder the menu.
That's not hard, right?
But if you actually hand code it, like it sucks, right?
You've got to go and change all those.
But on the other hand, if you've got to maintain databases, scalability, security patches, etc.,
that also can be painful.
Sometimes it's totally needed.
Like for my site, like it has to be dynamic.
It absolutely needs a server and a database.
But many sites don't, right?
Absolutely.
The history of it is, is we started with those raw HTML and CSS.
And that was absolutely a pain.
You're completely right.
I hate coding CSS.
But the reaction to that was to build these very complicated web servers.
And not everybody needs that either.
So these static site generators came around as a bit of a compromise for people who don't want to build these basic HTML sites with CSS by hand
because that is enormous pain.
But they don't need the power of an actual logic backend in a database.
All they really want is to make their lives easier creating the static content.
And things like Jinja really help a lot with that and web templating.
Right.
And so, for example, like you probably still want some kind of data-driven site created.
Like you could have a blog and you want to add a new entry and have that appear and be linked to possibly under various categories and stuff.
But you want to be able to – like I guess the thing is you don't need to show a different version of the blog to anyone, right?
It's totally fine if they all get the same one.
So instead of doing that at runtime on the server over and over but always giving the same result, like just crank out – like somehow on your machine at a like a move to production time, you generate that.
And then you just put the static files there, right?
And it ends up being really nice in production too because all these files are pre-built.
So all that you need to do is put the HTML on a CDN or something and then they get served incredibly quickly.
So it's very performant.
So from a production standpoint, it's faster to do this for the user than to actually query, for example, a Flask backend that is just doing the same thing.
Just generating a simple HTML for everybody and giving it out.
I guess you can even mix them, right?
So suppose you've got some section of your app that is like a store.
Obviously, you would like people to – that to be dynamic.
But the rest of it, maybe not.
You could use something like Nginx and like URL routing to say this part routes to like a Flask app or Pyramid app.
This part routes to the static content.
You can definitely do that.
Nginx is a great way to segregate those sorts of things.
So you can have a static site generator made with Lecter, for instance, and a Flask part of your website that is dynamic content, but not the blog because the blog doesn't need that.
Not the flat landing pages because it doesn't need that.
And you could even go and use JavaScript on a static page to query dynamic content if you wanted.
Right.
If you wanted to go down the sort of rich front-end web framework type thing like Angular, React, or whatever, that could be all fully static.
But then like maybe there's a Flask API, for example.
Exactly.
So you could even shimmy in some dynamic content into a static site.
And how easy that is depends on, of course, how comfortable you are with all those technologies and the complexity of your dynamic content.
So the more complicated that gets or the more comfortable you are with Python instead of JavaScript, maybe the more inclined you should be to use Flask or Django.
Yeah, that makes sense.
So maybe give us some examples.
We talked about Lecter.
Let's take one step back.
One thing that I think we did, obviously we touched on the deployment and simplicity.
One thing we haven't touched on is just the performance of static sites.
Obviously, you talked about the CDN and that's good, but the consequences of high-performing sites are really important.
Right?
I think Amazon did a study saying for every 500 milliseconds of latency they add, they lose 1% of sales or something like that.
It's a big deal.
Yeah, and I'm sure it's not linear.
You add that fifth bit and it's like eventually people are going to go away and it's going to become nonlinear.
But that's important, right?
Yeah, that margin is very important for most businesses.
And even if you're just trying to make your own personal blog become more popular, it matters to you if your user base grows by 1% or 10%.
It matters.
And it also, I think it feels different when you're on a site that you're like, dang, that is basically instant.
Like that feels almost instant to me as I work with it.
Whereas the other one, like if anyone has worked with SharePoint, they know what the opposite of that feels like.
Right?
It's just like slow, painful.
Right?
I'm waiting again.
Like it just, even if it doesn't make people go away, it engenders a different experience with your consumers or your people.
Absolutely.
It's got an effect on your mood.
Even if it's subconscious, even if it's slight, it makes you a little less happy, a little more agitated if it's slow.
This company doesn't really delight me, right?
Right.
Things like that.
So, and I think Google, you can never be sure what Google cues off of because they would be massively game as soon as they let that out of the bag.
But I think site performance also affects ranking.
Yes, I believe it does.
And they've got, Google's got their own tools even to check site speed.
And if you use a static site generator to produce static sites, you're generally going to score pretty well about that.
Exactly.
Yeah.
You can do little things to make your static content even faster and optimize above the fold content and things like this.
Like minification or bundling and things like that?
You can minify.
You can compress.
You can make sure that all the CSS that's relevant to content that you see above the fold, that is when you first load a page and not below it when you scroll, you can make sure that CSS is loaded first.
So that there's an even smaller flash of unstyled content.
Yeah, that's a cool idea.
Yeah.
So you can do things like that.
But if you simply just have static content on a CDN, no matter what, you're going to try to make it really slow.
That's absolutely right.
That's pretty awesome.
And so all those benefits accrue to those.
And I think the static site, there's also sort of data driven, but then maybe mixing either like backend services over JavaScript or like a section of various sections that are fully dynamic.
I think that's a pretty interesting way to think about going beyond just, well, I have a couple of pages and a static site.
Definitely.
You can interplay them.
You can mix and match.
And that will increase the speed of all the static components, which will make your users happier.
Yeah, absolutely.
This portion of Talk Python to me is brought to you by Linode.
Are you looking for bulletproof hosting that's fast, simple, and incredibly affordable?
Look past that bookstore and check out Linode at talkpython.fm/Linode.
That's L-I-N-O-D-E.
Plans start at just $5 a month for a dedicated server with a gig of RAM.
They have 10 data centers across the globe.
So no matter where you are, there's a data center near you.
Whether you want to run your Python web app, host a private Git server, or file server,
you'll get native SSDs on all the machines, a newly upgraded 200 gigabit network, 24-7 friendly support,
even on holidays, and a seven-day money-back guarantee.
Do you need a little help with your infrastructure?
They even offer professional services to help you get started with architecture, migrations, and more.
Get a dedicated server for free for the next four months.
Just visit talkpython.fm/Linode.
So let's talk about some of the tooling that we might use for this.
So Lector, obviously, is pretty awesome.
Yes.
But it's not the only one in the Python space, right?
There's a couple of others.
Maybe you could just touch on those, just so people know kind of what,
if they see built with something at the bottom, they're like,
oh, that was a static site generator.
Python or not, there's a handful of static site generators out there.
A couple popular ones, I think, are Pelican and Hugo.
Hugo is definitely very popular.
I think that's written in Go.
And they predated Lector, I believe.
And what they do is they use templating and Markdown.
So they try and make it pretty easy for you to make your content in Markdown.
And then they have a little backend functionality with template context.
So you can query things as you're building your static content
so that you don't have to write the same HTML over and over and over again,
which is really handy.
Right, like an overall layout page with your navigation and footer and CSS and JavaScript,
et cetera, right?
Yeah, not having HTML templates is a thing of the past that people should not be doing anymore.
But I deployed them from front page.
Come on.
Right.
People do still do them.
That's absolutely true.
But I don't think they really should anymore.
So using these templating languages and the static site generators make people's lives easier
so they can not have to expend very much effort to expand their website.
And a lot of these static site generators have themes.
So you can build off of the theme too, which is really handy for a lot of people.
So they don't have to come up with their own CSS, at least from the back.
But Lecter came after these and it learned from some of their deficiencies too.
Because again, going back to the history, I think the static site generators came out after the more complicated web frameworks did.
And these servers.
So they were a step back from the initial reaction against pure HTML.
So pure HTML was horrible.
Then it went to web frameworks.
And then we got static site generators that kind of existed in between.
And Lecter has all the features of those static site generators.
But it's got some of the additional features that the web frameworks have that make life even simpler.
Okay, like what?
Like a CMS.
So far as I know, Lecter is the only static site generator that comes with a built-in CMS out of the box, ready to go.
Yeah, that is pretty interesting.
So I guess maybe let's do like a high-level flyer of just how it works.
So with Lecter, Lecter is a thing you install onto your machine.
And then it's a command line application you run to generate the static file site structure.
And then you can use it also to serve up that content in a way that will let you basically click a button, like a little edit button on any page, type away on it.
And it will under the covers actually write or save the markdown and then live reload that as you develop your site.
And then I guess there's probably like some kind of Lecter deploy step that like finally builds it out to the final HTML.
Yes, there is.
So it is a program that you download and install and you run it over the CLI.
But you don't have to interact with the CLI very much to use it.
I had it generating sites on all these two commands.
Right.
Lecter quick start and then Lecter server.
And then I like I was I was going to get it.
I was playing with it.
It's fun.
Good.
And that's how it should be.
So the previous static site generators would mean that you had to either open markdown and edit it in whatever editor you like or just do everything in the terminal.
And not everybody is comfortable with that.
We're trying to make static sites accessible to people who aren't necessarily programmers.
Well, and static sites are especially beneficial to people who are not capable of like all the CLI stuff and running databases.
And that like it's exactly the people who don't want to do that.
The flask pyramid SQLAlchemy type stuff for whom this really benefits because the deployment and like running at the site is crazy easy.
And so, yeah, having that sort of focused in that area is really nice.
Right.
So would you say like if you're working in a company and somebody comes to you and says, hey, can we build this new site?
I'm thinking about using SharePoint or something like that because we have to be able to keep track of X, Y or Z or whatever.
Do you think Lecter might be like, hey, let's not use that.
Let's set up a Lecter thing and you can actually use the CMS and manage it and stuff like that.
Would that be good?
It depends on a few things.
Like if someone needs complicated user management, they want to segregate employees and say certain employees have certain permissions and others don't.
Then Lecter is not a fit for that.
Doesn't make sense.
Yeah, because static files don't have great permission role things other than at the OS level.
Right.
And that's actually part of the beauty of it really is because there's no user permissions.
Everybody can do everything and it makes things simpler.
It takes away a big chunk of complexity out of the system.
Yeah.
That's the tradeoff, right?
It's that you get this super fast, super simple system, but it doesn't do everything, right?
You could make it do everything, but then it wouldn't be simple.
It would be hard again.
It would be SharePoint.
What I have seen people do is they will run Lecter on a work LAN and they'll have all the employees use the same Lecter setup so they can all edit pages as they will.
And if you trust your employees to not make horrible mistakes, then that works really well.
You don't necessarily need all of those permissions.
Yeah.
I feel like a lot of companies go too far and they're like, well, we've got to lock everything down and restrict people and whatnot.
And it just blows my mind.
If you don't trust these people, why are you employing them?
Like if literally you're like, I will not trust this person to touch this file, like that person should probably just not work here as of this afternoon because like that is a strong statement for so many things.
But I see it over and over in these companies, right?
I mean, I guess there's a difference between protecting yourself from like I clicked on a phishing link, but there's a lot of these cases where it's like clearly we don't trust our employees and like, well, why are you employing them?
Right.
And I get to concerns about people doing things by accident.
So not like maliciousness, but accidentally deleting your content.
But you can protect against that in other ways, too.
You don't necessarily need complicated user permissions.
Dropbox even has backups you can take.
That's right.
And you could even have someone who is maintaining the lecture deployment locally on a LAN and five content producers adding in content.
But the one person actually making git commits every now and then by hand because they know what they're doing.
Right.
You could even automate that.
Right.
Just every hour, just commit the site.
Whatever it is, it's like this time of day we commit it.
And that way, at least you have a way to go roll back.
There's a version history there.
Exactly.
Yeah.
Very cool.
So maybe let's do like a rundown through some of the core features because I think they're pretty awesome.
We've been sort of touching on some of them.
The first one is around deployment.
Right.
It says basically you can deploy anywhere.
So I could go and I could set up something kind of probably over the top like an EC2 Linux machine or Windows machine and copy my files up there and serve it there.
Maybe I could set up like a $5 Linode server, which would be pretty sweet.
But being static files, there's even like more options.
Right.
Like I could go way basic and put it on S3 or where else could I put it?
Yes.
Well, just like you're saying, because it's a static site, it can go anywhere.
It can go on any CDN at all.
Basically, you can put it on S3.
What we do for our own personal website is we host it on S3 and then have CloudFront take care of it from there.
So it's got an S3 backup, but it's really just using a CDN.
And putting your site entirely on a CDN is groundbreaking.
You know, when I figured out that I could do that, I was all about it.
It's blazing fast now and I don't have to worry.
It's super easy.
There's no denial of service.
But I guess another thing that's pretty interesting about that is like if I open up my micro WSGI log, which is the worker processes for my websites, and I watch them.
A lot of times people are coming and requesting things that I want them to request, but, you know, periodically they'll be going to wp-login.php or some other malicious thing.
My site is not built in WordPress.
It's Python, so I don't really care.
Like I just like block those people.
But it's very clear that there's like a continuous sort of low burn attack against your servers, against your web server infrastructure, and your site.
And it's super hard to like SQL inject static on a static site, right?
Exactly.
Like any of these other things.
If you have no logic backend and you have no database that's going to get compromised, then you don't have to worry.
That's right.
I mean it's down to does Cloudflare itself get – or Cloudfront itself get hacked, right?
Is that a thing that can be done?
And if it can't, then you're good.
And even if it can, it's kind of not your problem.
Yeah.
You could be defaced, but it's not like you're going to be in the news because now a million recordings from a teddy bear between kids and their teddy bear are now on the internet.
And honestly, you probably won't even be defaced.
If Cloudfront is having any sort of problem, it's probably going to affect more than just you.
Yeah, exactly.
It also is better for security because that means if any problem happens, it'll get a lot of attention and get fixed faster because the rest of the world is looking at it, not just your users.
So it's cross-platform, right?
It works like the tooling to generate the static content.
I mean obviously HTML is cross-platform, but it does take some tooling to power the CMS to generate the data files that are then built into HTML, right?
Right.
Lecter is built in Python and it runs on Linux and Macs and Windows.
It should run everywhere.
And the CMS that Lecter runs internally is just front-end content.
It's built in React.
You open up a browser and you see your CMS.
Yeah, it's pretty cool.
There's no database, but it acts like a database, right?
There's a set of files and a file folder structure, basically a markdown and then these INI files, right?
Tell us maybe a bit like at the interplay of page templates, the markdown and these like sort of definition files.
Frameworks had like Django.
And in Django, you have model files and you've got admin files and you have databases that you connect to.
And Lecter is kind of thinking that, but it's making it a lot simpler and accessible.
So instead of a models file .py and writing Python code, we have simple INI files where you can define fields and content types and page types.
And all of that data is stored in flat files on your system.
And then the templates have access to all of that content in their template context.
So you can use the templates with the Jinja tags to query for body content, for titles, for URLs or checkboxes or anything like that.
That's really, really clever.
So let me repeat this back to see if I got that right.
So we have these INI files and they define like basically what data is in a given type of page.
So you have different types of page, like blog posts, landing pages, whatever.
So like you could say a blog post contains a title, content, an author and a published date, for example.
And then you have a markdown file that has those various pieces of information entered into them.
And finally, you have a Jinja 2 template that takes that bit of information and then renders it like however you would in like a regular HTML.
It's like a dynamic HTML expression of that.
That's all pretty close, but I think you got a little bit wrong.
So the information isn't stored all in one markdown file.
It's stored in a lectroproprietary format that is just a flat file of data.
Oh, right.
But you don't see that most of the time as a user.
What you see is you interact with the CMS.
So you define the INI file, which is a model file, and then you go to the CMS and you make a new page of a certain type.
And the CMS will show you all the content fields that you need to populate.
So the published date and the author and the body content are all separate fields in the CMS.
And do they have types?
Yes.
Those fields can be of type markdown or string or date or integer or checkbox or anything like that.
So the template, you can query for the specific fields and do template logic on those fields if you want or just simply display them.
It'll render the markdown for you if the type is a markdown file or markdown content.
That's pretty cool.
Another thing that's pretty nice for large sites is dependency tracking.
Right?
So initially, like who cares?
It's just a few files.
But, you know, you could easily see over time something with a thousand pages.
Right?
Like if, in say, a bookstore, like a data-driven bookstore, you might have a landing page, a category listing page, and individual book details page.
It might be just three pages, but the data drives like thousands of distinct URLs.
In a static site generator, you literally build a thousand pages.
Right?
And then, like, generate because your database is this stuff.
So with dependency tracking, like, that could be a big deal.
Right?
So what's the story of that?
That can be a big deal.
So you're swapping out a database like Postgres for flat files, and that has certain tradeoffs.
Right?
Postgres is optimized on the file system, and it's fast.
Flat files are not optimized for size or speed, but they're much easier to read by human.
And that means you can actually version control all this.
It's not binary anymore.
You're actually storing the text from the various fields of these books or the author information or publication information.
So if I – like, maybe you change, like, the details about one book, and you want to redeploy it.
You don't want to regenerate a thousand files.
You want to just regenerate that one HTML file, right?
Right.
And that's what would be generated.
So if you're changing the information about one book, then there would be a single file that is touched that's devoted to that one book in your flat file system in Lecter.
And that would have a line change in it when you change the author information.
You're correcting a typo.
And then Lecter would notice that change and auto-rebuild the relevant HTML file for it.
And it wouldn't build all of it.
When you run Lecter server the first time, it builds all of your files.
But then it just is looking for changes.
And it builds files for anything that is touched.
That's nice.
It'll even do that, like, as you interact with them, either on the file system or through the CMS, right?
That's pretty slick.
And unless you're changing content that's actually relevant to a thousand pages, you're not going to build a thousand pages all over again.
Yeah, that's real cool.
This portion of Talk Python to Me has been brought to you by Rollbar.
One of the frustrating things about being a developer is dealing with errors.
Relying on users to report errors, digging through log files, trying to debug issues, or getting millions of alerts just flooding your inbox and ruining your day.
With Rollbar's full-stack error monitoring, you get the context, insight, and control you need to find and fix bugs faster.
Adding Rollbar to your Python app is as easy as pip install Rollbar.
You can start tracking production errors and deployments in eight minutes or less.
Are you considering self-hosting tools for security or compliance reasons?
Then you should really check out Rollbar's Compliance SaaS option.
Get advanced security features and meet compliance without the hassle of self-hosting, including HIPAA, ISO 27001, Privacy Shield, and more.
They'd love to give you a demo.
Give Rollbar a try today.
Go to talkpython.fm/Rollbar and check them out.
So it has built-in image sort of thumbnailing tools and stuff.
That's pretty nice.
It has a plug-in system.
So that sounds like you could really take sort of Lecter out of the box and then make it really adaptable to kind of the way you might with, say, a traditional data-driven app.
You can also mess with the Jinja templates, which is sort of a programmable layer as well, right?
Yes, you can do a lot with the plug-ins system, and a lot has been done.
We've got more than a handful of plug-ins on our website.
There are plug-ins that change functionality inside your Jinja templates.
There's plug-ins that change how your markdown is rendered.
So if you want special formatting and you want to enable certain niceties in the markdown.
Like GitHub flavor table sort of thing?
Yeah, you can change the renderers or the lecters in the markdown.
You can add classes.
You can have special markdown syntax that's your own and have the rendered markdown use custom HTML classes.
There's admonitions.
There are plug-ins that allow you to deploy to various different places.
Like the S3 deploy isn't built in in Lecter.
That's provided by a plug-in as opposed to GitHub page deploys, which is native.
Oh, that's pretty sweet.
So you can actually go in there and sort of tell the plug-in your API key and then just run a command, and then you have a new version online?
That's right, and you can hook into – so you can write a plug-in to deploy to anywhere you like.
That's awesome.
It also has a Python API.
So is that just like a different angle to get in there instead of writing a plug-in, maybe from the outside instead of the inside?
Or like what's an example of what I might do with the API or what I could do?
The first common use of the API is within the plug-ins itself.
It's telling you how to access every part of the Lecter system through a plug-in.
So the plug-ins have various events, various event hooks inside the Lecter code that they respond to.
So when something happens, it trips an event, and any plug-in listening on that event can have the opportunity to do something.
But that plug-in has access to all of the Lecter source files in Python, so it can import everything and change anything.
I see.
That's pretty cool.
So maybe some part – let's say, for an example, like part of your site depends upon data you might get from an API or something like that, like a web service, right, that is not part of your site.
Maybe at build time, you just like go and hit that, refresh it, and then dump it in there and render it.
So it's kind of live.
It's sort of cached from the moment of generation anyway.
Yes, you could do that, definitely.
You could do that.
You could change the types that are in Lecter.
You could add a content type that's entirely new, a field type.
You could generate entirely different kinds of pages.
There are plug-ins that add template functionality to allow Atom feeds and Discus really easily.
Right, right.
Discus is pretty nice.
And, yeah, I guess you could even, like, regenerate, like, an RSS file sort of thing, right, say, for blogs or a store.
You want to have the new items or whatever.
Yes, there is an RSS Atom feed plug-in.
Yeah, very cool.
So one of the areas that I was – when I was playing with Lecter, I'm like, this thing is really nice.
One thing I would like to see when I was playing with it is maybe ways to configure how it looks, right?
Because it comes out and it looks like it looks.
But I'm like, oh, can I, like – you know, one of the things I love about Bootstrap is, like, themes, right?
I can go to, like, say, Wrap Bootstrap or something like that and go, here's a whole bunch of stuff.
I'm not a web designer, but that looks really close to what I want.
Let's see if I can hack that into the shape I wanted, you know?
What's the story around themes for what you guys are doing?
Themes are really new in Lecter, actually.
They did not exist prior to a few months ago.
But as of Lecter 3.1, which came out in January or February, there is now theming functionality in Lecter.
There's not a lot of themes available yet, but they're growing in numbers slowly.
So people should go ahead and try this out and make some and publish them for us.
We'll add them to our list.
Yeah, that sounds really good.
So themes allow you to easily modify the look and feel of your website, of course.
But in Lecter, they can do more than that.
They can add content type.
You can have models in your themes, even.
I see.
So if I wanted to have a theme for, like, a photography store, I could have, like, a photo model type that has, like, the picture and the little copyright information and the date and things like that.
Yes, absolutely.
And as you would expect with a theme, anything can be overwritten or extended, too.
So when you're using a theme, like a photography theme, you could easily add your own models for videos if you also do videography.
I see.
If you decide you don't like after using the theme for a while, you don't like the model and you want to extend it, you can do that, too.
Okay.
Yeah, that sounds really nice.
So when I went through and sort of created this little playground site that I was playing with, I went through and I installed Lecter.
I did that off of GitHub.
And then I typed Lecter Quick Start, and I got the site, and I played with it.
Like, how would I incorporate themes into that?
Like, how do I know what themes are there?
Is there a way to type, like, quick start photography one or whatever?
There is not a command line tool to just grab the theme so easily yet, but we've got some issues on the GitHub for making that.
Right now, what you would have to do is, where you have your Lecter code, you make a themes directory, and in that themes directory, you put any themes that you like.
And the themes that exist right now are in their own repository on GitHub, Lecter Themes.
So you can check those out and download or clone any of those, and they will be picked up by your Quick Start.
You'll have to, the Quick Start provides its own default styling and templates and models.
So when you add the theme, you'll have to remove some of the ones that Quick Start added because they will be overwriting the theme.
Right.
Because that's a feature of the themes is that they're overwritable.
Right, right.
Well, that sounds pretty cool.
I think maybe a really cool plugin would be, like, list all the themes, activate the themes.
Who knows?
Would it be possible to write one that did that?
It seems like it would.
It would.
Though that sounds like such a good feature.
I'd like it in Electric Core.
Yeah.
Yeah, that's true.
It should just be built in, right?
Maybe some experimental stuff could be done as a plugin that could be brought in, right?
Kind of like maybe Python standard library style.
Yeah, it could be.
But definitely, theming is new, but it's very exciting.
So I'm trying to push the themes because I think that'll make people's lives a lot easier.
Yeah, I think it's really, really great.
And it definitely could open up sort of like a way for people to reuse the design of sites they like.
There is a showcase, right, at getlector.com slash showcase of people.
Yes.
Or sites that people have created, which is pretty cool.
A lot of them seem to be created by Armin Roeniger of Flask fame.
Like, he was involved in the early days of this project.
Is that right?
Yes, absolutely.
Armin is the creator of Lecter.
Yeah, that's kind of what I thought, but I wasn't 100% sure.
Armin is the creator of all the Palettes projects.
And he's the creator of Jinja and WorkZoog and Flask.
So he's done a lot.
And Lecter uses many of those tools and wraps them up nicely.
Yeah, when I typed Lecter server, I got localhost 5000.
I'm like, I see, Flask.
Okay.
Yep.
Absolutely.
Flask is what's driving the CMS.
So it's a Flask server using a bunch of React.
Okay.
Nice.
And the showcase, since he made Lecter and he made GetLecter's original website, the first few pieces of the showcase are his websites.
But to scroll down, you can see others.
One that I really love is a PyCon website that is on the showcase.
It's a beautiful example of a single page website.
Yeah, PyCon Philippines.
Yeah.
Yep.
Nice.
Yeah, that's a good looking site.
Very, very cool.
So speaking of single page, like maybe the last thing we'll have time to touch on is you have a bunch of guides.
So there's a quick starter, but it's like, all right, well, I want to create something that has categories or discus things or how do I deal with error pages, right?
So you have a bunch of guides that go through those types of things.
Do you want to tell people a bit about those?
Categories, for instance, is a really handy guide because it's a very commonly wanted feature.
You want to be able to have projects or pages that are cross-linked somehow by certain categories.
Right.
It's one thing if it had like one category, you just put it like in that subfolder and it's fine or something.
Right.
But if it could be like in science and engineering, like, oh boy, now you're in trouble, right?
Right.
Exactly.
Because you could manually create all of this and make them all separate pages, but that's not using a lot of the power that's available to you.
Right.
And data duplication as well.
And though Lecter can be used in a very simple fashion, some of the power is a little tricky.
So we've got some guides to help sort that out for you.
So you don't have to create all of the Gingelogic and the macros in your templates to figure out how to appropriately use the categories and how to set up the models so that you have the categories functionality.
Yeah, that's really cool.
Yeah.
So you've got the distance comments, the error pages, which, you know, is always a nice thing to have.
Pagination.
Yeah.
Pagination is critical, especially if you have a blog.
There's a guide for single page apps, which is really nice because if you have in mind a static site generator building an HTML file for every page that you have and every page is a different topic, then it might not be obvious to you how to concatenate or combine all of that content into a single page, which is what you really want sometimes.
Yeah, yeah.
Like, just show me the latest flat down the list in order, things like that.
Sitemap is also sitemap.xml.
We talked about the SEO benefits of static sites.
Like, here's a big one.
Yes.
So there's a quick guide to get that up and running, which is really handy.
There's also a Webpack guide.
So it can get you started integrating some of the fanciness of the rich JavaScript tooling.
Webpack does all sorts of interesting things like SCSS or less or bundling or a whole bunch of those types of things if you want them, right?
Yes.
You could even make a fairly complicated React app on top of Lector if you wanted.
Yeah, that's cool.
I've looked at using Webpack for my sites, and I decided I would rather have something simpler.
And I just use, like, super aggressive caching.
Like, cache this for a year.
Everything that's static, just cache it for a year.
So the very first time, you might hit four JavaScript files and three CSS files, but you'll never get them again.
So, like, I'm not going to go to the layers of trying to regenerate that stuff.
I know a lot of people make use of Webpack, and it works for them.
Yep, and there's a lot of Webpack functionality around modifying your build system, too, so you can minimize all of your content.
We were talking before about minimizing even a static site.
So a lot of that can be done with Webpack or Browserify or Grunt or anything like that.
So where does continuous integration fit into this whole story of static sites in Lector?
Like, does it make even sense to set up continuous integration and have, like, something like Travis CI or various, you know, servers, like, looking at your possibly your Git history and checking those out and verifying everything, like, compiles correctly with Lector?
You can wrap up tests in Travis, and you can automate builds with Travis.
Our own website is actually done like that.
So every time we push a commit to the Lector website repository on GitHub, Travis goes, and it builds all of that, and if it's good, then it deploys it.
So that's really handy.
Does it just use, like, the S3 plugin and just, like, run that command on successful build?
Basically.
It's not using S3, but it is deploying on the successful build.
Nice.
Okay.
Well, that's really, really cool.
Yeah, and you can automate all of that with other things, too.
It doesn't have to be Travis.
You can make it on a cron job if you wanted.
Right.
Anything like that.
Yes, but it doesn't, I guess it doesn't depend as much.
It's not like there's not as much logic, but I guess you could still screw up one of your Gingita templates or something like that, right?
So you might want to test.
Yes, and that's not too hard to do.
Lector is appropriately verbose, I think, in how it outputs information.
It'll tell you when it's building all the web pages, and it'll tell you their success or failure.
So you can look out for those basic failures.
So you can test for the page building successfully.
That won't, of course, test if the content is correct.
So if you made some typos, then you'll still have to catch that by hand.
Yes.
Maybe something with Selenium against, like, I don't know, something local.
Who knows?
But a build server is not going to do it, for sure.
Nice.
To me, it sounds like Lector fills a pretty interesting middle ground, I guess, between full stack stuff and static sites.
Having models is a game changer for a static site, and the CMS baked in is a really easy way to use it.
So you have a lot of the features that, for instance, Django would have, except for the actual backend processing, which you often don't need.
But that doesn't mean you don't want models, you don't want to define different content types and different data structures.
You just don't need the additional complexity of actually using Postgres or having backend Python.
You don't need the servers and the databases and all of that comes with taking those on board.
Those are more like puppies, right?
Once you get them, you have to raise them and care for them continuously, basically.
Oh, yes.
And there's loads of headache that you avoid in not having to do that, too.
You don't have to manage your own CI CD.
It becomes really simple, and you don't have to worry about security as much, which we talked about.
Yeah.
And it's blazing fast on top of it.
Scalability, distribution denial of service, these are not real issues, yeah.
Right.
You let the CDN handle that.
Yeah, exactly.
Very cool.
All right.
So let's leave it there for Lector, but I'll ask you the two final questions before I let you out of here.
So if you're going to work on some Python code, what editor do you use?
Four years, I've been an Emacs user.
Oh, right on.
I've got my own .emacs store in a Git repo that I can't live without.
Nice.
So it's right there.
Anytime you got to check it out, you're all good, right?
Yep.
.emacs.d.
Yeah, perfect.
And then notable PyPI package.
Obviously, there's Lector, right, which is great, but what else?
A personal favorite of mine for DevOps is SaltStack.
I've been doing a lot of DevOps at Terminal Labs the last couple of years, and SaltStack is enormously feature-rich.
It can do almost everything I can think of.
And we've got our own open source project named Rambo, which is built on top of SaltStack partially to make DevOps even easier.
We're building it as a provider-agnostic provisioning framework.
Oh, nice.
So Rambo makes great use of Vagrant and SaltStack to make a lot of things a lot easier.
So it helps get up and running with a lot of things pretty fast.
The other thing, which isn't exactly a pit package that I really want to plug, is Kondo.
Are you familiar with Kondo?
Yeah, Kondo's awesome, especially if you're trying to do something on Windows involving data science and vcvaralls.bat or something like that, right?
Just basically, pip installing a lot of the data science tooling requires funky compilers that are not easily accessible on Windows.
And this pre-builds it, and you kind of get everything pre-packaged and ready to roll, right?
Absolutely.
Even outside of data science and machine learning, but just normal Python development, Kondo has saved my butt more than once.
You never know.
It always surprises me when I try and pip install something that has some C dependency that I didn't have on my system that it doesn't work.
So I would either have to Google around and fight it for an hour or two trying to figure out how to install it, or use Kondo and it just works.
Kondo install.
Forget this thing.
We're out of here.
That's cool.
Yeah, really nice.
Really nice.
All right.
Final call to action.
People are all excited about static sites, about Lecter.
What can they do?
Go check it out, firstly.
Try it out.
Hopefully you'll love it.
If you do, please, if you're a Python developer, try contributing.
We have an active Gitter, and I'm pretty active in the community.
I'll try to respond to anything.
Yeah, I think that's how I actually got in touch with you is I found you on the Gitter channel.
Yeah, so that was cool.
Right.
Well, I would love to see more plugins being made.
You can do anything with a plugin, and we also would love to see more themes.
So if you are a theme developer and even not a Python developer, you want to spread your theme around to the Lecter world instead of just Hugo or WordPress, then come on over.
Yeah, that sounds awesome.
All right.
Well, I think Lecter is pretty cool.
People should check it out.
And Joseph, thanks for being on the show.
It was great to talk about all this stuff with you.
Thank you.
This has been another episode of Talk Python to Me.
Our guest on this episode has been Joseph Nix, and the episode was brought to you by Linode and Rollbar.
Linode is bulletproof hosting for whatever you're building with Python.
Get four months free at talkpython.fm/Linode.
That's L-I-N-O-D-E.
Rollbar takes the pain out of errors.
They give you the context and insight you need to quickly locate and fix errors that might have gone unnoticed
until your users complain, of course.
As Talk Python to Me listeners, track a ridiculous number of errors for free at rollbar.com slash Talk Python to Me.
Want to level up your Python?
If you're just getting started, try my Python jumpstart by building 10 apps or our brand new 100 days of code in Python.
And if you're interested in more than one course, be sure to check out the Everything Bundle.
It's like a subscription that never expires.
Be sure to subscribe to the show.
Open your favorite podcatcher and search for Python.
We should be right at the top.
You can also find the iTunes feed at /itunes, Google Play feed at /play,
and direct RSS feed at /rss on talkpython.fm.
This is your host, Michael Kennedy.
Thanks so much for listening.
I really appreciate it.
Now get out there and write some Python code.
Thank you.