#160: Lektor: Beautiful websites out of flat files Transcript

00:00 Michael Kennedy: What is the fastest most scalable web platform out there? Is it Pyramid running on top of MongoDB with a Redis cache? Maybe Flask and Postgres as a service with some high performance cluster? Or some funky Go API framework? No, it's static files but today static files does not mean that you write a bunch of HTML. With static site frameworks and generators like Lektor and Pelican, you can use data to drive the creation of static sites and then host them wherever makes the most sense for you. On this episode you'll meet Joseph Nix who works on Lektor, a Python-based static site generator. This is Talk Python to Me, Episode 160, recorded April 19th, 2018. Welcome to Talk Python to Me. A weekly podcast on Python, the language, the library, the ecosystem, and the personalities. This is your host, Michael Kennedy, follow me on Twitter where I'm @MKennedy, keep up with the show and listen to past episodes at talkpython.fm and follow the show on Twitter via @talkpython. This episode is sponsored by Linode and Rollbar. Please check out what they're offering during their segments, it really helps support the show. Hey, Joseph, welcome to Talk Python.

01:17 Joseph Nix: Hi, thanks for having me.

01:19 Michael Kennedy: It's great to have you here. I'm really excited to talk about Lektor and more broadly the static site generators because they're such an interesting way to build web applications that I think are really quite the opposite of full stack, data-driven, scale out this and that, right?

01:38 Joseph Nix: Yep, definitely. But they definitely have a huge place. So, they're a good need, I think.

01:43 Michael Kennedy: Yeah, and I think Lektor lives in a pretty interesting space in that realm as well. Before we get into it, though, let's start with your story. How did you get into programming and Python?

01:54 Joseph Nix: My story is kind of fun, I think. So, I didn't start out wanting to be a programmer at all. I grew up interested in physics and math, that was all that I was about. I wanted to be an astrophysist since I was a little, little kid. I went to school, UAH in Rocket City, Alabama, that's where they built the Saturn V, and I wasn't taking any programming classes except for one that was mandatory, it was a C++ class that I struggled through. But I met some cool friends and, for fun, we would spitball ideas, we'd talk about cosmology and things like this and we ended up, again for fun, inventing a kind of rocket engine. So, I didn't see that coming. It's a small thing, at least physically. It's a kind of electric propulsion device which I would, I could talk about forever, basically, but it's pretty cool. We researched it on paper and we decided to form a company to protect the intellectual property with aspirations of actually doing physical R&D and doing something with it. So, we did that and then we realized, okay, now we've got this company and we want to do rocket research and we have no money at all because we're poor college students.

03:24 Michael Kennedy: Right, there's no hiring engineers or things like that, right?

03:27 Joseph Nix: No. We considered various ways to remedy that but what we decided was, we looked at everybody else that was doing something similar. There's all sorts of space entrepreneurs going around at the time and still now and almost all of these that we could find were founded or funded by people who made their money in software.

03:51 Michael Kennedy: Right. I mean, we've got SpaceX, we've got the Blue Origin.

03:56 Joseph Nix: Yeah, Blue Origin.

03:57 Michael Kennedy: Yeah, yeah, yeah, exactly, and some probably a bunch of others I don't know about.

04:00 Joseph Nix: Virgin Galactic, Richard Branson, I mean it's not entirely software but a lot of it but absolutely, people, all these software executives and founders, it seems like they maybe even have it in their mind a long time ago that this is what they wanted to do, they really just wanted to go to space. They thought of some novel software applications that are useful in and of themselves and interesting, in and of themselves, but will help them get to space as well.

04:29 Michael Kennedy: That's really interesting. So, how'd that get you into programming?

04:31 Joseph Nix: We decided we needed to teach ourselves software so we can make some money to do this rocket research so that's exactly what we did. Kind of from the very beginning, for me, I was talking with my friends and we decided what we wanted to do. We looked in the various programming languages out there from a completely fresh perspective because none of us are programmers really. Looked at PHP, looked at Fortran, looked at C, Python, and we kind of felt that Python was maybe the way to go. It seemed like it was becoming popular, we read about why, we experimented with a few languages, and we liked Python so we dove in and it stuck. It was very easy to pick up.

05:14 Michael Kennedy: That's definitely one of it's super powers. Also, so many of the languages that are easy to pick up, they're that way because they give up sort of the professional high end proper architecture and Python seems to live in the space where like you can start without all the structure and objects and separation and packages but it can grow into that as you need it, right? So, I think that's part of the secret sauce.

05:38 Joseph Nix: Yeah, definitely. When we first started out learning Python, I was still having a hard time wrapping my head around OO. Just doing very basic, imperative and functional things to get our feet wet but it grew and I've been doing Python now for a decade and I feel pretty comfortable.

05:56 Michael Kennedy: We'll get into Lektor and all this stuff, I just want to ask you, that story, did you watch Falcon Heavy launch?

06:02 Joseph Nix: Yes, I did, oh my goodness, it made me cry.

06:06 Michael Kennedy: I thought that would be kind of a major thing but then like you watch it and it's like oh my gosh, people are still doing amazing stuff in space even though the government seems to have given up on it.

06:15 Joseph Nix: Yes. That was really frustrating too at the time so we started learning Python, I don't know, back in like the mid 2000's which was right before the economic crash in 2008 and people, which is when we were starting to get good at Python, we were trying to be consultants but it's hard to find a job, people weren't really hiring then and all these space flight companies were struggling too because, again, everything was kind of suppressed a little bit. They were trying to get off the ground running with something aggressive in the first place but they've pushed through and they've done great things and I'm super excited to see where all of them go.

06:55 Michael Kennedy: Yeah, watching those two outer rockets land, well, I guess the third one tried to land, was just, it was like, oh my gosh, we are in science fiction land but it is like happening. Right? It's just, when I saw that, I'm like, that is something you would see in a movie, that is not something you would see in realty.

07:13 Joseph Nix: No, it was just unbelievable. I think they had a video where they tried to show what this will look like before it actually happened and they had the two rockets land, separated in time, and I don't know if they were planning on them actually landing just identically together. My guess is they wanted that but they weren't going to bank on it, so they didn't want to get everybody's hopes up. Boy, it just looked ridiculous.

07:38 Michael Kennedy: Yeah, yeah, yeah. I'm so looking forward to the next five years, what's happening around there. All right, we could totally turn this into a space show but let's stay focused on the stuff that came after your project, just to sort of close it out, does that company still exist? Has anything happened of it?

07:55 Joseph Nix: Yes, definitely. We have grown in skill and we're now a stable company. Our company is named Terminal Labs. We're a small Python consulting shop, we work contracting for other companies to do whatever it is they need. In the past we've done a lot of full stack web development, we've worked with Django and Flask, some really big Django projects and we've also loved it. Any chance we can get, to do any science, so, data science and machine learning are really exciting and we're getting more and more into that all the time. We've got a pretty strong DevOps background, so all of this is keeping us employed and raising our skill level further so that we can eventually make a rocket.

08:40 Michael Kennedy: Awesome. All right, so that sort of brings us to what you're doing today and that's really where you're still focused is sort of continuing to grow the consultancy and stay and basically play in the science space when possible.

08:54 Joseph Nix: Yes, absolutely and making connections and yeah. Our end goal is to get to space and how exactly we're going to transition there, time will tell. We've got some ideas. But this has been a pretty good course so far and I think it'll work.

09:11 Michael Kennedy: Very cool. Glad to see that that's still going. So, let's talk about the project that you've been involved with, sort of, most broadly at first, with static site generators. So, many people who listen to podcasts are web developers but a lot of people are scientists or data scientists or even students. So, maybe just like, could you compare and contrast say a static site generator experience to like let's say like a traditional web app? You talked about Flask and Django, either of those?

09:42 Joseph Nix: Yeah, absolutely. So, with a traditional web app, you have a server, there's a server on the cloud that a user will hit and make a query to and then that sort of does some logic and often connects with a database to dynamically generate something to return to your browser. A static site generator doesn't need any of that and often I'd recommend against it. Like, you could put it behind Flask but there's really no need. The build products of a static site generator are basic HTML.

10:15 Michael Kennedy: The trade off, it seems at first like so the trade off is I could just create a bunch of JS, CSS and HTML files and just write them but that becomes super tedious super quickly, right? Like, even if you just want the same look and feel across multiple pages, right? Like, if you've got 20 pages like that and somebody comes in, either boss, client, whatever, says I would love for that background over here to be blue and the font to like reorder the menu. That's not hard, right? But if you actually hand coded it, like, it sucks, right? You got to go and change all those. But on the other hand, if you got to maintain databases, scalability, security patches, etc. that also can be painful. Sometimes it's totally needed. Like, from my site, it has to be dynamic, it absolutely needs a server and a database but many sites don't, right?

11:08 Joseph Nix: Absolutely. The history of it is we started with, it was raw HTML and CSS and that was absolutely a pain, you're completely right. I hate coding CSS. But the reaction to that was to build these very complicated web servers and not everybody needs that either so the static site generators came around as a bit of a compromise for people who don't want to build these basic HTML sites with CSS by hand because that is an enormous pain but I don't need the power of an actual logic backend and a database. All they really want is to make their lives easier creating the static content and things like Jinja really help a lot with that and web templating.

11:54 Michael Kennedy: Right. And so, for example, like, you probably still want some kind of data driven site created like you could have a blog and you want to add a new entry and have that appear and be linked to, possibly, under various categories and stuff but you want to be able to like I guess the thing is you don't need to show a different version of the blog to anyone. It's totally fine if they all get the same one so instead of doing that at run time on the server over and over but always giving the same result. Like just crank out, like, somehow on your machine at like a moved production time you generate that and then you just put the static files there, right?

12:31 Joseph Nix: It ends up being really nice in production too because all these files are pre-built, so all that you need to do is put this HTML on a CDN or something and then they get served incredibly quickly so it's very performant. So, from a production standpoint, it's faster to do this for the user than to actually query, for example, on a Flask backend, that is it's just doing the same thing. Just generating a simple HTML for everybody and giving, giving it out.

13:00 Michael Kennedy: I guess you can even mix them, right? So, suppose you've got some section of your app there that's like a store, obviously, you would like people too, that could be dynamic, but the rest of it maybe not. You could do something like Nginx and like URL routing to say this part routes to like a Flask app or Pyramid app, this part routes to the static content.

13:21 Joseph Nix: You could definitely do that. Nginx is a great way to segregate those sorts of things so you can have a static site generator made with Lektor, for instance, and a Flask part of your website that is dynamic content but not the blog because the blog doesn't need that, not the flat landing pages because it doesn't need that and you could even go and use JavaScript on a static page to query dynamic content if you wanted.

13:48 Michael Kennedy: Right. If you wanted to go down the sort of rich front end web framework type thing like Angular, React, or whatever, right? That could be all fully static but then maybe there's a Flask API for example.

13:59 Joseph Nix: Exactly. So, you could even shimmy in some dynamic content into a static site and how easy that is depends on, of course, how comfortable you are with all those technologies and the complexity of your dynamic content. So, the more complicated that gets or the more comfortable you are with Python instead of JavaScript, maybe the more inclined you should be to use to Flask or Django.

14:23 Michael Kennedy: Yeah, that makes sense. So, maybe give us some examples, you know, we talked about Lektor. Before we jump, well, let's take one step back, one thing that I think we did, obviously we touched on the deployment and simplicity. One thing we haven't touched on is just the performance of static sites. Obviously, we talked about the CDN and that's good but the consequences of high performing sites are really important, right? Like, I think Amazon did a study saying like for every 500 milliseconds of latency they had they lose like 1% of sales or something like that.

14:58 Joseph Nix: It's a big deal.

14:59 Michael Kennedy: Yeah and I'm sure it's not literate, right? Like you add that fifth bit and it's like, it's eventually people are going to go away and it's going to become nonlinear but that's important, right?

15:09 Joseph Nix: Yeah, that margin is very important for most businesses and even if you're just trying to make your own personal blog become more popular, it matters to you if you're user base grows by one or 10%.

15:22 Michael Kennedy: It matters and it also, I think it feels different when you're on a site that you're like dang that is basically instant. Like, that feels almost instant to me as I work with it whereas the other one, you're like, like if anyone works with SharePoint, they know what the opposite of that feels like, right? It's just like slow, painful, like, I'm waiting again, and like it just, even if it doesn't make people go away, it engenders a different experience with your consumers or your people.

15:51 Joseph Nix: Absolutely. It's got an effect on your mood. Even if it's subconscious, even if it's slight, it makes you a little less happy, a little more agitated if it's slow.

16:00 Michael Kennedy: This company doesn't really delight me, right?

16:03 Joseph Nix: Right.

16:04 Michael Kennedy: Things like that. So, and I think Google, you can never be sure what Google cues off of because they would be massively gamed as soon as they let that out of the bag. But I think site performance also effects ranking.

16:18 Joseph Nix: Yes, I believe it does. They've got, Google's got their own tools even to track site speed and if you use a static site generator to produce static sites, you're generally going to score pretty well about that.

16:31 Michael Kennedy: Exactly. Yeah.

16:33 Joseph Nix: You can do little things to make your static content even faster and to optimize above the fold content and things like this.

16:41 Michael Kennedy: Like minification or bundling, things like that?

16:44 Joseph Nix: You can minify, you can compress, you can make sure that all the CSS that's relevant to content that you see above the fold that is when you first load a page and not below it when you scroll, you can make sure that CSS is loaded first so that there's an even smaller flash of unstyled content.

17:03 Michael Kennedy: Yeah, that's a cool idea.

17:04 Joseph Nix: Yeah, so you could do things like that but if you simply just have static content on a CDN, no matter what, you're going to try to make it really slow.

17:14 Michael Kennedy: That's absolutely right, that's pretty awesome. All those benefits acro to those and I think the static site that is also sort of data driven but then maybe mix in either like backend services over JavaScript or like a section, various sections that are fully dynamic. I think that's a pretty interesting way to think about going beyond just well I have a couple pages in a static site.

17:35 Joseph Nix: Definitely. You can interplay them, you can mix and match, and that will increase the speed of all the static components which will make your users happier.

17:45 Michael Kennedy: Yeah, absolutely. This portion of Talk Python to Me is brought to you by Linode. Are you looking for bulletproof hosting that's fast, simple, and incredibly affordable? Look past that bookstore and check out Linode at talkpython.fm/linode. That's L I N O D E. Plans start at just five dollars a month for a dedicated server with a gig of ram. They have 10 data centers across the globe so no matter where you are, there's a data center near you. Whether you want to run your Python web app, host a private Git server or file server, you'll get native SSD's on all the machines, a newly upgraded 200 gigabit network, 24/7 friendly support, even on holidays, and a seven day money back guarantee. Do you need a little help with your infrastructure? They even offer professional services to help you get started with architecture, migrations, and more. Get a dedicated server for free for the next four months. Just visit talkpython.fm/linode. So, let's talk about some of the tooling that we might use for this. So, Lektor, obviously, is pretty awesome.

18:49 Joseph Nix: Yes.

18:51 Michael Kennedy: But it's not the only one in the Python space, right? There's a couple others. Maybe you could just like touch on those? Just so people know kind of what, if they see built with something at the bottom and are like oh, that was a static site generator.

19:01 Joseph Nix: Python or not, there's a handful of static site generators out there. A couple popular ones, I think, are Pelican and Hugo. Hugo's definitely very popular, I think that's running Go, and they predated Lektor, I believe. What they do is they use templating and Markdown so they try and make it pretty easy for you to make your content in Markdown and then they have a little backend functionality with template context so you can query things as you're building your static content so that you don't have to write the same HTML over and over and over again, which is really handy.

19:38 Michael Kennedy: Right, like an overall layout page with navigation, a footer, CSS, and JavaScript, etc. right?

19:44 Joseph Nix: Yeah. Not having HTML templates is a thing of the past that people should not be doing anymore. People--

19:52 Michael Kennedy: But I deploy them from front page, come on.

19:56 Joseph Nix: Right. People do still do them, that's absolutely true but I don't think they really should anymore. So, using these templating languages and these static site generators make people's lives easier so they can not have to extend very much effort to expand their website and a lot of these static site generators have themes so you can build off of the theme too which is really handy for a lot of people so they don't have to come up with their own CSS at least from the back but Lektor came after these and it learned from some of their deficiencies too. Because, again, going back to the history, I think static site generator's came out after the more complicated web frameworks did and these servers, so, they were a step back from the initial reaction against pure HTML. So, pure HTML was horrible then it went to web frameworks and then we got static site generators that kind of existed in between. Lektor has some of the, has all the features of the static site generator's but it's got some of the additional features that the web frameworks have that make life even simpler.

21:07 Michael Kennedy: Okay, like, what?

21:08 Joseph Nix: Like a CMS. So, as far as I know, Lektor is the only static site generator that comes with a built in CMS, out of the box, ready to go.

21:15 Michael Kennedy: Yeah, that is pretty interesting. So, I guess, maybe let's do a high level fly over of just how it works so with Lektor, Lektor is a thing you install onto your machine and then it's a command line application you run to generate the static's file site structure and then you can use it also to serve up that content in a way that will let you basically click a button, like a little edit button on any page, type away on it, and it'll under the covers write or save the markdown and then live reload that as you develop your site. I guess there's probably like some kind of Lektor deploy step that like finally builds it out to the final HTML?

21:59 Joseph Nix: Yes, there is. So, it is a program that you download and install and you run it over the CLI but you don't have to interact with the CLI very much to use it.

22:11 Michael Kennedy: I had it generating sites and I only used two commands.

22:13 Joseph Nix: Right.

22:14 Michael Kennedy: Lektor Quick Start and the Lektor server and then I like, I was kind of good, I was playing with it, it was fun.

22:19 Joseph Nix: Good. And that's how it should be. So, the previous static site generators would mean that you had to either open Markdown and edit it in whatever editor you like or just do everything in the terminal and not everybody's comfortable with that. We're trying to make static site's accessible to people who aren't necessarily programmers.

22:41 Michael Kennedy: Well and static site's are especially beneficial to people who are not capable of like all the CLI stuff and running data bases. It's exactly the people who don't want to do that, the Flask, Pyramid, SQLAlchemy type stuff for whom this really benefits 'cause the deployment and like running the site, it's crazy easy and so yeah, having that sort of focused in that area is really nice.

23:04 Joseph Nix: Right.

23:05 Michael Kennedy: So would you say like if you're working in a company and somebody comes to you and says hey, can we build this new site, I'm thinking about using SharePoint or something like that because we have to be able to keep track of XY or Z or whatever, do you think Lektor might be like hey, let's not use that, let's set up a Lektor thing and you can actually use the CMS and manage it and stuff like that. Would that be good?

23:27 Joseph Nix: It depends on a few things like if someone needs complicated user management, they want to segregate employees and say certain employees have certain permissions and others don't then Lektor is not a fit for that.

23:42 Michael Kennedy: It does make sense. Yeah 'cause static files don't have great permission role things. Other than at the OS level.

23:50 Joseph Nix: Right. And that's actually part of the beauty of it, really, is because there's no user permissions, everybody can do everything and it makes things simpler. It takes away a big chunk of complexity out of the system.

24:04 Michael Kennedy: Yeah. That's the trade off, right? It's that you get the super fast, super simple system but it doesn't do everything, right? You could make it do everything but then it wouldn't be simple, it would be hard again. It would be SharePoint.

24:14 Joseph Nix: What I have seen people do is they will run Lektor on a work LAN and they'll have all of the employees use the same Lektor setup so they can all edit pages as they will and if you trust your employees to, you know, not make horrible mistakes, then that works really well. You don't necessarily need all those permissions.

24:36 Michael Kennedy: Yeah. I feel like a lot of companies go too far and they like, well, we got to lock everything down and restrict people on whatnot and like, it just blows my mind like if you don't trust these people, why are you employing them? Like, if literally you're like, I will not trust this person to touch this file, like, that person should probably just not work here as of this afternoon because like that is a strong statement for so many things but I see it over and over in these companies, right? I mean, I guess there's a difference between protecting yourself from like, I clicked on a phishing link but there's a lot of these cases where it's like clearly we don't trust our employees and like, well, why are you employing them, right?

25:11 Joseph Nix: Right. And I get to concerns about people doing things by accidents and not like maliciousness but accidentally deleting your content. But you can protect against that in other ways, too. You don't necessarily need complicated user permissions. Dropbox even has backups you can take.

25:28 Michael Kennedy: That's right.

25:29 Joseph Nix: You can even have someone who is maintaining the Lektor deployment locally on a LAN and five content producers adding and entering content but the one person actually making Git commits every now and then by hand because they know what they're doing.

25:46 Michael Kennedy: Right. You could even automate that, right? Just every hour, commit the site. Whatever it is, it's like this time of day we commit it and that way at least you have a way you can rollback. There's a version history there.

25:56 Joseph Nix: Exactly.

25:57 Michael Kennedy: Yeah, yeah, yeah, very cool, very cool. So maybe let's do like a rundown through some of the core, core features, because I think they're pretty awesome. We've been sort of touching on some of them. The first one is around deployment, right? So, it's basically, you can deploy it anywhere. So, I could go and I could setup something kind of probably over the top like an EC2 Linux machine or Windows machine, copy my files up there, and serve it there. Maybe I could setup like a five dollar Linode server which would be pretty sweet but being static files, there's even like more options, right? Like, I could go way basic, I could put it on S3 or where else could I put it?

26:37 Joseph Nix: Yes, well, just like you're saying, because it's a static site, it can go anywhere. It can go on any CDN at all, basically. You could put it on S3. What we do for our own personal website is we host it on S3 and then have CloudFront take care of it from there, so, it's got an S3 backup and it's really just using a CDN and putting your site entirely on a CDN is groundbreaking, you know? When I figured out I could do that, I was all about it. It's blazing fast now and I don't have to worry. It's super easy.

27:12 Michael Kennedy: There's no denial of service. But I guess another thing that's pretty interesting about that is like if I open up my uWSGI log, which is the worker processes for my websites, and I watch them, a lot of times people are coming requesting things that I want them to request but, you know, periodically they'll be going to wp-login.php or some other malicious thing. My site is not built in WordPress, it's Python, so I don't really care, like I just like block those people but it's very clear that there's like a continuous sort of low burn attack against your servers, against your web server infrastructure and your site and it's super hard to like SQL inject static on a static site, right?

28:02 Joseph Nix: Exactly.

28:02 Michael Kennedy: And these other things.

28:04 Joseph Nix: If you have no logic backend and you have no database that's going to be a compromise then you don't have to worry.

28:10 Michael Kennedy: That's right. I mean it's down to down CloudFlare itself get, or CloudFront itself get hacked, right? Is that a thing that can be done? And if it can't, then you're good.

28:22 Joseph Nix: And even if it can, it's kind of not your problem.

28:24 Michael Kennedy: Yeah. You could be defaced but it's not like you're going to be in the news because now a million recordings from a teddy bear between kids and their teddy bear are now on the internet.

28:37 Joseph Nix: And honestly, you probably won't even be defaced. If CloudFront's having any sort of problem, it's probably going to effect more than just you.

28:42 Michael Kennedy: Yeah, exactly.

28:43 Joseph Nix: It also is better for security because that means if any problem happens, it'll get a lot of attention and get fixed faster 'cause the rest of the world is looking at it, not just your users.

28:56 Michael Kennedy: So, it's cross-platform, right? It works like the tooling to generate the static content. I mean, obviously HTML's cross-platform but it does take some tooling to power the CMS to generate the data files that are then built into HTML, right?

29:10 Joseph Nix: Right. Lektor is built in Python and it runs on Linux and Macs and Windows, it should run everywhere. And the CMS that Lektor runs internally is just front end content, it's built in React, it's you open up a browser and you see your CMS.

29:28 Michael Kennedy: Yeah, it's pretty cool. There's no database but there are, it acts like a database, right? There's a set of files and a file folder structure basically of Markdown and then these INI files, right? Tell us maybe a bit like at the inner play of page templates, the Markdown, and these like sort of definition files.

29:48 Joseph Nix: Frameworks had like Django and with Django you have model files and you've got admin files and you have databases that you connect to and Lektor is kind of faking that but it's making it a lot simpler and accessible. So, instead of a model's file.py and writing Python code, we have simple INI files where you can define fields and content types and page types and all of that data is stored in flat files on your system and then the templates have access to all of that content in their template context so you can use the templates with the Jinja tags to query for body content for titles, for URLs or check boxes or anything like that.

30:39 Michael Kennedy: That's really clever. So, let me repeat this back to you, see if I got that right. So, we have these INI files and they define like basically what data is in a given type of page and you have different types of page like blog posts, landing pages, whatever, so like you could say a blog post contains a title, content, an author and a published date, for example. And then, you have a Markdown file that has those various pieces of information entered into them and finally you have Jinja2 template that takes that bit of information and then renders it however you would in a regular HTML, it's like a dynamic HTML expression of that.

31:20 Joseph Nix: That's all pretty close but I think you got a little bit wrong. So, the information isn't stored all in one Markdown file, it's stored in a Lektor proprietary format that is just a flat file of data.

31:33 Michael Kennedy: Oh, right.

31:35 Joseph Nix: But you don't see that most of a time as a user. What you see is you interact with the CMS. So, you define the INI file which is a model file and then you go to the CMS and you make a new page of a certain type and the CMS will show you all the content fields that you need to populate. So, the publish date and the author and the body content are all separate fields in the CMS.

31:58 Michael Kennedy: And do they have types?

31:59 Joseph Nix: Yes. And those fields can be of type markdown or string or date or integer or checkbox or anything like that. So, the template, you can query for the specific fields and do template logic on those fields if you want or just simply display them, it'll render the markdown for you if the type is a markdown file or markdown content.

32:26 Michael Kennedy: That's pretty cool. Another thing that's pretty nice for large sites is dependency tracking, right? So, initially, like who cares, it's just a few files but, you know, you could easily see over time something with a thousand pages. Right? Like if, in say, a bookstore, like a data driven bookstore, you might have landing page, a category listing page, and an individual book details page. It might just be three pages but the data drives like thousands of distinct URLs. In a static site generator, you literally build a thousand pages, right? And then generate because your database is the stuff. So, with the dependency tracking, like, that could be a big deal, right? So, what's the story with that?

33:13 Joseph Nix: That can be a big deal. So, you're swapping out a database like Postgres for flat files. That has certain trade offs, right? Postgres is optimized on the file system and it's fast. Flat files are not optimized for size or speed but they're much easier to read by human and that means you can actually version control all those. It's not binary anymore, you're actually storing the text from the various fields of these books or the author information or publication information.

33:49 Michael Kennedy: So, maybe you changed like the details about one book and you want to read and play, you don't want to regenerate a thousand files. You want to just regenerate that one HTML file, right?

34:01 Joseph Nix: Right. And that's what would be generated. So, if you're changing information about one book then there would be a single file that is touched that's devoted to that one book in your flat file system in Lektor and that would have a line change in it when you change the author information, you're correcting a typo, and then Lektor would notice that change and auto-rebuild the relevant HTML file for it and it wouldn't build all of it. When you run Lektor server the first time, it builds all of your files but then it just is looking for changes and it builds files for anything that is touched.

34:39 Michael Kennedy: Yeah, it's nice. It'll even do that like as you interact with them either through the file system or through the CMS, right? Yeah, it's pretty slick.

34:45 Joseph Nix: And unless you're changing content that's actually relevant to a thousand pages, you're not going to build a thousand pages all over again.

34:51 Michael Kennedy: Yeah, that's real cool. This portion of Talk Python to Me has been brought to you by Rollbar. One of the frustrating things about being a developer is dealing with errors. Relying on users to report errors, digging through log files, trying to debug issues, or getting millions of alerts just flooding your inbox and ruining your day. With Rollbar's full static air monitoring, you get the context, insight, and control you need to find and fix bugs faster. Adding Rollbar to your Python app is as easy as pip intall rollbar. You can start tracking production errors and deployments in eight minutes or less. Are you considering self-hosting tools for security or compliance reasons? Then you should really check out Rollbar's compliance SaSS option. Get advanced security features and meet compliance without the hassle of self-hosting including HIPAA, ISO 27001, Privacy Shield and more. They'd love to give you a demo. Give Rollbar a try today. Go to talkpython.fm/Rollbar and check them out. Too it has builtin image sort of thumbnailing tools and stuff, that's pretty nice. It has a plugin system, so that sounds like you could really take sort of Lektor out of the box and then make it really adaptable the kind of the way you might with say a traditional data driven app. You can also mess with the Jinja templates which is sort of a programmable layer as well, right?

36:14 Joseph Nix: Yes. So, you can do a lot with the plugins to something and a lot has been done. We've got a handful of, more than a handful of plugins on our website. There are plugins that change functionality inside your Jinja templates. There's plugins that change how your Markdown is rendered so if you want special formatting and you want to enable certain niceties in the markdown.

36:36 Michael Kennedy: Like GitHub Flavored table sort of thing?

36:39 Joseph Nix: Yeah, you can change the renderers of the lexors in the markdown, you can add classes so you can have special markdown syntax that's you're own and have the rendered markdown use custom HTML classes. There's admissions, there are plugins that allow you to deploy to various different places like the S3 deploy isn't builtin in Lektor that's provided by a plugin as opposed to GitHub page deploys which is native.

37:14 Michael Kennedy: Now, that's pretty sweet. So, you get to go in there and like, sort of tell the plugin your API key and they just run a command and then you have a new version online?

37:24 Joseph Nix: That's right. And you can hook into, so you can write a plugin to deploy to anywhere you like.

37:30 Michael Kennedy: Man, that's awesome. It also has a Python API, so, is that just like a different angle to get in there instead of writing a plugin, maybe from the outside to the inside or, like, what's an example of what I might do with the API or what I could do?

37:46 Joseph Nix: Most common use of the API is within the plugins itself, it's telling you how to access every part of the Lektor system through a plugin. So, the plugins have various events, various event hooks inside the Lektor code that they respond to, so, when something happens, it trips an event and any plugin listening on that event can have the opportunity to do something but that plugin has access to all of the Lektor source files in Python so it can import everything and change anything.

38:19 Michael Kennedy: I see. That's pretty cool. So, maybe some part, let's say, for an example, like part of your site depends upon data you might get from an API or something like that. Like a web service, right? That it's not part of your site. Maybe at build time you just like go and hit that, refresh it, and then dump it in there and render it, so it's kind of live, it's sort of cached from the moment of generation anyway.

38:44 Joseph Nix: Yes, you could do that, definitely. You could do that. You could change the types that are in Lektor, you could add a content type that's entirely new, a field type. You could generate entirely different kinds of pages. There are plugins that add template functionality to allow Atom feeds and Disqus really easily.

39:06 Michael Kennedy: Right, right. Disqus is pretty nice and yeah. I guess you could even like regenerate like an RSS file sort of thing, right? Say, for blogs or a story, you wouldn't have the new items or whatever.

39:19 Joseph Nix: Yes. There is an RSS Atom feed plugin.

39:22 Michael Kennedy: Yeah, very cool. So, one of the areas that I was, when I was playing with Lektor, I'm like, this thing is really nice. One thing I would like to see when I was playing with it is maybe ways to configure how it looks, right? 'Cause it comes out and it looks like it looks but I'm like oh, can I like, you know, one of the things I love about Bootstrap is like themes, right? I can go to like say Wrap Bootstrap or something like that and go oh, here's a bunch of stuff, I'm not a web designer but that looks really close to what I want, let's see if I can hack that into the shape I want it, you know? What's the story around themes for what you guys are doing?

39:56 Joseph Nix: Themes are really new in Lektor, actually. They did not exist prior to a few months ago but as of Lektor 3.1 which came out in January or February, there is now themeing functionality in Lektor. There's not a lot of themes available yet but they're growing in numbers slowly. So, people should go ahead and try this out and make some and publish them for us, we'll add them to our list.

40:22 Michael Kennedy: Yeah, that sounds really good.

40:23 Joseph Nix: So, themes allow you to easily modify the look and feel of your website, of course, but in Lektor they can do more than that, they can add content type, you can have models in your themes even.

40:36 Michael Kennedy: I see. So, if I wanted to have a theme for like a photography store, I could have a photo model type that has like the picture and the little copyright information and the date and things like that?

40:51 Joseph Nix: Yes, absolutely. And, as you would expect with a theme, anything can be overwritten or extended, too. So, when you're using a theme, like a photography theme, you could easily add your own models for videos if you also do videography.

41:06 Michael Kennedy: I see.

41:07 Joseph Nix: If you decide you don't like, after using the theme for awhile, you don't like the model and you want to extend it, you can do that too.

41:13 Michael Kennedy: Okay, yeah, that sounds really nice. So, when I went through and sort of created this little playground site that I was playing with, I went through and I installed Lektor, I did that off of GitHub and then I typed Lektor Quick Start and I got the site then I played with it. Like, how would I incorporate themes into that? Like, how do I know what themes are there? Is there a way to type like Quick Start Photography1 or whatever?

41:40 Joseph Nix: There is not a command line tool to just grab the theme so easily yet but we've got some issues on the GitHub for making that. Right now what you would have to do is where you have your Lektor code, you make a themes directory and in that themes directory you put any themes that you like and the themes that exist right now are in their own repository on GitHub, Lektor themes, so you can check those out and download or clone any of those and they will be picked up by your Quick Start. You'll have to, the Quick Start provides its own default styling and templates and models. So, when you add the theme, you'll have to remove some of the ones that Quick Start added because they will be overriding the theme.

42:31 Michael Kennedy: Right.

42:32 Joseph Nix: Because that's a feature of the themes is that they're overridable.

42:35 Michael Kennedy: Right, right. Well, that sounds pretty cool. I think maybe a really cool plugin would be like list all the themes, activate the themes, who knows. Would it be possible to write one that did that? Seems like it would.

42:47 Joseph Nix: It would. Though that sounds like such a good feature, I'd like it in Lektor core.

42:52 Michael Kennedy: Yeah. Yeah, that's true. It should just be builtin, right? Maybe some experimental stuff could be done as a plugin and then it could be brought in, right? Kind of like maybe Python standard library style.

43:03 Joseph Nix: Yeah, could be. But definitely themeing is new but it's very exciting so I'm trying to push the themes because I think that will make people's lives a lot easier.

43:13 Michael Kennedy: Yeah, I think it's really, really great and it definitely could open up, sort of, like a way for people to reuse the design of sites they like. There is a showcase, right? At getlektor.com/showcase of people or sites that people have created which is pretty cool. A lot of soon-to-be created by Armin Ronacher of Flask Theme, like what, he was involved in the early days of this project, is that right?

43:38 Joseph Nix: Yes, absolutely. Armin is the creator of Lektor.

43:40 Michael Kennedy: Yeah, that's kind of what I thought but I wasn't 100% sure.

43:43 Joseph Nix: Armin is the creator of all the pallets projects and he's creator of Jinja and Werkzeug and Flask, so he's done a lot and Lektor uses many of those tools and wraps them up nicely.

43:57 Michael Kennedy: Yeah, when I typed Lektor Server, I got look, oh, there's 5,000 of them, like, I see Flask, okay.

44:03 Joseph Nix: Yep. Absolutely. Flask is what's driving the CMS. So it's a Flask server using a bunch of React.

44:11 Michael Kennedy: Okay, nice.

44:12 Joseph Nix: And the showcase, since he made Lektor and he made GetLektor's originally, getlektor.com, the original website, the first few pieces of the showcase are his websites but you'll scroll down, you can see others. One that I really love is a Python website that is on the showcase. It's a beautiful example of a single page website.

44:39 Michael Kennedy: Yeah, Pack on Philippines, yeah, nice. Yeah, that's a good looking site. Very cool. So, speaking of single page, maybe the last thing we'll have time to touch on is you have a bunch of guides, so, there's a Quick Start but it's like all right, well, I want to create something that has categories or Disqus things or how do I deal with error pages, right? So you have a bunch of guides that go through those types of things. Do you want to tell people a bit about those?

45:04 Joseph Nix: Categories, for instance, is a really handy guide because it's a very commonly wanted feature. You want to be able to have projects or pages that are cross-linked somehow by certain categories.

45:17 Michael Kennedy: Right. It's one thing if it had like one category, you just put it in that subfolder and it's fine or something, right? But if it could be like in science and engineering, like, oh boy, now you're in trouble, right?

45:27 Joseph Nix: Right. Exactly. Because you could manually create all of this and make them all separate pages but that's not using a lot of the power that's available to you.

45:36 Michael Kennedy: Right. And data duplication as well.

45:38 Joseph Nix: And though Lektor can be used in very simple fashion, some of the power is a little tricky. So, we've got some guides to help sort that out for you. So you don't have to create all of the Jinja logic and the macros and your templates to figure out how to appropriately use the categories and how to setup the models so that you have the categories functionality.

46:03 Michael Kennedy: Yeah, that's really cool. Yeah, so you got the Disqus comments, the error pages, which, you know, is always a nice thing to have. Pagination, yeah.

46:10 Joseph Nix: Pagination is critical, especially if you have a blog. There's a guide for single page apps which is really nice because if you have in mind a static site generator building an HTML file for every page that you have and every page is a different topic, then it might not be obvious to you how to concatenate or combine all of that content into a single page which is what you really want sometimes.

46:37 Michael Kennedy: Yeah, yeah. Like, just show me the latest flat down the list in order, things like that. Sitemap is also sitemap.xml. We talked about the SEO benefits of static sites, like, here's a big one.

46:50 Joseph Nix: Yes. So, there's a quick guide to get that up and running which is really handy. There's also a web pack guide, so, it can get you started integrating some of the fanciness of the rich JavaScript tooling.

47:06 Michael Kennedy: Webpack does all sort of interesting things like SCSS or less or bundling or a whole bunch of those types of things if you want them, right?

47:16 Joseph Nix: Yeah. So, you could even think of fairly complicated React app on top of Lektor if you wanted.

47:22 Michael Kennedy: Yeah, that's cool. I've looked at using Webpack for my sites and I decided I would rather have something simpler and I just use like super aggressive caching like cache this for a year, everything that's static just cache it for a year. So, the very first time you might hit four JavaScript files and three CSS files but you'll never get them again so, like, I'm not going to go through the layers of trying to regenerate that stuff but I know a lot of people make use of Webpack and it's, you know, it works for them.

47:50 Joseph Nix: Yep and there's a lot of Webpack functionality around modifying your build system too so you can minimize all of your content. We were talking before about minimizing even a static site, so a lot of that can be done with Webpack or Browserify or anything like that.

48:07 Michael Kennedy: So, where does continuous integration fit into this whole story of static sites and Lektor? Like, does it make even sense to setup continuous integration and have something like Travis CI or the various, you know, servers. Like, looking at your, possibly your Git history and checking those out and verifying everything like compiles correctly with Lektor?

48:29 Joseph Nix: You can wrap up tests in Travis and you can automate builds with Travis. Our own website is actually done like that so every time we push a commit to the Lektor website repository on GitHub, Travis goes and it builds all of that and if it's good then it deploys it. So, that's really handy.

48:50 Michael Kennedy: Just use like the S3 plugin and just like run the command on successful build?

48:54 Joseph Nix: Basically. It's not using S3 but it is deploying on a successful build.

48:57 Michael Kennedy: Nice, okay. Well, that's really, really cool.

49:00 Joseph Nix: Yeah. And you can automate all that with other things too. It doesn't have to be Travis, you can make it on a cron job if you wanted, anything like that.

49:07 Michael Kennedy: Yeah. But, I guess it doesn't depend as much. It's not like, there's not as much logic but I guess you could still screw up one of your Jinja templates or something like that, right? So, you might want to test.

49:19 Joseph Nix: Yes. And that's not too hard to do. Lektor is appropriately robust, I think, in how it outputs information, it'll tell you when it's building all the webpages and it'll tell you their success or failure so you can look out for those basic failures. So, if you can test for the page building successfully, that won't, of course, test if the content is correct, so if you made some typos, then you'll still have to catch that by hand.

49:48 Michael Kennedy: Maybe something with Selenium against, like, I don't know. Something local, who knows, but not, a build server's not going to do it for sure. Nice. To me it sounds like Lektor is this, fills a pretty interesting middle ground, I guess between full stack stuff and static sites.

50:05 Joseph Nix: Having models is a game changer for a static site and the CMS baked in is a really easy way to use it. So, you have a lot of the features that, for instance, Django would have except for the actual backend processing which you often don't need but that doesn't mean you don't want models, you don't want to define different content types and different data structures, you just don't need the additional complexity of actually using Postgres or having backend Python.

50:37 Michael Kennedy: You don't need the servers and the databases and all of that comes with taking those onboard, those are more like puppies, right? Once you get them you have to raise them and care for them continuously basically.

50:49 Joseph Nix: Oh, yes, and there's loads of headache that you avoid and not having to do that too. You don't have to manage your own. CI/CD becomes really simple, you don't have to worry about security as much as we talked about and it's placing fast all on top of that.

51:02 Michael Kennedy: And scalability, denial of service, these are not real issues, yes?

51:06 Joseph Nix: Right. We let the CDN handle that.

51:09 Michael Kennedy: Yeah, exactly. Very cool. All right, so, let's leave it there for Lektor but I'll ask you the two final questions before I let you out of here. So, if you're going to work on some Python code, what editor do you use?

51:21 Joseph Nix: For years I've been an Emacs user.

51:23 Michael Kennedy: Oh, right on.

51:24 Joseph Nix: I've got my own .emacs and Git repo that I can't live without.

51:29 Michael Kennedy: Nice. So, it's right there. Anytime you got to check it out, you're all good, right?

51:33 Joseph Nix: Yep. a .emax.d.

51:35 Michael Kennedy: Yeah, perfect. And then notable PyPI package. Obviously there's Lektor, right? Which is great but what else?

51:41 Joseph Nix: A personal favorite of mine for DevOps is SaltStack. I've been doing a lot of DevOps at Terminal Labs the last couple of years and SaltStack is enormously feature rich. It can do almost everything I can think of. I mean, we've got our own opensource project named Rambo which is built on top of SaltStack partially to make DevOps even easier. We're building it as a provider agonostic provisioning framework, so.

52:07 Michael Kennedy: Oh, nice.

52:10 Joseph Nix: Rambo makes great use of Vagrant and SaltStack to make a lot of things a lot easier. So, it helps get up and running with a lot of things pretty fast. The other thing, which isn't exactly a pip package that I really want to plug is Conda. Are you familiar with Conda?

52:25 Michael Kennedy: Yeah, Conda's awesome. Especially if you're trying to do something on Windows involving data science and vcvarsall.bat or something like that, right? Just basically pip installing a lot of the data science tooling requires like funky compilers that are not easily accessible on Windows and this pre-builds it and you kind of get everything pre-packaged and ready to roll, right?

52:47 Joseph Nix: Absolutely. Even outside of data science and machine learning, but just normal Python development, Conda has saved my butt more than once. You never know, it always surprises me when I try and pip install something that has some C dependency that I didn't have on my system that it doesn't work so I would either have to Google around and fight it for an hour or two, trying to figure out how to install it or use Conda and it just works.

53:13 Michael Kennedy: Conda install, forget this thing, we're out of here. That's cool. Yeah, really nice, really nice. All right, final call to action, people are all excited about static sites, about Lektor, what can they do?

53:23 Joseph Nix: Go check it out firstly. Try it out, hopefully you love it. If you do, please, if you're a Python developer, try contributing. We have an active Gitter and I'm pretty active in the community, I'll try to respond to anything.

53:36 Michael Kennedy: Yeah, I think that's how I actually got in touch with you is I found you on the Gitter channel, yeah, so it was cool.

53:40 Joseph Nix: Right. Well, I would love to see more plugins being made. You can do anything with a plugin and we also would love to see more themes. So, if you are a theme developer and, you know, not a Python developer and you want to spread your theme around to the Lektor world instead of just Hugo or WordPress then come on over.

53:57 Michael Kennedy: Yeah, that sounds awesome. All right, well, I think Lektor's pretty cool, people should check it out, and Joseph, thanks for being on the show, it was great to talk about all this stuff with you.

54:06 Joseph Nix: Thank you.

54:07 Michael Kennedy: This has been another episode of Talk Python to Me. Our guest on this episode has been Joseph Nix and the episode was brought to you by Linode and Rollbar. Linode is bulletproof hosting for whatever you're building with Python. Get four months free at talkpython.fm/Linode. That's L I N O D E. Rollbar takes the pain out of errors. They give you the context insight you need to quickly locate and fix errors that might have gone unnoticed until your users complain, of course. And as Talk Python to Me listeners track a ridiculous number of errors for free at rollbar.com/talkpythontome. Want to level up your Python? If you're just getting started, try my Python Jumpstart by Building 10 Apps or our brand new 100 Days of Code and Python. And if you're interested in more than one course, be sure to check out the Everything Bundle. It's like a subscription that never expires. Be sure to subscribe to the show, open your favorite podcatcher and search for Python. We should be right at the top. You can also find the iTunes feed at /itunes, Google Play feed at /play, and direct RSS feed at /rss on talkpyton.fm. This is your host, Michael Kennedy. Thanks so much for listening, I really appreciate it. Now, get out there and write some Python code.