Zensical - a modern static site generator
Episode Deep Dive
Guest Introduction
Martin Donath (known online as Squidfunk) is an independent developer and consultant with over 20 years of experience. He writes primarily in TypeScript, Python, and Rust. Martin is best known as the creator of Material for MKDocs, one of the most popular documentation frameworks in the Python ecosystem, used by tens of thousands of projects including AWS, Microsoft, OpenAI, FastAPI, and uv. His latest project is Zensical, a modern static site generator built from the ground up with a Rust core to solve the scalability and flexibility limitations he encountered over a decade of building Material for MKDocs. Martin describes himself as a "free spirit" who loves building products from front to back.
- github.com/squidfunk -- Martin Donath's GitHub profile
- zensical.org -- Zensical project website
What to Know If You're New to Python
If you are newer to Python and want to get the most out of this episode, here are a few things worth knowing:
- Static site generators take plain text files (usually Markdown) and turn them into complete HTML websites. You do not need a database or server to host the result. This episode centers on a new Python-installable static site generator called Zensical.
- pip install is Python's standard way to install packages. Martin mentions that getting started with Zensical is as simple as running
pip install zensical. Understanding virtual environments and pip is essential context for this episode. - Markdown is the lightweight markup language used to author content for tools like Zensical and MKDocs. If you have written a README on GitHub, you have likely already used it.
- PyO3 is a library that lets Rust code interoperate with Python. It comes up frequently because Zensical's core is written in Rust but exposes Python bindings so users can write modules in Python rather than needing to learn Rust.
Key Points and Takeaways
1. Zensical is a new open-source static site generator from the creators of Material for MKDocs
Zensical is, in Martin's words, "a tool to build beautiful websites from a folder of text files." You write in Markdown, run a build, and get a complete static HTML site with modern design, built-in search, code annotations, admonitions, and more. It is fully open source under the MIT license and installable via pip. The project targets documentation as its primary use case today, but the stretch goal is a full knowledge management and documentation solution that could serve as an alternative to SaaS tools like Confluence and Notion. With a Rust-powered core, modular architecture, and differential builds, Zensical is designed to scale from a single page to 100,000 pages.
- zensical.org -- Zensical project home
- github.com/zensical/zensical -- Zensical source code on GitHub (MIT licensed, 84% Rust, 15% Python)
2. The MKDocs 2.0 breaking changes forced a fresh start
After nine years of MKDocs development, the original maintainer returned and began working on MKDocs 2.0 with a radically different direction: no plugin API and customization via templating alone. Martin and other key ecosystem maintainers were invited to a video call where they learned about these plans and raised objections, but they were dismissed. MKDocs 2.0 as planned would break Material for MKDocs, render roughly 300 ecosystem plugins useless, and affect tens of thousands of projects. Faced with this, Martin's team had two options: fork MKDocs or start from scratch. Forking was impractical because of how Python dependencies work -- all 300+ plugins have a hard dependency on the mkdocs package, and switching them all would be "like moving an entire city at once." So they chose to build Zensical from scratch with 10 years of lessons learned baked into the architecture.
- mkdocs.org -- MKDocs project site
- github.com/squidfunk/mkdocs-material -- Material for MKDocs (26.3k stars)
3. A Rust core with PyO3 delivers bare-metal performance with Python extensibility
Zensical's runtime (called ZRX) is written entirely in Rust and handles orchestration, threading, caching, and parallelization. Python module authors do not need to learn Rust because PyO3 provides bindings that let them write Zensical modules in Python while still benefiting from the Rust runtime's speed. Martin described Rust as "seriously one of the best languages ever made" for its expressiveness and bare-metal performance, though he was candid about the steep learning curve -- he spent a month banging his head against the borrow checker before making any progress. Samuel Colvin (of Pydantic fame) was the one who originally recommended Rust to Martin, and Martin called it the "best investment" he has made.
- rust-lang.org -- Rust programming language
- pyo3.rs -- PyO3: Rust bindings for Python
4. Differential builds make changes visible in milliseconds, not minutes
The number one complaint from Material for MKDocs users was that MKDocs is slow and does not scale. Fixing a single typo triggered a full rebuild that could take minutes on large sites. Zensical solves this with differential builds: only the pages and elements that actually changed get rebuilt. For example, if you change a page title, only that page and the places referencing it are rebuilt. This is critical for enterprise users -- Martin mentioned one company with 2,500 internal documentation projects built on Material for MKDocs -- where build performance directly impacts developer productivity.
5. Modules all the way down: a truly modular architecture
Unlike MKDocs, where the plugin system was somewhat of an afterthought with specific hooks that could be limiting, Zensical implements all core functionality as modules. This means any module can be swapped, extended, or replaced. Users can use the built-in modules, write their own in Python or Rust, or pull in third-party modules. Martin wrote 12 MKDocs plugins over the years and knows intimately where the old architecture's hooks fell short -- for example, making plugins cooperate was a constant pain point, with plugin ordering issues being one of the most common bug reports across the ecosystem.
6. Compatibility with Material for MKDocs is a top engineering priority
Zensical can already read mkdocs.yml configuration files, and the goal is to let users migrate their projects with a single command. The team deliberately uses Python Markdown (the same parser as MKDocs) right now to ensure Markdown rendering is identical, even though they plan to transition to CommonMark in the future. They are also mapping MKDocs plugin configurations to equivalent Zensical modules. Martin emphasized that if you force users to manually rewrite their documentation, you will not get adoption -- the migration path has to be easy, especially for enterprises with thousands of projects.
- python-markdown.github.io -- Python-Markdown library
- commonmark.org -- CommonMark specification
7. Disco: a from-scratch search engine that is 20x faster
Zensical includes a completely rewritten client-side search engine called Disco, which Martin says will eventually be open-sourced as a standalone project. It is 20 times faster than the search in Material for MKDocs. Most open-source search libraries use BM25 (a bag-of-words ranking algorithm) which does not pair well with autocomplete and produces "dancing results" as you type. Disco uses a tie-breaking algorithm that searches the title first, then the navigation path, then the body, with all of this being configurable. The search index ships as a single compressed JSON file (typically under 200KB after gzip), giving you instant client-side search with no hosting costs. Martin believes this approach works for 95-99% of documentation sites.
- pagefind.app -- Pagefind, another client-side search library mentioned in comparison
8. The Material for MKDocs origin story: from side project to $200K/year
Material for MKDocs started in 2015 as a side project when Martin wanted to release a C library called Protobluff and found existing documentation tools looked dated. He spent nine months building the first version, then maintained it on Sundays while doing client work during the week. As the project grew, maintenance crept into mornings and evenings, becoming harder to justify. The turning point came in 2020 when someone suggested a donate button. Instead, Martin tried an Amazon wish list and received everything on it within two days. This led him to survey his users and discover that only 7% were front-end developers -- meaning he had a unique competitive advantage in the Python documentation space. He adopted a "sponsorware" model where sponsors got early access to features, and each feature had a funding goal. Once the goal was met, the feature became free for everyone. This eventually grew to $200K per year in revenue, allowing him to build a team.
9. Zensical Spark: a new sustainability model for open source
With Zensical, Martin is moving away from the sponsorware model. Everything is open source from the start, free for all users. Revenue now comes from Zensical Spark, a membership program where companies get a seat at the table to help shape Zensical's direction. Spark members participate in high-level discussions, collaborate on Zensical Advancement Proposals (ZAPs), get professional support, and join open video calls with the development team. Martin frames this as a competitive advantage: by talking directly to enterprise users, the team learns things they "never would have come up with" on their own. The model focuses on creating "degrees of freedom" so organizations can adapt Zensical to their specific workflows rather than just shipping feature after feature.
- zensical.org/spark/ -- Zensical Spark membership program
10. The docs-as-code philosophy is driving enterprise adoption
Martin explained that Zensical builds on the docs-as-code philosophy, which means treating documentation exactly like source code. You use the same tools and workflows you use for code -- version control, pull requests, code review -- for your documentation. This approach is gaining fast adoption among companies moving away from proprietary tools toward open-source solutions. The key benefit is that non-front-end developers (93% of Material for MKDocs users) can write documentation in Markdown without fighting HTML, CSS, or JavaScript. They just want to "get the content out" and Zensical handles the rest.
11. The strange attractor: meaningful design backed by physics and math
The Zensical website features a mesmerizing animation that is not just eye candy -- it is a strange attractor, a concept from physics defined by just three equations that govern how points move in three-dimensional space. Simple rules produce complex shapes, which Martin says symbolizes "the process of evolving ideas through writing." There is even a hidden easter egg: hovering in the bottom-left corner of the homepage lets you change the coefficients of the animation in real time, potentially making it chaotic and unstable. The strange attractor also inspired the Zensical logo.
12. Moving to CommonMark and MDX for the future
While Zensical currently uses Python Markdown for backward compatibility, the plan is to gradually transition to CommonMark, which is a standardized Markdown specification used broadly across the industry. With CommonMark comes the possibility of supporting MDX, a dialect that allows reusable components to be embedded in Markdown -- something front-end developers have been using for years via tools like Docusaurus. Martin's team will provide automated tooling to help users migrate their Markdown from Python Markdown syntax to CommonMark when the time comes.
- commonmark.org -- CommonMark specification
- mdxjs.com -- MDX: Markdown + JSX components
13. Code annotations and rich components for technical writing
One of Zensical's flagship features, inherited from Material for MKDocs, is code annotations. These let you place a small bubble on any line of a code block; clicking it opens a tooltip that can contain rich Markdown content including lists, tables, and diagrams. Other built-in components include admonitions, tabs, data tables, Mermaid diagrams, buttons, and an icon/emoji integration with over 10,000 icons. All of these are designed so technical writers can focus on content without needing front-end development skills.
- mermaid.js.org -- Mermaid diagramming library
14. mkdocstrings integration and the growing Zensical team
Timothee Mazzucotelli (known as Pawamoy), the author of mkdocstrings -- the second biggest project in the MKDocs ecosystem -- has joined the Zensical team. mkdocstrings enables automatic API documentation generation from source code and is already supported in Zensical. This is significant because mkdocstrings is a critical tool for many Python projects, and having its creator on the Zensical team ensures a smooth path forward for its users.
- github.com/mkdocstrings/mkdocstrings -- mkdocstrings: automatic documentation from sources
Interesting Quotes and Stories
"In very simple terms, it's a tool to build beautiful websites from a folder of text files." -- Martin Donath, on what Zensical is
"MKDocs 2.0, as it looks right now, is incompatible with Material for MKDocs. 300 plugins in the ecosystem will become useless and tens of thousands of projects will be affected. And for us, we had absolutely no choice." -- Martin Donath, on why Zensical had to be built
"I basically banged my head against the keyboard for a month, wasn't making no progress at all because, you know, fighting with the borrow checker." -- Martin Donath, on learning Rust
"I think Rust is seriously one of the best languages ever made because it allows you to express ideas extremely clearly with extreme clarity. And you get bare metal performance." -- Martin Donath
"Spoiler alert, it did work. And in the end we made 200K a year of it and could build a team and everything." -- Martin Donath, on the sponsorware model his developer friends said would never work
"Forking is not really possible because of the way Python dependencies work. All of the plugins have a dependency on MKDocs. So we would also need to fork all plugins. It would be like moving an entire city at once. And it's frankly impossible." -- Martin Donath, on why they had to start from scratch
"Only 7% of users are front-end developers. So I kind of had an edge there in the Python space." -- Martin Donath, on discovering his audience
"We don't intend to just ship feature, feature, feature, but we intend to create degrees of freedom so that you can adapt Zensical to the processes within your organization." -- Martin Donath, on the Zensical Spark philosophy
"When I started it, I would never believe that this would be my job at some point." -- Martin Donath, on Material for MKDocs becoming his full-time work
Key Definitions and Terms
- Static site generator (SSG): A tool that takes source files (typically Markdown) and produces a complete website of static HTML, CSS, and JavaScript files. No database or server-side processing is needed, meaning sites can be hosted for free on platforms like GitHub Pages or Cloudflare.
- Docs-as-code: A philosophy of treating documentation the same way you treat source code -- stored in version control, reviewed through pull requests, and built through automated pipelines.
- Differential builds: A build strategy where only the content that has changed (and its dependents) is rebuilt, rather than regenerating the entire site from scratch. This is the core performance innovation in Zensical.
- PyO3: A Rust library that provides bindings between Rust and Python, allowing developers to write native Python extensions in Rust. Zensical uses PyO3 so that module authors can write in Python while the core runs in Rust.
- CommonMark: A strongly defined, standardized specification of Markdown, created to eliminate ambiguities in the original Markdown spec. Zensical plans to move from Python Markdown to CommonMark.
- MDX: A format that extends Markdown with JSX components, allowing interactive and reusable elements to be embedded directly in Markdown documents.
- BM25: A standard "bag of words" ranking algorithm used in information retrieval. Zensical's Disco search engine goes beyond BM25 with a tie-breaking algorithm for better relevance.
- Sponsorware: A monetization model for open source where sponsors get early access to new features, which become freely available to everyone once a funding goal is met.
- ZAPs (Zensical Advancement Proposals): Detailed written proposals on specific topics the Zensical team intends to work on, shaped in collaboration with Spark members.
- Strange attractor: A concept from physics where a system defined by simple equations produces complex, non-repeating patterns. Used as the visual identity and philosophical metaphor for Zensical.
- Monomorphization: A Rust compiler technique where generic code is specialized for each concrete type at compile time, enabling "zero-cost abstractions" with no runtime performance penalty.
Learning Resources
Here are resources to learn more and go deeper on the topics covered in this episode:
- Static Sites with Sphinx and Markdown: A Talk Python Training course covering static site generation in Python using Sphinx and Markdown, directly relevant to the documentation tooling discussed in this episode.
- Python for Absolute Beginners: If you are new to Python and want to build a foundation before exploring tools like Zensical, this course will get you started from scratch.
- #100DaysOfWeb in Python: Includes a dedicated section on building static sites with Python, along with broader web development skills that complement the topics in this episode.
Overall Takeaway
Martin Donath's journey from a Sunday side project to powering the documentation of AWS, OpenAI, and FastAPI -- and then having the courage to start over from scratch -- is a masterclass in long-term thinking for open source developers. Zensical is not just a faster static site generator. It represents a decade of hard-won lessons about architecture, community stewardship, and sustainable open source business models. The decision to build a Rust core with Python extensibility via PyO3 reflects the broader trend of Rust becoming the performance escape hatch for the Python ecosystem. And the emphasis on compatibility -- being able to read existing MKDocs configurations and providing migration tooling -- shows a deep respect for the community that supported Material for MKDocs for 10 years. Whether you are a technical writer, a Python developer maintaining docs, or an open source maintainer thinking about sustainability, this episode is packed with practical insight and genuine inspiration.
Links from the show
Martin Donath: github.com
Zensical: zensical.org
Material for MkDocs: squidfunk.github.io
Getting Started: zensical.org
Github pages: docs.github.com
Cloudflare pages: pages.cloudflare.com
Michaels Example: gist.github.com
Material for MkDocs: zensical.org
gohugo.io/content-management/shortcodes: gohugo.io
a sense of size of the project: blobs.talkpython.fm
Zensical Spark: zensical.org
Watch this episode on YouTube: youtube.com
Episode #542 deep-dive: talkpython.fm/542
Episode transcripts: talkpython.fm
Theme Song: Developer Rap
🥁 Served in a Flask 🎸: talkpython.fm/flasksong
---== Don't be a stranger ==---
YouTube: youtube.com/@talkpython
Bluesky: @talkpython.fm
Mastodon: @talkpython@fosstodon.org
X.com: @talkpython
Michael on Bluesky: @mkennedy.codes
Michael on Mastodon: @mkennedy@fosstodon.org
Michael on X.com: @mkennedy
Episode Transcript
Collapse transcript
00:00 If you built documentation in the Python ecosystem, chances are you've used Martin Donoff's work.
00:05 His material for MKDocs powers docs for FastAPI, uv, AWS, OpenAI, and tens of thousands of other projects.
00:13 When MKDocs 2.0 took a direction that would break material in 300 ecosystem plugins, Martin went back to the drawing board.
00:20 The result is Zenzicle, a new static site generator with a Rust core, differential builds in milliseconds instead of minutes, and a migration path designed to bring the whole community along.
00:31 This is Talk Python To Me, episode 542, recorded February 17th, 2026.
00:37 Welcome to Talk Python To Me, the number one Python podcast for developers and data scientists.
00:59 This is your host, Michael Kennedy.
01:01 I'm a PSF fellow who's been coding for over 25 years.
01:05 Let's connect on social media.
01:06 You'll find me and Talk Python on Mastodon, BlueSky, and X.
01:10 The social links are all in your show notes.
01:12 You can find over 10 years of past episodes at talkpython.fm.
01:16 And if you want to be part of the show, you can join our recording live streams.
01:19 That's right, we live stream the raw, uncut version of each episode on YouTube.
01:24 Just visit talkpython.fm/youtube to see the schedule of upcoming events.
01:29 Be sure to subscribe there and press the bell so you'll get notified anytime we're recording.
01:33 This episode is brought to you by Sentry.
01:34 You know Sentry for the error monitoring, but they now have logs too.
01:39 And with Sentry, your logs become way more usable, interleaving into your error reports to enhance debugging and understanding.
01:46 Get started today at talkpython.fm/sentry.
01:50 Martin, welcome to Talk Python To Me.
01:52 Great to have you here.
01:53 Thanks for having me.
01:53 I'm excited to talk about static sites and the next big platform for building them here in Python and beyond.
02:03 So really excited to talk about Zensical.
02:06 Am I saying that right?
02:08 Yeah, pretty much.
02:09 Zensical.
02:09 Zensical.
02:10 Okay.
02:11 Yeah.
02:11 Great.
02:11 Yeah.
02:12 I know MKDocs, the material for MKDocs has been really, really popular.
02:17 And you all have made a big splash announcing this new project.
02:22 So I'm really looking forward to diving into it.
02:24 Before we do, though, let's just get a little bit of background on you.
02:28 Who is Martin?
02:28 Of course.
02:29 So hi, my name is Martin Donut.
02:31 Most people probably know me as Squidfunk.
02:34 I've been an independent developer and consultant for the last 20 years now.
02:39 And I mostly write in TypeScript, Python, and lately a lot of Rust.
02:44 So I've become a huge fan of Rust, actually.
02:48 I'm kind of a free spirit.
02:50 So I love doing my own thing and building products from front to back, basically.
02:55 So doing the front end as well as the back end.
02:57 And for the past 15 years, I contributed a lot to open source.
03:01 As already mentioned, my most popular project so far is material for MKDocs.
03:06 And it's, well, millions of people looking, basically look at sites that are built with it every day.
03:14 Yeah.
03:14 Well, and Zensical, my latest project will hopefully go far beyond that.
03:18 So we're working very hard on it.
03:19 And this is why I'm here today.
03:20 So excited to talk about it.
03:22 Yeah, I am as well.
03:24 Well, and let's just start by admiring your website a little bit.
03:30 Thanks.
03:31 Brian and I spoke about this over on our Python Bytes podcast.
03:35 And we kind of just got distracted just staring at the website.
03:39 It's this beautiful flow of, I don't know, colors.
03:44 It looks a little bit like a black hole worm, a white wormhole sort of experience.
03:48 I don't know.
03:48 What was the inspiration there with this cool design?
03:51 Yeah, this is actually a strange attractor.
03:53 So this is something from physics.
03:55 I'm not very, very proficient in physics, but those strange attractors, I had a fascination for them for a very long time.
04:03 And they follow very simple rules.
04:06 So it's just three equations that define how their points move in three-dimensional space.
04:12 And yeah, but still with those simple rules, a very complex shape can emerge.
04:18 And this is for us actually symbolizes the process of evolving ideas through writing.
04:24 So if you have slightly different conditions from the start, it's still orbiting around the same shape, but it might look a little bit different.
04:33 And there's actually, I can share this now, there's actually a little Easter egg.
04:37 Nobody has found it so far.
04:38 So if you hover over the homepage on zensicle.org with the mouse in the left bottom corner, you can actually change the coefficients of the animation.
04:52 And if you do this, you can click on them and then you can use your cursor.
04:56 I'm changing beta.
04:58 We're running beta 0.22 right now.
05:01 Oh, it really does change it.
05:02 Yeah.
05:02 Oh my goodness.
05:03 Yeah.
05:04 So it takes a little time, but if you change the coefficients in a specific way, it might be completely chaotic and become unstable.
05:15 So this is what I really find fascinating about those strange attractors.
05:18 And it's also the inspiration for the logo.
05:20 So we're building on this image a lot.
05:23 Okay.
05:24 I thought it was just a cool design.
05:25 I didn't realize it had all this meaning and actual math and physics behind it.
05:30 That's super cool.
05:31 I love chaos theory and all of this, these fractal type of ideas here.
05:36 And yeah, it's super neat.
05:38 Okay.
05:39 So what is zensicle?
05:40 Why did you build it?
05:41 Why not just more material?
05:43 So there are a lot of questions in there, actually.
05:45 Maybe let me just start by shortly speaking about what it is.
05:49 So in very simple terms, it's a tool to build beautiful websites from a folder of text files.
05:55 So you just write a markdown and can generate a static site.
05:59 You don't need a database for it.
06:00 So to those that don't know what a static site is, you don't need a database or server.
06:05 It's just static HTML, which means you just pip install zensicle and you're ready to go within a few minutes.
06:11 And it's fully open source, MIT licensed.
06:14 And to maybe explain a little bit more about static sites.
06:17 So the big benefit of it, you can host it for free in many places.
06:21 For instance, on GitHub Pages or Cloudflare.
06:23 And they're secure and fast by default because there's only static file serving involved.
06:28 And zensicle.
06:28 So we try to make it pretty with a modern design, many built-in features and fun, according to the feedback of our users, which is kind of unusual for writing documentation.
06:37 So, yeah.
06:38 Yeah.
06:39 Very cool.
06:40 And if anyone's tried to manually create a static site, it quickly becomes a challenge if you're just writing.
06:50 Say, hey, it's only five HTML pages.
06:52 I can just write the HTML.
06:54 You know what I mean?
06:54 But, well, what if you want to have common navigation or you want to change the look and feel?
07:00 You know, oh, well, now I've got to go edit that in five places, right?
07:04 And so if even just beyond, basically beyond one page, having something that generates the static site is, it's super valuable, right?
07:13 Because it'll generate the wrapper navigation, the common CSS, the footer, all those kinds of things, right?
07:21 Yes.
07:21 So it depends on what you want to do.
07:23 So, of course, if you have a small site, like a personal website or so, you can just write basic HTML if you're proficient in it.
07:30 For instance, the users of Material, only 7% of them are front-end developers.
07:38 We will dive a little bit into how Zensical relates to Material later.
07:43 And what Zensical is being used for primarily is for documentation.
07:47 So it builds on the Docsys code philosophy, which means that you treat your documentation exactly like your source code.
07:55 So you primarily write documentation.
07:57 You don't want to fight front-end development problems.
08:01 You just want to keep the content, like get the content out.
08:04 And with this Docsys code, what the cool thing about it is, is you can use the same tools and processes and workflows like you use for code, like versioning and PRs to make changes.
08:16 And the adoption is growing really fast, actually, among companies in recent years as they're moving away from proprietary tools to open source solutions.
08:24 So Zensical is for you, or a static site generator in general is for you if you just want to get your writing out.
08:32 And of course, you can also customize it and make it pretty as you want, but you don't necessarily need to know HTML, CSS, and JavaScript.
08:41 And that's quite difficult.
08:43 And you talked about writing, and you even have your metaphor with strange attractors.
08:47 I personally find if I'm just in a clean space where it's really just about the ideas, I don't have to worry about the design.
08:56 It makes it so much easier to just focus on the actual writing.
09:00 You know, you're in a Markdown editor.
09:01 My favorite is TypeHora, but you can use whatever variety that you want, right?
09:06 And you're just there.
09:08 You're not worried even hardly about the formatting of the Markdown.
09:10 You're just writing.
09:11 And I find that very, a good creative space, I guess.
09:15 Yeah, that's the beauty of Markdown.
09:16 So you can just write, as you mentioned.
09:19 And how you, in the end, use it, you can still decide that afterwards.
09:24 So if you want to build a website, if you want to create a PDF of it, if you just want to use it for internal note-taking or so.
09:30 And this is the big benefit of Markdown.
09:34 It takes away a lot of the headache of having to remember a lot of Markup in order to get your ideas out of the door.
09:42 Can you actually put Markup in it if you need to?
09:44 You know, for example, maybe you need a particular image, two of them side by side that are links and you want them to open in a new tab if somebody clicks them.
09:54 Can you set it into basically an unsafe mode and let it do embedded Markup?
09:58 Yeah, that's a great question.
10:00 So, yes, it's possible.
10:01 You can just use HTML within Markdown.
10:04 We currently depend on Python Markdown, which we inherited from material for MKDocs.
10:09 We are gradually moving towards CommonMark, which, so just as a context, Python Markdown has some oddities when you use HTML within Markdown.
10:18 For instance, it won't replace relative URLs correctly.
10:23 This is like an annoying thing.
10:25 But once we move to CommonMark, we will also have like predefined components that you can use because you can't express everything like more complex things in plain Markdown.
10:36 So there are only things like you can make text bold, you can have lists, tables, etc. But if it's more complex, some, as you mentioned, aligning to images or having an image with a caption or so, you need a basically HTML.
10:51 And this is possible already, but we will make it much easier in the future.
10:54 The front end world already knows this.
10:56 So they use MDX.
10:57 They've been using MDX for quite a while, which is a dialect on top of Markdown, which adds more liberty with components and so on.
11:06 So you can create reusable components that you can use.
11:10 Yeah.
11:10 But, yeah.
11:11 So it's possible.
11:13 It's our users already also do it.
11:16 We also have it on some examples on the documentation, and we will make it much more powerful in the future.
11:21 Yeah.
11:21 Very nice.
11:22 I do think, you know, regular Markdown is just a few missing things.
11:27 I love the simplicity of it.
11:29 And, you know, hat tips, John Gruber for creating it.
11:32 But it's just like, I just need to maybe put a class here or just do a little, if I could just control this a little bit more, then you could basically escape HTML.
11:42 Obviously being careful to not just recreate HTML with square brackets instead of angle brackets, right?
11:47 Yeah, there's been a lot of work on Python Markdown.
11:49 So in Python Markdown, there are some extensions that allow you to add classes at least to block elements.
11:54 So in Markdown, you need to distinguish between inline and block elements.
12:00 Oh, no, it also works.
12:01 Sorry.
12:01 It also works on inline elements like links and so on.
12:03 But this is special syntax.
12:05 So Python Markdown is a dialect that is not standardized like CommonMark.
12:09 In CommonMark, this is not easily possible to add specific classes.
12:13 But with CommonMark, as I mentioned, you have MDX, which is a de facto standard.
12:18 I don't know if they've standardized it already.
12:20 That allows for much, much more.
12:22 So what is Zensical for?
12:24 Is this a documentation generating tool?
12:28 Is it a just open-ended static site generator?
12:31 What is possible and what is your goal or your target with this project?
12:38 Yeah.
12:38 So as I mentioned right now, we're focusing on documentation.
12:42 So because this is the thing we're coming from.
12:45 But we're building Zensical for much, much more.
12:47 So our stretch goal is to have like a fully-fledged knowledge management and documentation solution.
12:54 There are already a lot of companies that use it internally for knowledge management.
13:00 Basically, as an alternative to a ZAS-based solution like Confluence and Notion, we are aware that for this we need WYSIWYC.
13:08 So what you see is what you get.
13:09 A visual editor that is also usable by non-technicals.
13:12 And if you scroll, if you check out our roadmap and scroll down all the way, you will see it as a stretch goal, which is basically something we're working towards.
13:23 Because this would actually allow so much more people within organizations to use it.
13:30 And in general, Zensical, with Zensical, we focus on three key areas that make us different from other static site generators, which is, well, a modern design.
13:42 So, of course, some also have a modern design.
13:44 But within the Python ecosystem, some options might look a little bit dated or a little bit...
13:52 So we try to be a little bit more on the edge, actually.
13:55 And it should be flexible and it should be fast.
13:58 So those three things.
13:59 Because the design, actually, is the thing that people notice first.
14:04 So what we offer is a design that is customizable, brandable.
14:09 You have tons of options with which you can change how navigation is laid out, how you can also change colors, fonts, etc. And we have a lot of components that make it ready for technical writing.
14:22 As you mentioned, you just want to start writing.
14:24 So we have stuff like admonitions tabs.
14:28 And one very specific feature that we have is code annotations that we inherited from Material for MKDocs, which is quite unique among static site generators, which allows you to put a little bubble onto any line of code.
14:43 You have to visit our documentation.
14:44 This is our...
14:45 You're currently browsing our...
14:47 The other side.
14:49 All right, all right.
14:49 Hold on.
14:50 I got it.
14:50 Keep going.
14:51 I'll get to say.
14:52 Right, right.
14:52 No worries.
14:53 Yeah.
14:54 And there you have to search for code annotations.
14:56 Yeah.
14:57 So code annotations, which allow you to create a bubble in any line of code.
15:03 And if you click that bubble, there opens a tooltip.
15:06 And within this tooltip, you can use any rich content.
15:08 So you can have lists, any formatted markdown tables, diagrams, basically anything you can use anyway within markdown.
15:18 And this is a very popular feature in Material.
15:20 And so, of course, we brought it over.
15:23 So users can still use it.
15:25 So the second thing I talked about is it should be flexible.
15:29 So what makes Ansical different is we have a modular architecture or say we're working towards a modular architecture.
15:34 We're still in alpha.
15:36 So we're close to finishing the module system.
15:40 And in Zendigl, it's modules all the way down, which means all core functionality is implemented as modules, which is different from other solutions where the plugin system sometimes is more or less an afterthought.
15:53 So there's a plugin system added with specific hooks, extension points where you can hook into.
15:57 And this might seem sufficient at first, but in the end, so for us, for instance, MKDocs in the end was a little bit limiting.
16:07 And this allows you to basically swap, extend, replace all modules.
16:11 You can use our modules.
16:12 You can write your own, pull in third-party modules.
16:14 And as I mentioned, Rust.
16:16 So don't worry.
16:17 You don't need to learn Rust.
16:18 You will also be able to write modules in Python because we are super happy users of Pyro 3, which is absolutely amazing library.
16:24 And Pyro 3 has really become a super important foundation of Python these days.
16:30 It's almost like the C bindings for CPython.
16:35 Exactly.
16:35 So, yeah.
16:36 So with Pyro 3, it allows us to have a Rust runtime.
16:40 So all of the orchestration and how, in which order, so in which order things are run, threading, caching, parallelization, etc. All is happening in Rust.
16:48 And we will provide Python binding so that you still can use Python to write modules.
16:53 And they're still running fast.
16:55 This portion of Talk Python To Me is brought to you by Sentry.
16:59 You know Sentry for their great error monitoring.
17:02 But let's talk about logs.
17:03 Logs are messy.
17:05 Trying to grep through them and line them up with traces and dashboards just to understand one issue isn't easy.
17:11 Did you know that Sentry has logs too?
17:14 And your logs just became way more usable.
17:16 Sentry's logs are trace-connected and structured, so you can follow the request flow and filter by what matters.
17:22 And because Sentry surfaces the context right where you're debugging, the trace, relevant logs, the error, and even the session replay all land in one timeline.
17:31 No timestamp matching.
17:33 No tool hopping.
17:34 From front-end to mobile to back-end, whatever you're debugging, Sentry gives you the context you need so you can fix the problem and move on.
17:40 More than 4.5 million developers use Sentry, including teams at Anthropic and Disney+.
17:45 Get started with Sentry logs and error monitoring today at talkpython.fm/sentry.
17:51 Be sure to use our code, talkpython26.
17:54 The link is in your podcast player's show notes.
17:56 Thank you to Sentry for supporting the show.
17:59 Which brings me to the last point where we're different.
18:02 We have a very heavy focus on performance, so our goal is to let you start with one page because, of course, all documentation sites, all projects start small, and let you scale that to something like 100,000 pages.
18:15 How we do it is through differential builds.
18:18 We have created our own runtime, which is called ZRX, and differential builds mean that we are only rebuilding what changed.
18:26 So, for instance, if you only change the page title, only that page and all instances where the page title is used are being rebuilt.
18:32 And this means that changes are visible in milliseconds and not minutes.
18:35 Yeah, that's super cool.
18:37 And so I'm presuming the build system itself is Rust-based, right?
18:41 Yeah, exactly.
18:42 It's 100% Rust, yeah.
18:43 Yeah, yeah.
18:44 Coming from a Python background, what was that experience like building that?
18:49 Yeah, so that's kind of a tricky question because I'm not really coming from a long history.
18:57 So I don't have a long Python background.
19:00 I wrote many in TypeScript, and I only started 2021 writing Python.
19:07 So this is actually the history, how materials started, and how all of this unfolded.
19:14 But I've written in several languages.
19:18 So I also have written in C, Erlang, Ruby, Python, TypeScript.
19:22 Rust was still extremely hard to learn.
19:24 So I basically banged my head against the keyboard for a month, wasn't making no progress at all because, yeah, you know, fighting with the borrow checker.
19:31 So, and once you get past that, and then, of course, lifetimes and higher rank trade bounds and some other features, I'm now some kind of like 3,000 hours, 4,000 hours in, something like that.
19:46 It gets really good.
19:47 So I think Rust is seriously one of the best languages ever made because it allows you to express ideas extremely clearly, with extreme clarity.
19:56 And this is due to the very good type system, of course, and you get bare metal performance.
20:04 So I find it kind of insane having a language like Rust because it's so easy to write once you're used to it.
20:12 You will be very productive and still have bare metal performance.
20:17 It's completely insane.
20:18 Yeah, that's wild.
20:19 But it's got a little bit of a learning curve compared to like Python or TypeScript or something like that.
20:24 Yeah, so I had, I think, 18 years of experience with many languages.
20:30 As I mentioned, I also did a lot of C and I still found it very hard to learn.
20:36 But it's worth it.
20:38 It's worth it.
20:39 And my recommendation probably would be to learn it on something that you really care about so that you want to build because otherwise you will probably lose the drive since you're running against those walls.
20:55 Maybe for you, it's, or for somebody else, it's much easier to learn.
20:59 So maybe it's just, I'm a bad example that I needed so long.
21:04 I don't know.
21:04 But because after that month, it wasn't that I was completely up to speed.
21:09 So it was just, I was making very, very tiny progress, at least progress, because for a month I wasn't making progress at all.
21:16 The next show that I'm doing after this one, which actually is, in real, on clock time, wall time, it's happening in like two hours or less from now is with Samuel Colvin from Pydantic talking about Monty,
21:30 a Python runtime.
21:32 He's, he and his team are rewriting in Rust, specifically targeting AI.
21:37 So the Rust theme will continue.
21:40 It's definitely a very, it caught me a little bit off guard, like how much people love it, but it's also, it's also, you know, it makes perfect sense that we want this nice modern language for writing lower level things, even if it plugs into Python, right?
21:54 Yeah.
21:54 So the fun thing is I also talked to Samuel a long time ago and he was the one recommending to me to write it in Rust.
22:02 it's one of, it's one of the reasons I, yeah, definitely I looked into it and it made, it made a lot of sense also during the time, the progress we're making and so on and the walls we're hitting that to reconsider learning Rust
22:16 best investment.
22:17 Yeah.
22:18 Yeah.
22:18 Amazing.
22:19 Amazing.
22:19 So I wanted, I want to dig into your component structure and some of those things, but maybe before we do, let's, let's talk about the origins a little bit.
22:28 So what, let's talk about how you went from material for MK docs.
22:33 Why, why even change?
22:34 Why not just more material?
22:36 Yeah.
22:37 So this is, this is a great question and this is a little bit of a story.
22:41 So there are several stories in there actually, because it's, so it's 10 years.
22:45 I try to go make it as compact as possible while keeping the most important things.
22:51 So to those who don't know, material for MK docs is a very popular documentation framework.
22:55 It's used by tens of thousands of projects.
22:57 There are prominent users like AWS, Microsoft, open AI, also large open source projects use it.
23:04 Like for instance, FastAPI, UVK native, and it's built on top of MK docs as the name says, which is one of the most, which became one of the most popular static site generators.
23:13 And it also eventually became my job.
23:15 So I could make it my job.
23:17 I could work in open source and earn a living somehow.
23:21 I'm getting there how that worked.
23:24 And, but at some point we needed a new foundation and we've like kind of outgrown MK docs because it was not evolving at the pace that we needed.
23:32 So we began exploring alternatives and yeah, so there's a lot of lessons learned in, in materials.
23:38 So let me show us short, shortly, maybe talk about how it started because like it started as a side project in 2015.
23:45 Like many things start because I wanted to release a, actually a C library, zero, a zero copy protocol buffers library I wrote called proto bluff.
23:53 But then I realized that it needed more than a read me.
23:57 So I looked at the existing static site generators which were Hugo, Jekyll, Sphinx, MK docs, something like that.
24:03 And they all look a little bit dated.
24:05 I'm not a designer but I wanted something more modern and Google was pushing material design quite, quite hard for app development at the time.
24:13 So, and I've also seen it being used in the web.
24:16 So I thought, well, maybe combine this.
24:19 I quickly settled on MK docs, was easy to use, simple templating, enough for a side project basically.
24:24 So it was a side project.
24:26 Did what most devs do, I checked the license but didn't do any further due diligence.
24:30 So even put MK docs in the name to show the connection.
24:33 So, which is common for themes.
24:35 And that actually turned out to be one of the biggest decisions I made in my career since I was basing my complete work on somebody, something I don't control and it shaped the next 10 years of all of the work I was doing and is actually the reason
24:49 why Zensical exists today.
24:52 I see.
24:52 So, after I started developing it, I, like nine months later, released the first version and send a good users, a lot of feature requests and, you know, it was a side project.
25:04 So I was doing client work at the time.
25:06 As I mentioned, I've been a, like a consultant and developer freelancer for 20 years and I only had Sundays to work on it.
25:16 So, which at first was efficient, but the more popular it got, the more maintenance that came.
25:21 So it kind of crept into my mornings and evenings and I was doing triage, like answering questions and trying to fix bugs before I went to the client and it was getting harder and harder to justify in front of my partner actually because I was doing it in my spare time
25:36 and so I did what eventually all projects that start as side projects and where you don't have the full time to work on it, how they, yeah, so what basically happens is you start turning down feature requests
25:50 and many open source projects don't cross this line and for me it was a first.
25:55 yeah, and also additionally, so I mentioned before that I started writing Python in 2021.
26:01 At the time I was focusing, so I only had Sundays to work on it, I didn't know Python so I said that, okay, I will focus on the templating stuff, I will do the HTML, CSS, JavaScript, all of this, make it beautiful and try to solve as much, as many problems as possible
26:16 in the front end but I won't start learning Python because it wasn't a language that I was using at that time and I couldn't make up the time for it so that's where I drew the line.
26:28 Yeah, yeah, and then I tried to...
26:30 It's probably going to be a fad that Python thing anyway.
26:32 I don't think so but...
26:35 Well, at the time, in 2015 it wasn't clear that it was going to be as popular as it was as it is now, right?
26:41 It's really...
26:42 It started to become popular then but it's really taken over the world and...
26:47 Absolutely.
26:48 For a lot of reasons, so...
26:49 Of course, yeah, I think one of the main reasons is because it's very popular in the ML community and all of the LLM AI work that's happening and so on made it extremely popular so...
27:02 And I also think that Rust is doing a very good job on keeping it that way because finally you have a very easy way to offload work to native code which is much easier than fiddling with
27:16 C and C++ and void pointers and whatever so as I mentioned, Pyro 3 is just an absolutely amazing library.
27:22 It's so easy to write Rust code.
27:24 Yeah, I think you're right.
27:25 I think...
27:26 Rust has really provided an important escape hatch for...
27:29 I wrote it this way.
27:30 It's not fast enough.
27:31 Like, well, this part, we're going to make it as fast as it can be, basically.
27:34 Yeah.
27:35 Yeah.
27:36 So...
27:37 Sorry, I interrupted you.
27:38 Keep going.
27:38 Oh, no worries.
27:39 No worries.
27:39 Yeah, no, no.
27:40 Yeah, so as I mentioned, I tried to keep it basically afloat for the first four years and at the time I didn't see the potential at all.
27:48 It was just a theme, not a kind of product or so, but yet I felt responsible and kept on maintaining it and my developer friends didn't understand why I was doing that.
27:57 So, but for me, it was like, you know, it was kind of cool because I had a growing project.
28:03 I had no immediate plans.
28:04 I don't know.
28:05 Let's see where that, where I can, where I can take it and, yeah, so, and with this steady and slowly growth over years, then companies and organizations started using it.
28:14 so they were basing their public facing documentation on me, like the guy that maybe works on this project on a Sunday and yet I felt responsible enough to trying to fix the bugs reported as quickly as possible.
28:29 Yeah, and, yeah, then in 2020 actually came the turning point.
28:33 So when I was working on version five of it, I shared my progress publicly as I did before and somebody mentioned the donate button.
28:38 So, and I think the wording was something like so that I can order pizza to survive the long Sunday coding sessions.
28:48 But I heard from another developer who did this on his project, well, successful project for five years, a donate button and he made $90.
28:57 So I immediately said that's not going to work.
29:00 But I said, let's try an Amazon wish list.
29:04 You know, I just put some stuff on there and maybe if somebody thinks my work is useful, then he can order me, like, make me a present and something send me a present.
29:13 So, yeah, and I basically received everything on that wish list.
29:17 It was completely insane.
29:18 So there were two consecutive days that felt like Christmas.
29:20 I even put like, so, I put some, you know, books and, but then also single malt.
29:27 I love Scottish, Scottish single malt.
29:30 It was a whiskey that cost $120 and I received that that as well.
29:34 So it was like, what's happening?
29:37 And that led me to start thinking actually about demographics.
29:40 So in, that I needed to better understand the audience of material for MKDocs.
29:45 And I did a poll and the results were absolutely eye-opening.
29:49 I mentioned before, 7% only of front, of users are front-end developers, which means, and material is a front-end heavy project.
29:57 So I kind of had an edge there in the Python space because, yeah, you know, it's based on Python.
30:03 So front-end developers that write in JavaScript, they rather go for something like DocuSaurus or React-based or whatever.
30:10 And technical writers were quite happy with the project.
30:13 I didn't know even technical writers existed.
30:15 So I had no clue that this job, that this is a job because I thought at the time, and it's in hindsight completely naive, of course, I thought that as a developer, you need to write the documentation, you know.
30:26 And so I learned about that and accidentally built a product for technical writers.
30:32 And by the way, when I say product, I mean something that is not necessarily something you pay for, but something that doesn't feel engineered.
30:39 So something that is like polished and designed and that you actually want to use.
30:46 And yeah, so I had a product that has like product market fit and but at the time I didn't earn any money off it.
30:54 So at the same time I read about SponsorWare and this, like, I'm not sure if you heard of it, but it's like a new model of monetization for open source.
31:03 At the time it was quite new so that you can get paid for your work.
31:07 So you can, so some developers for instance, they sell course material or access to gated content or code or nothing at all.
31:14 So if you have a popular project you can just try to raise sponsorships from, and some companies are very generous when it comes to open source.
31:22 And what we did with Material was we gave away early access to the latest features to the sponsors and each feature was tied to a funding goal and when that funding goal was met it became free for everyone.
31:34 So it was like kind of a funded feature development in multiple stages.
31:40 And that's what I thought of.
31:42 Sorry?
31:43 Yeah.
31:43 That's super clever.
31:44 I really love the idea of providing something for the sponsors but still not turning it into well, here's a paid version of our product and here's the open source version.
31:54 but there's always this tension of how do you reward the people who support you without undermining the open source project and that's a clever angle.
32:04 Yeah.
32:04 So that's extremely challenging.
32:06 So as I'm telling this so this is what I came up with and I thought maybe it could work something like that and again my developer friends they said will never work nobody will pay for open source you're insane.
32:19 Spoiler alert it did work and in the end we made 200k a year of it and could build a team and everything so I know in Silicon Valley terms this is probably minimum wage but in Europe it's quite an amount with which you can work very well.
32:34 And yeah so I started this program in 2020 and it grew steadily and it finally allowed me to work on features outside of the Sunday so invest more hours into it and finally learned Python in 2021 wrote my first plugin
32:48 and started hacking the MKDocs features that well that got turned down that we upstreamed but where the maintainer said ah it's maybe not a good fit or we don't have the time for it and yeah in total
33:03 I wrote 12 MKDocs plugins so it started as a theme but it turned into a popular sorry into a powerful docs framework in the end and this worked quite well for several years until it didn't anymore and that's the reason why Zensical
33:17 then came into being.
33:18 So the way it didn't work is that just where you want to take it started to diverge from MKDocs or you you couldn't get your changes upstreamed or committed back?
33:30 So the thing was that MKDocs was not evolving as we needed it in the it's so historically MKDocs had a sequence of single maintainers and as far as I know all of them worked on it in their spare time because they had
33:45 regular jobs and material was evolving quickly because you know we had funding we could we could invest much more time in it we could then of course then an open source project that is only maintained in the spare time and so it was changing too slowly
34:00 so we started a lot of discussions on necessary API changes because for many users material for MKDocs was MKDocs so we were kind of like the storefront where most of the issues and like bug reports and feature requests came in
34:14 because many people are using material for MKDocs and with this MKDocs basically and the main challenges that we faced were performance and plugin orchestration I mentioned I wrote 12 plugins and
34:29 it's very hard to make them cooperate and if you look at any popular MKDocs plugins issue tracker you will find issues that go something like well this plugin is incompatible with this plugin well if I change the
34:43 order of the plugins in the configuration this and this happens and both of those problems were brought to us again and again by the users with which we talked and so it was coming up a lot then suddenly
34:58 after nine years the original maintainer returned to MKDocs and we were super optimistic because the project was maintained again he also started a sponsorship program we upstreamed some of our funding immediately and supported his work so before MKDocs had no
35:13 way to sponsor them and the moment this went live we immediately supported it and some PRs were finally merged and issues were closed but yeah then the works went silent and it started working
35:28 in basically in the quiet and three months later we were invited to a video call so we as maintainers from so I as a maintainer for material for MKDocs and some other key ecosystem maintainers
35:42 and we learned that MKDocs that the plans for MKDocs 2.0 were completely different from what existed at so what currently exists MKDocs 1.x which primarily means no plugin
35:57 API and customization via templating alone so we already knew this is not enough because that's what we've done the first four years where as I mentioned I was only doing templating and some things with templates for instance
36:11 having a tag support where you need to pull in different tags from different pages and then render them on another page or so you need synchronization efforts and you can't do this with templating by the way all of this information is public
36:26 so you can read it on the mkdocs issue tracker so I'm not telling anything secret or so it's a completely different direction than they were dismissed
36:40 so mkdocs 2.0 as it looks right now is incompatible with material for mkdocs 300 plugins and ecosystem will become useless and tens of thousands of projects will be affected and for us so we had absolutely no choice than to
36:55 start building something so to start make something of this because at the time we had already 50,000 projects 50,000 public projects depending on us we talking to enterprise users and we knew that this number
37:10 is much much higher so for instance one of our professional users they already also sponsored material they have two and a half thousand projects internally so only one company and they have a dedicated team of
37:25 individuals that maintain their customizations on top of material for mkdocs for all of the teams inside the company it's a very big company so that's incredible what you could infer from the I
37:39 could believe it I couldn't believe it at all so absolutely insane yeah so as I mentioned we had no choice so what we did was we immediately went back to the drawing board with the learnings from the almost 10 years
37:54 that passed since I started material we built a lot of prototypes in typescript and Python iterated on them we did a lot of conceptual work things what could actually be done with a radically different architecture
38:09 because writing 12 plugins I know the ins and outs of mkdocs I had to do a lot of hacks to make the block plugin of material work with the way navigation works in mkdocs and the number
38:24 one complaint as I mentioned was mkdocs is slow and it doesn't scale so fixing a typo you're doing a full rebuild and this can take minutes so our design work centered exactly around this problem and after
38:39 a short while so we knew exactly what mkdocs should look like and we didn't want to let our users down and we so in essence we had two options we know what it should look like we could fork it or we could start from scratch and
38:53 forking is not really possible the way because of the way how python dependencies work so all of the plugins have a dependency on mkdocs and this means that we would also need to fork all of so without doing black magic with
39:08 imports which might not be the best idea so we would also need to fork all plugins or all plugins would need to switch to the fork so this would be like moving an entire city at once and it's frankly
39:23 impossible so and if we would fork it we wouldn't be able to realize our learnings that we gained in the groundwork that we did so we had to start from scratch actually right plus you'd have to convince the entire community to at
39:37 least create a parallel package because when you pip install that other plugin it's going to say hey PyPI I need mkdocs and now you'd it
39:52 would be a big battle wouldn't it just technically with or you'd have to move the community which is a very challenging thing to do yeah and so for us the most sensible thing was to just you know we just start from scratch we make it
40:07 as compatible as possible it became quite clear very quickly that we need to optimize for compatibility because if you create something that is not compatible work
40:22 to get over to something else you won't get a lot of adoption all you gotta do is think about that 2500 project team like okay how do I keep them working with this right yes yes yeah so what we
40:37 then did is we had an idea how it should look then we started with Rust because it was recommended to us so it was very hard at first and it in total of
40:52 this but it was not only writing code it was also exactly knowing where we want to go because we're starting fresh so we better be sure that we are going into a direction where we actually want to go for the next 10 to 20 to 30 years
41:07 depends so we are really in for this for the long game so the 10 years that I've been doing this I see that this is only the start and we wrote a lot of things from scratch so the runtime as I mentioned it's like the
41:21 heart of Zensical it already has something like 15,000 lines of code a tiny HTTP middleware framework for file serving because we also want to make the file server extensible and don't want users to force them into async
41:36 Rust and also don't have a dependency on Tokyo in JavaScript for instance there's Lerna and it has 800
41:51 dependencies so when you install it what you pull down is just insane so we worked a lot on the processes as well that we can make releases very easy and we have a good way of working basically and we are very careful about
42:06 our choice of dependencies so if it's not something that you can write quite quickly actually and we rather own in order to make changes ourselves we rather write it from scratch I
42:21 think that's a very healthy philosophy and also I think this agentic AI world that we're in these days if you just need one or two functions and you used to think well maybe I'll lean on this and in your case a crate or maybe a PyPI package
42:36 or something but if it's just I started using pip dash audit for a lot of my projects and I would say for my
42:51 bigger projects every two weeks I get at least one CVE vulnerability notification for something I'm using but here's the thing it's in a situation of that probably a piece of code or functionality of that
43:05 package that I don't even use or care about so it doesn't really apply to me but then I've got all these here's an issue that I'm
43:20 going to be fine you know what I mean I think things are swinging back a little bit from let's just pull in everything because it's going to help us to well maybe not everything yeah and also you
43:35 can't just change things easily and you depend on other APIs so for instance one of the reasons why we choose to build a lot of things from scratch is that we want to control the public API so
43:49 the worst thing for us would probably just be to export a third party API that we're using as part of our public interface because it's rust so it would mean that if those this public API would change the entire ecosystem would break so we're very
44:04 careful !
44:05 to what APIs we expose and rather wrap it in order to be safe so we can replace things replaceable so maybe you have the philosophy of it might be okay to use this crate but we don't
44:19 exchange its types as the public as part of our API or something along those lines we don't expose it so we in some instance the wrappers that I've wrote are identical to the types
44:34 that we use from using our own types or just wrapping them because in Rust the nice benefit is you have zero cost abstraction so all the code is monomorphized in line so you don't pay for wrapping code that's
44:49 the absolute crazy thing so it's you can finally create a really clean architecture without runtime penalties if you do it right oh that's wild yeah yeah yeah very interesting so you can see I have this huge list of
45:07 I'd like to go back to this component wrong search there you have components that was in the other part let's just talk through some of these things here so you've got like admonitions
45:22 buttons code blocks let's talk through some of the building blocks I guess that you think are interesting here yeah so I think most of the so if you if you're new to technical writing most of the stuff shouldn't be quite new so like
45:36 admonitions code blocks stuff like that you've probably seen or data tables diagrams are just mermaid diagrams as they are as you can use them on github one of the so like the flagship features in material or
45:51 and now Zensical as I mentioned like code annotations which is a part of code blocks otherwise we also have an icon and emoji integration so you can use one of I think we have something like over 10,000 icons now
46:06 with a quite simple syntax that's not standard markdown that's the problem so that's like a Python markdown extension and we're working on moving this over to common mark and finding a way to migrate this over because
46:20 right now Zensical uses Python markdown for compatibility with materials for mkdocs which means that for markdown
46:32 we the same parsing engine to have this strong compatibility right we can even read mkdocs YML configuration so you can build an mkdocs project with Zensical as it stands the thing that we currently don't support in its entirety is the
47:16 plugins from the ecosystem we already support some plugins for instance the mkdocs strings plugin the author is also part of the Zensical team now with mkdocs strings being the second biggest project in the mkdocs space so we are
47:31 happy to have Tim on board and several other plugins but as I mentioned so Zensical uses modules so what we will do in the end is we will still always be able to read mkdocs configuration and map the plugin
47:46 configurations to equivalent Zensical modules so the logic will be completely rewritten but you will be able to migrate your project with a command that's our goal because there has so much work been going on
48:01 into projects built with material and mkdocs so we need to make it easy for users and organizations to switch and this is the main part we're working on in 2026 I think this is
48:15 it's critical right if your absolute best users you know like that big company but many others of course they're not going to rewrite everything well maybe they will but many of them won't rewrite everything they'll just use an old version and grin and bear it
48:30 as long as they have to you know what I started working on search before we started working on Zensical yeah I noticed how nice the search was when I was playing with it we're in so is zensicle.org itself built in Zensical yeah of course and it's actually built with an mkdocs YML
50:07 because we're dog fooding so you can also build it with mkdocs with material for mkdocs the project layout is exactly the same yeah you know I find that there's just a bunch of static sites that seem to have I don't know
50:21 what's going on with them but their search is really bad either they've just integrated some kind of Google thing where it's a site colon and they use your URL and the search which is a real bad experience or you go search and it spins and it spins and then
50:36 eventually it pulls up so it looks like you are pre-computing these types of things or something with your search engine or you've got some cool data structure to make that fast right well it's not one cool data structure that would be great because then everybody could just use it but
50:51 several months of work went into the search of course so it's a project of its own as I !
51:03 it's !
51:03 also completely modular and the reason why most of the search engines that are open source so like the libraries that you can use not services you have to pay for that they don't provide results that are really relevant or
51:18 is that they use BM25 which is the standard bag of words ranking algorithm for information retrieval and this doesn't nicely pair with autocomplete so what you get is you start typing
51:34 documents the balancing will be off because the relevance is computed based on the occurrence of a word in the entire corpus so you add a new document those weights change again
51:49 the search that we have we of course as a baseline also have a BM25 implementation but the implementation you're seeing is a tie breaking implementation which provides much better accuracy and you can configure it so tie breaking
52:03 means okay we first look into the title of the document then and see if we have matches then how many matches then where they are then we look into the navigate in the path and then in the body of the document and so on all of this is configurable
52:18 and this is also why we believe that this go alone will also be a very interesting project for other for for instance side generators to integrate and you asked about pre-computing so no this is a search from the
52:32 documents we build a search index which is a strapped down version of the HTML that is rendered when you load the page it's one JSON that we ship to the client and for most pages actually this JSON is below one megabyte you can
52:47 GC it so compress it then it's something like 200k and you have extremely fast search on the client with no cost and so we believe that for 90 95 maybe 99 percent of
53:02 documentation sites or sites in general this client site search is basically the way to go because it's fast and it doesn't require you to pay for anything and there are several SaaS based services that can be extremely expensive
53:16 when you do the math so yeah you only need to use a server basically when you when the index becomes too big to ship to the client and we're also working on that by the way okay that's really cool you could shard the index or
53:31 something like that right I suppose like you could say we're going to have 20 26 index bits and if only if the word starts with an A do you
53:48 some other interesting solutions like page find is a pretty interesting library it does a completely different approach but it's not as snappy as the search that we ship to the client I use page find
54:03 for my personal website which is a static site yeah it's also a great great solution but some things you won't be able to implement in page find properly because so it's with software it's trade-offs all the way so well I'm
54:18 already thinking like I better pay attention to disco when it comes out so maybe adopt it for some stuff beautiful okay we got a couple interesting questions sort of following up from the component side of things jamsack says do you
54:32 foresee community led templates or themes for zensical I know you have like two themes that I see something along those lines a couple of themes that you can choose now but what is the theme story I guess I want to ask you more broadly yeah so
54:47 absolutely so right now we have only this one theme we have this variant setting where you can choose like the classic variant which is when you move over from material for mkdocs it looks exactly the same this is also why we
55:02 needed to keep the HTML as it is also with the once we move to the component system we will make it possible to one use components
55:17 within markdown and two also create a template engine that is based on components this will allow us much much faster rendering because for instance if you render the header for a site it's a lot of HTML because there's
55:32 the search box in it and some other stuff but only the title changes so we will also make the rendering differential as part of the build that's the plan and with this we will also make it open to theme developers of course so there will be the
55:46 packaging for instance compilation of ZAS styles or TypeScript or so will be part of Zensico so you don't need to precompile the theme like we need to do for the last 10 years for material
56:00 so it will have a proper asset pipeline it will have a proper process to install themes all of this is planned but right now we focus on feature parity so in order to make it possible for more users to migrate right now that's really interesting
56:15 that you would deliver the theme as basically its original source not its rendered compiled or transpiled version right to keep it yes part of the build step right yes exactly because
56:32 the sidebar disappears too early for my taste and this is not so for this you have to go through the compilation step again and basically fork the theme
56:47 and recompile it we want to make this configurable so that you can use yeah so you know configure the theme and build it and it just works so this like you know it just works that's like the
57:02 we're working towards make it as simple as possible yeah yeah very cool let's maybe get short on time here maybe wrap up our our chat talking about two things the future where are you going you talked about compatibility being a big
57:17 part of things going forward in 2026 but also sustainability right you had all these great supporters for material for mkdocs which you must have just been absolutely !
57:28 thrilled to realize how successful that was right going from I'll put up a wish list and then actually people love this I can put all my energy into it I know how great of a feeling that is right that's completely insane and when I
57:43 started it I would never believe that this would be my job at some point really I feel the same way about the podcast and it's just I'm so grateful for it's amazing but then with this transition to Zensical how does that change anything
57:57 or what's the story how do you bring that support over to Zensical as we don't have a lot of time I try to explain it as compact as possible so we are saying goodbye to this pay for extra features so
58:12 in material you needed to be a sponsor in order to get the latest features earlier what we will do is everything is open source from the start so for users it's completely free and we are shifting our model from the sponsorships
58:27 to something we call Zensical Spark because what we discovered talking a lot to our professional users is that the more we know about the problem space and the better we understand the problem space and the more we can collaborate with them the
58:41 better degrees of freedom we can provide so we don't intend to just chip feature feature feature but we intend to create degrees of freedom so that you can adapt Zensical to the processes within your organization how
58:56 they work to the workflows etc which are all different which is all very diverse basically so Spark is a space where you as a company can get a seat and together with us shape Zensical as part of high level discussions where we
59:11 explore the problem space we create proposals so on the website you have clicked on the Spark section there's this Zaps in progress we call them Zaps Zensical advancement proposals it's on the left side we write very elaborate detailed
59:26 proposals on specific topics that we intend to work on and then with the feedback that we get iterate on them and create
59:39 a solution that is opinionated but that is as unopinionated as possible and the third thing that you get besides the opportunity to
59:53 have high level discussions with us is professional support we've been asked for quite a lot by companies in Spark you can basically get
01:00:08 direct access to the team and also we have those open video calls where we share our progress and where you can get a window of support and we talk about any problem that is keeping you up at night
01:00:23 basically and stuff like migrations or how do you do this and this in Zensical and yeah it's been a blast so we're really happy that the organizations are enrolling into this
01:00:38 that might translate quite well to other projects because you get a huge competitive advantage you know exactly what to build yeah you're on you're talking to the actual users they're saying this is the thing that really is hard for us or you just get maybe they don't say it but
01:00:53 you see it right exactly yes yes and talking to the users is the best !
01:01:08 and this new project I'm very excited to see it coming along and it looks like it's going to be great maybe a final call to action for people like can they go ahead and start using Zensical if
01:01:38 it has a lot of built in functionality already you get all of these components that we talked about free search that you don't have to host a very modern static site that is great on mobile so just give it a try and we have a newsletter
01:01:52 where we once a month share the latest updates and that might also be worth checking !
01:02:08 it great to see as many users as possible and shape the future of Zensical together with all you to our sponsors be sure to check out what they're offering it really helps support the show
01:03:08 best of all there's no subscription in sight browse the catalog at talkpython.fm and if you're not already subscribed to the show on your favorite podcast player what are you waiting for just search for python in your podcast player we should be right at the top if you enjoy that
01:03:22 geeky rap song you can download the full track the link is actually in your podcast blur show notes this is your host Michael Kennedy thank you so much for listening I really appreciate it I'll see you next time
01:03:59 is the norm


