Python apps with LLM building blocks

Episode Deep Dive Links Transcript

It's BLACK FRIDAY at Talk Python
Get the Black Friday deals

In this episode, I’m talking with Vincent Warmerdam about treating LLMs as just another API in your Python app, with clear boundaries, small focused endpoints, and good monitoring. We’ll dig into patterns for wrapping these calls, caching and inspecting responses, and deciding where an LLM API actually earns its keep in your architecture.

Play on YouTube

Watch the live stream version

Episode Deep Dive

Guest Introduction and Background

Vincent Warmerdam is a data scientist, educator, and developer advocate currently working at Marimo, a modern notebook environment for Python. He's best known for his educational work through CalmCode.io, where he provides free Python tutorials, and his YouTube channels covering both Python programming and ergonomic keyboards (which has grown to over 5,000 subscribers). Vincent previously worked at Explosion (makers of spaCy) and has extensive experience in machine learning and natural language processing. He's a strong advocate for pragmatic, boring technology that just works, and has a passion for building tools that make developers more productive. His episode on spaCy from the previous year was the number one most downloaded Talk Python episode of that year.

What to Know If You're New to Python

If you're new to Python and want to get the most out of this episode's content on integrating LLMs into Python applications, here are the key concepts to understand:

Decorators and functions: LLM integration often uses Python decorators to wrap functions with caching, validation, or API calls. Understanding how functions work and how decorators modify their behavior is essential.
Type hints and Pydantic: Modern LLM work relies heavily on type annotations to define structured outputs. Familiarity with Python's type hint system (strings, integers, lists, optional types) will help you understand how to constrain LLM responses.
Async programming basics: While not deeply covered, many LLM APIs benefit from async/await patterns for better performance when making multiple API calls.
Caching concepts: Understanding what caching is and why it matters (avoiding redundant expensive operations) is central to building cost-effective LLM applications.

Key Points and Takeaways

1. Treat LLMs as Unreliable APIs That Need Defensive Programming

LLMs represent a fundamentally different kind of building block compared to traditional functions in your code. Unlike normal functions where you put something in and predictably get the same thing out, LLMs are stochastic - you can put the same input in twice and get different outputs. This means you need to think defensively, putting boundaries and validation around LLM calls. The core principle is to acknowledge that LLMs are weird building blocks that require special handling, including caching, validation, and careful monitoring of what goes in and what comes out. Vincent emphasized this is not just about using the right tools, but about developing the right mindset for working with these probabilistic systems.

Links and Tools:

LLM Building Blocks for Python Course
Pydantic for data validation
Instructor library for retry mechanics with validation

2. DiskCache Is Essential for LLM Development

DiskCache is a SQLite-based caching library that should be in every LLM developer's toolkit. It allows you to cache LLM responses to disk so they persist across program restarts, preventing you from making the same expensive API call twice. Vincent and Michael both praised it as "unbelievably good" - it works like a simple dictionary or decorator but stores everything in SQLite, making inspection easy. You can set time-to-live values, add custom keys based on model/prompt/settings tuples, and even store multiple outputs for the same input by adding an integer to the key. This not only saves money but dramatically speeds up development iteration since you're not waiting for the same API calls repeatedly.

Links and Tools:

DiskCache
SQLite-backed persistent caching
Works across application restarts
Decorator pattern for easy integration

3. Simon Willison's LLM Library Provides Boring, Reliable Abstractions

Vincent called Simon Willison's LLM library "the most boring" as a compliment - it does a few things in a very predictable, unsurprising way without walling you into complex abstractions. Originally designed as a command-line utility, it has a clean Python API that makes it trivial to swap between different LLM providers (OpenAI, Anthropic, Mistral, etc.) through a plugin ecosystem. This means individual providers can maintain their own plugins rather than one maintainer drowning in compatibility work. The library handles the basics extremely well and is particularly good for rapid prototyping and experimentation, making it Vincent's go-to tool when he wants to quickly test something with LLMs.

Links and Tools:

LLM by Simon Willison
Plugin ecosystem for multiple providers
Command-line and Python API
SQLite logging of prompts and responses

4. Structured Output With Pydantic Transforms LLMs Into Programmable Components

One of the biggest challenges with early LLMs was that text went in and unpredictable text came out, making it hard to build reliable software. Modern LLMs are now trained on structured output tasks, allowing you to define a Pydantic model and receive guaranteed JSON that matches your schema. You can specify you want a list of strings, a classification from specific values using literals, or complex nested objects. Vincent noted that while Pydantic can do more than the JSON schema spec supports, and simpler structures work more reliably than deeply nested ones, this capability fundamentally changes LLMs from chat interfaces into programmable components you can integrate into your software architecture.

Links and Tools:

Pydantic
JSON schema generation
Type-safe LLM outputs
Validation and parsing

5. The Instructor Library Uses Validation Errors as Prompts for Self-Correction

Instructor takes Pydantic validation a step further with a clever trick: if an LLM returns something that doesn't validate against your Pydantic model, Instructor takes the validation error message, combines it with what the LLM returned, and asks the LLM to try again with the hint. This retry mechanism can handle cases where your Pydantic validation is more sophisticated than the JSON schema can express. Vincent noted this was more critical in earlier LLM eras when structured output wasn't as reliable, but it's still valuable for lightweight models or complex validation rules. The library uses the tenacity package under the hood to manage retries with limits.

Links and Tools:

Instructor
Validation retry mechanism
Works with Pydantic models
Uses tenacity for retry logic

6. LLMs Often Lose to Traditional ML When Properly Evaluated

Vincent shared stories from PyData conferences where data scientists, given budget to experiment with LLMs, ended up putting scikit-learn or spaCy into production instead. The mandate from above to "do AI" gave them permission to build proper evaluation frameworks and compare approaches rigorously. When they tested LLMs against lightweight traditional models with enough training data (which they needed anyway for evaluation), the traditional models often performed as well or better while being faster, cheaper, and deterministic. This isn't a failure of LLMs but a reminder that evaluation methodology matters more than hype, and sometimes the boring solution is the right one.

Links and Tools:

scikit-learn
spaCy
Importance of evaluation frameworks
Cost and performance trade-offs

7. Open Router Provides One API Key for All LLM Models

Open Router is a service that routes to any LLM model you want with a single API key, making it trivial to experiment with the latest models as soon as they're released. If you've set up caching and evaluation functions properly, switching between models is just changing a string. They aggregate multiple GPU providers competing for the lowest prices and add about a 5% fee. This is particularly valuable for rapid experimentation - Vincent uses it to quickly test whether a 7B, 14B, or larger model hits the right quality/cost sweet spot for a specific task. The service has direct integration with tools like Cline for easy access to hundreds of models.

Links and Tools:

Open Router
Single API for multiple LLM providers
Competitive GPU provider pricing
Easy model comparison and experimentation

8. Run Your Own Models Locally With Ollama or LM Studio

For smaller models or privacy-sensitive applications, running models locally on your own hardware is increasingly practical. Ollama is a command-line utility that makes it easy to download and run models, providing an OpenAI-compatible API endpoint. LM Studio offers a more visual UI for discovering, configuring, and running models, then exposes them via API. Michael runs the GPT OSS 20 billion parameter model on his M2 Pro Mac mini with 32GB RAM as his default LLM. Running local models eliminates API costs for experimentation and gives you control, though quality varies significantly across open-source models. Both tools support running models that can then be accessed through the LLM library or any OpenAI-compatible client.

Links and Tools:

Ollama
LM Studio
Local model hosting
OpenAI-compatible APIs

9. SmartFunc Pattern Uses Docstrings as Prompts

Vincent demonstrated his SmartFunc library which turns Python functions into LLM calls using decorators. You define a function with typed parameters, add a decorator specifying which LLM backend to use, and write your prompt in the docstring. The function parameters can be referenced in the docstring (which can be a Jinja template), and return types can enforce structured output. This pattern makes it very quick to prototype LLM functionality - you can create a summarization function, a classification function, or any other LLM-powered operation with minimal boilerplate. While Vincent doesn't necessarily recommend others use SmartFunc itself, it demonstrates how you can build your own syntactic sugar on top of foundational libraries like LLM.

Links and Tools:

SmartFunc
Decorator-based LLM integration
Docstring as prompt
Type-driven structured output

10. Pydantic AI Uses Types to Handle Conversational State

Pydantic AI and similar higher-order frameworks use Python's type system in sophisticated ways to manage conversational flows. For example, in a pizza ordering bot, you might define a type that's either a Pizza object (with size, toppings, etc.) or a string representing a follow-up question. The LLM decides whether it has enough information to return the structured Pizza object or needs to ask the user for more details by returning a string. This lets you handle complex form-filling conversations where different fields might be optional or conditional, all driven by type definitions rather than manual state management logic. Vincent emphasized that while these frameworks are powerful, keeping the base layer boring and focusing on well-defined types can get you very far.

Links and Tools:

Pydantic AI
Type-driven conversation management
Union types for flow control
Created by Samuel Colvin and team

11. Cline Code Assistant Emphasizes Plan Mode vs Execute Mode

Cline (formerly Claude Dev) is an open-source VS Code extension that distinguishes between planning what to do and actually executing changes, with a UI that shows your context window usage and costs in real-time. Unlike some AI assistants, Cline makes you very aware of the cumulative cost as you work (the Fibonacci sequence effect - each interaction costs what you've spent plus more). It's model-agnostic, allowing you to bring your own API keys, and has direct integration with Open Router. The emphasis on plan mode before execution, combined with cost visibility, encourages more thoughtful use of AI assistance rather than just mashing the "continue" button. They recently introduced a CLI as well for command-line usage.

Links and Tools:

Cline
Open-source code assistant
Plan vs execute workflow
Real-time cost tracking
Model-agnostic with API key support

12. Evaluation and Caching Enable Rapid Model Comparison

The combination of proper caching and evaluation functions creates a powerful workflow for LLM development. Once you've built a cached function and an evaluation approach (which might be manual inspection, automated metrics, or statistical testing), you can iterate extremely quickly. You can run the same prompt through different models, different temperature settings, or different prompting strategies, and because of caching, you only pay for each unique combination once. Vincent emphasized being "really strict about evaluations" as the methodology that lets you discover that sometimes scikit-learn beats an LLM, or a 7B model is good enough, saving significant costs.

Links and Tools:

Importance of evaluation frameworks
Statistical testing approaches
A/B testing for LLM applications
Cost optimization through comparison

13. Marimo Notebooks Enable LLM Prototyping With UI

Vincent now works at Marimo, a modern notebook environment where notebooks are pure Python files under the hood, enabling proper version control, pytest integration, and dependency management via uv. For LLM work specifically, Marimo excels at blending Python code with interactive UI elements, making it easy to build quick prototypes with text boxes, sliders, and other controls. The reactivity model (using abstract syntax trees to understand cell dependencies) prevents the hidden state problems that plague Jupyter. Vincent uses Marimo for all his rapid LLM prototyping because he can quickly wire up an LLM call to UI elements and iterate. It also supports running notebooks as command-line scripts and has native support for unit tests within notebooks.

Links and Tools:

Marimo
Reactive notebook environment
Pure Python file format
Integrated UI components for LLM experimentation
Git-friendly and testable

Interesting Quotes and Stories

"I just found the LLM library by Simon Willison by far to be the most boring. And I mean that really in a good way, just unsurprising, only does a few things. The few things that it does is just in a very predictable way." -- Vincent Warmerdam

"If you haven't used DiskCache before, it definitely feels like one of these libraries that because I have it in my back pocket, it just feels like I can tackle more problems." -- Vincent Warmerdam

"Your previous experience as a proper engineer will still help you write good LLM software. However, from a similar perspective, I also think that we do have this generation of like data scientists... thinking analytically being quite critical of the output of an algorithm. That's also like a good bone to have in this day and age." -- Vincent Warmerdam

"I've heard a story a bunch of times now that because of the hype around LLMs and AI, after it was implemented, after they did all the benchmarks, it turns out that AI is the reason that scikit-learn is now in production in a bunch of places." -- Vincent Warmerdam

"If you can just focus on the right types and make sure that all that stuff is kind of sane, then you also keep the extractions at bay which is I think also convenient especially early on." -- Vincent Warmerdam

"Definitely expose yourself to LLMs. And if that inspires you, that's great. But also try to not overly rely on it either... I'm building flashcard apps for myself so I'm still kind of in the loop." -- Vincent Warmerdam

"If you have better tools you should also have better ideas and if that's not the case then something is wrong because that's then you get into the self-learned helplessness territory." -- Vincent Warmerdam

"I would be really disappointed if I'm the only person making packages that I've never made before that are definitely trying to reach new heights... Try to inspire yourself a bit more and do more inspirational stuff so more cool things happen on my timeline." -- Vincent Warmerdam

Key Definitions and Terms

Structured Output: The ability of modern LLMs to return responses in a guaranteed format (like JSON matching a specific schema) rather than free-form text, enabling reliable integration into software systems.

DiskCache: A Python library that provides a persistent, SQLite-backed caching system that survives program restarts, particularly valuable for expensive operations like LLM API calls.

Plugin Ecosystem: An architectural pattern where a core library provides extension points for third parties to add support for different services (like different LLM providers) without the core maintainer handling all integrations.

Pydantic: A Python library for data validation and settings management using Python type annotations, commonly used to define structured schemas for LLM outputs.

Validation Retry: A pattern where failed validation errors are fed back to an LLM as hints to try generating a valid response again, implemented by libraries like Instructor.

Evaluation Framework: A systematic approach to testing and comparing LLM outputs, essential for determining whether an LLM approach is better than traditional methods or whether one model performs better than another.

Open Router: A service that provides a single API endpoint and key to access dozens of different LLM models from various providers, simplifying model experimentation.

Ollama: A command-line tool for running large language models locally on your own hardware with an OpenAI-compatible API.

Marimo: A modern Python notebook environment where notebooks are pure Python files with reactive cell dependencies, designed to avoid hidden state problems.

Learning Resources

Before diving into the learning resources, these courses and materials will help you build on the concepts discussed in this episode. Whether you're looking to integrate LLMs into your applications or strengthen your Python foundations, these resources provide hands-on, practical knowledge.

LLM Building Blocks for Python: Vincent Warmerdam's course that teaches you to integrate large language models into your Python applications with practical, code-first techniques for real-world LLM development including structured outputs, caching, async pipelines, and production-ready patterns.

Just Enough Python for Data Scientists: This course provides essential Python and software engineering practices for data scientists, covering clean functions, importable packages, git workflows, debugging, and reproducible environments - all foundations that make LLM work more reliable.

Python for Absolute Beginners: If you're new to Python and want to understand the fundamentals before diving into LLM integration, this comprehensive beginner course covers all the core concepts at a pace designed for those just starting their programming journey.

Rock Solid Python with Python Typing: Since type hints are central to modern LLM work with Pydantic and structured outputs, this course teaches you the ins-and-outs of Python's type system and how frameworks leverage it.

Data Science Jumpstart with 10 Projects: Matt Harrison's hands-on course teaches practical data science skills through 10 diverse projects, building the analytical mindset Vincent emphasized as valuable for critical evaluation of LLM outputs versus traditional approaches.

Overall Takeaway

This episode challenges the prevailing narrative that LLMs should replace traditional programming approaches. Instead, Vincent Warmerdam presents a more nuanced view: LLMs are powerful but unpredictable tools that require defensive programming, rigorous evaluation, and the wisdom to know when a boring, traditional solution is actually better. The key insight is that LLM integration is as much about mindset and methodology as it is about tools - you need caching to avoid waste, evaluation frameworks to measure success, type systems to constrain outputs, and the discipline to keep learning rather than falling into "learned helplessness."

The episode is packed with practical tools (DiskCache, Simon Willison's LLM library, Pydantic, Instructor, Open Router) but the deeper message is about maintaining your capabilities as a developer even as automation improves. Vincent's call to action is inspiring: don't just use AI tools to make what you already make faster; use them to build things you couldn't build before. Be deliberate about maintaining your own knowledge through flashcards and continuous learning. Evaluate rigorously enough that you sometimes choose scikit-learn over an LLM. And most importantly, guard against the WALL-E future where convenience leads to helplessness - instead, channel the mandate for AI from above into permission to do better engineering with proper testing and evaluation. The future belongs not to those who can push the "continue" button most effectively, but to those who can thoughtfully integrate these new capabilities while maintaining their fundamental problem-solving skills.

Links from the show

Vincent on X: @fishnets88
Vincent on Mastodon: @koaning

LLM Building Blocks for Python Co-urse: training.talkpython.fm
Top Talk Python Episodes of 2024: talkpython.fm
LLM Usage - Datasette: llm.datasette.io
DiskCache - Disk Backed Cache (Documentation): grantjenks.com
smartfunc - Turn docstrings into LLM-functions: github.com
Ollama: ollama.com
LM Studio - Local AI: lmstudio.ai
marimo - A Next-Generation Python Notebook: marimo.io
Pydantic: pydantic.dev
Instructor - Complex Schemas & Validation (Python): python.useinstructor.com
Diving into PydanticAI with marimo: youtube.com
Cline - AI Coding Agent: cline.bot
OpenRouter - The Unified Interface For LLMs: openrouter.ai
Leafcloud: leaf.cloud
OpenAI looks for its "Google Chrome" moment with new Atlas web browser: arstechnica.com

Watch this episode on YouTube: youtube.com
Episode #528 deep-dive: talkpython.fm/528
Episode transcripts: talkpython.fm

Theme Song: Developer Rap
🥁 Served in a Flask 🎸: talkpython.fm/flasksong

---== Don't be a stranger ==---
YouTube: youtube.com/@talkpython

Bluesky: @talkpython.fm
Mastodon: @talkpython@fosstodon.org
X.com: @talkpython

Michael on Bluesky: @mkennedy.codes
Michael on Mastodon: @mkennedy@fosstodon.org
Michael on X.com: @mkennedy

Episode Transcript

Collapse transcript

WebVTT format On GitHub

00:00 In this episode, I'm talking with Vincent Wormerdam about treating LLMs as just another API in your

00:05 Python app with clear boundaries, small focused endpoints, and good monitoring. We'll dig into

00:11 patterns for wrapping these calls, caching and inspecting responses, and deciding where an LLM

00:17 API actually earns its keep in your architecture. This is Talk Python To Me, episode 528,

00:23 recorded October 23rd, 2025.

00:44 Welcome to Talk Python To Me, the number one podcast for Python developers and data scientists.

00:48 This is your host, Michael Kennedy. I'm a PSF fellow who's been coding for over 25 years.

00:54 Let's connect on social media.

00:55 You'll find me and Talk Python on Mastodon, Bluesky, and X.

00:59 The social links are all in the show notes.

01:02 You can find over 10 years of past episodes at talkpython.fm.

01:06 And if you want to be part of the show, you can join our recording live streams.

01:10 That's right.

01:10 We live stream the raw, uncut version of each episode on YouTube.

01:15 Just visit talkpython.fm/youtube to see the schedule of upcoming events.

01:20 And be sure to subscribe and press the bell so you'll get notified anytime we're recording.

01:24 This episode is brought to you by Sentry.

01:27 Don't let those errors go unnoticed.

01:28 Use Sentry like we do here at Talk Python.

01:30 Sign up at talkpython.fm/sentry.

01:34 And it's brought to you by NordStellar.

01:36 NordStellar is a threat exposure management platform from the Nord security family,

01:41 the folks behind NordVPN that combines dark web intelligence,

01:45 session hijacking prevention, brand and domain abuse detection,

01:50 and external attack surface management.

01:52 Learn more and get started keeping your team safe at talkpython.fm/nordstellar. Hey, welcome to Talk Python To Me. Hi. Welcome back. Pleasure

02:01 to be here. Yeah, it's been like, this is our second Talk Python interview, I think. We've

02:06 interacted before also on socials and all that, but like, yeah, second time. Yay. Happy to be here.

02:10 Yeah. Yeah. It's very exciting to have you back. I, you know, I'm trying to remember, I did an

02:15 blog post on the most popular episodes

02:20 of last year.

02:22 I think, if I do recall correctly...

02:25 Yeah, the number one last year was our

02:28 interaction, I think, on Spade.

02:29 That's not terrible, is it?

02:31 Yeah, it was a bragging right for a day.

02:34 Yeah, it's pretty cool. So your episode and double bragging

02:38 rights as your episode was released in, I don't know when that was,

02:42 something like October or something, almost exactly a year ago to the day. And still it was the number one most downloaded for the whole year. So

02:50 fantastic. And I think that might be an indication of where we're going to another very exciting

02:56 topic, you know? Yeah. Like ML and AI only went up, right? So, so.

03:03 You know what? I would love to get your thoughts on this. I, I'm seeing out on the internet and I

03:08 want to put this up as kind of a, to counter this, this thought or whatever. But, but I'm seeing a

03:13 lot of people get really frustrated that AI is a thing. You know, OpenAI just released their Atlas

03:21 browser, which I'm not super excited about, but it's like their version of what an AI browser

03:26 wrapping Chromium looks like, you know what I mean? And it's fine. But the comments in Ars Technica,

03:32 which are usually pretty balanced, are just like people losing it over the fact that it has AI.

03:37 Are people tired of this stuff? Or why do you think that's the reaction, I guess, is really what I

03:42 want to ask you i mean i guess i have like a couple of perspectives but like i do like i think i could

03:47 say like we are all tired that like every tech product out there is trying its best to find a

03:52 way to put ai in the product like oh any any gmail any calendar app let's put ai in there and it

03:58 gets a bit tiring like i wouldn't be too surprised if like fire hoses and refrigerators also come with

04:02 ai features these days like it feels a bit nuts i know it's like what what is my refrigerating ai for

04:09 Yeah, or like soccer ball of whatever object, like it feels like people are overdoing the AI thing. So okay, that can be like legitimate source of frustration. But then it also becomes a little bit personal, because I can also imagine if you put like, your heart and soul into like being a good engineer, and like you want to take the internet serious, like you consider it something is somewhat sacred, because it helped you so much in your career, then I definitely can also imagine like, Oh, my God, please don't turn that into a slop. Like I built a career on that. So that might be the source of all these things. I guess like my main way of dealing with it,

04:39 is more along the lines of like, okay, like, this genie is not going to go back into the

04:43 ball in a way. But if we're able to do more things with these LLMs, if they're able to

04:47 automate more boilerplate and that sort of thing, then it seems that then I can try and

04:52 get better at natural intelligence as well, instead of this artificial intelligence. So

04:56 like, okay, I can vibe code all these things now. So the bottleneck is just do I have good

05:01 ideas? And maybe I should invest in good ideas. And oh, maybe I can actually learn about

05:06 JavaScript, but then I should not focus on the syntax, but it shouldn't focus maybe on

05:09 the browser and like how does CSS work and oh I should actually maybe do a little bit of stuff

05:15 with flashcards just so I know the syntax just enough so I can review okay like I try to be very

05:19 conscious about it that way that's kind of my approach more and I it's easy to get really lazy

05:24 right and just push the button and say do the next thing and not use it as an opportunity like oh it

05:28 did something I didn't know let me have a conversation and try to understand the uh I

05:33 didn't know I think the key word here is you want to be deliberate about it like I think if you can

05:37 sort of say like okay i'm deliberate about the way i use this stuff and i'm also learning as i'm doing

05:41 this like one thing i really like to do is i just give give like some sort of bonkers front-end task

05:46 to the llm that i have no idea how i would solve it and then study the output like that's the thing

05:50 i actually do once in a while um but yeah i mean i do get it that like people have mixed emotions

05:56 about it and that's totally cool and fine it's just that for me in my situation this is how i

06:01 deal with it yeah i think that's fair i think also i i definitely have long since been tired of like

06:07 every email app having to rewrite this with AI and it usually destroys the email.

06:12 You know, like removes all the formatting.

06:14 And then you just, you know, there's always the cartoon of like,

06:17 I wrote an email as bullet points.

06:19 I asked AI to expand it.

06:20 I sent it to you.

06:21 You're like, what a long email.

06:23 I'm gonna ask AI to summarize this.

06:25 We should have just sent the bullet points.

06:27 - Yeah, I mean, I do fear the amount of slop though.

06:30 Like I do, it almost feels like every good tech YouTuber,

06:33 there aren't a lot of good tech YouTubers anymore they're all doing AI stuff instead of doing like actual really great programming. There's still a

06:40 few. Anthony writes code is like a really good one still. Like he's awesome. Definitely follow

06:45 him if you want to learn Python. He does cool stuff. But yeah, the main thing, I guess from my

06:51 like selfish perspective, YouTube used to be better because nowadays it's all about the AI and you've

06:55 always got this thumbnail of a guy pulling his hair out like, oh my God, this changes everything.

06:59 Yeah. I would love to see a video that's maybe not that.

07:03 Have you heard that there's an AI bubble that's going to burst?

07:05 um yeah well we'll see exactly how far i think so actually i mean we can speculate like i i can

07:12 argue that it's been over invested i can also argue there's still so much potential the one

07:16 thing i will say though um we have clod kinds of tools and like we're all looking forward to like

07:21 the vibe coding oh it's never been easier and that sort of thing but i would have expected an explosion

07:24 of awesome apps to be going along with it and it doesn't necessarily feel like we have much better

07:29 software even though we supposedly have way better tools so something about that is making me a little

07:34 a bit suspicious, but it might also just be that the apps that are being built that I'm not the

07:38 target audience for, because I can imagine that. Because let's say you have something awesome for

07:42 dentists, they can still be an awesome crud app that you could build with Claude, but I'm not a

07:46 dentist. So, you know, wow, there's a lot of new cool dentistry management apps. You're right.

07:53 If a dentist can outprogram, right? Like I do believe in the story that, oh, as a dentist,

07:58 you might be best equipped to write an app for dentists, right? And if they're now more empowered

08:02 to maybe do some stuff with code.

08:04 I mean, there's a story there that every single niche profession

08:07 will have someone who can do enough clod to make the app for that profession.

08:12 Yeah.

08:13 Well, time will tell.

08:13 It used to be that you have to have a programming skill

08:16 and a little bit of, let's say, dentistry experience to build the right sort of specialized vertical.

08:21 And now I think it maybe is reversed.

08:23 You need a lot of dentistry and a little bit of coding skill to go along with an AI.

08:28 Yeah.

08:29 The character sheet used to be 80 points here, 20 points there.

08:32 and now it's 80 points there and 20 points here.

08:34 Yes, exactly.

08:35 That's exactly what I was thinking.

08:36 Like, I think maybe that's actually shifted until you said that.

08:39 I've never really thought about it, but it just may be.

08:41 I do want to point out for people listening, we're not going to really talk too much

08:47 about like using agentic tools to write code for us.

08:51 Like we have been speculating about this theoretical dentist.

08:55 But what instead we're going to do is we're going to talk to you, Vincent,

08:59 about how can I use an LLM like a library or an API to add functionality, features, behaviors to an existing data science

09:08 tool or to an existing web app or whatever it is we're building, right?

09:12 Yes.

09:13 So also people might probably know it, but I made a course for Talk Python, of course,

09:18 and we're going to talk about some of those details.

09:21 But the idea for that course was not necessarily like, how can I use LLMs to build me software?

09:26 It's more, oh, how can I build software that also uses LLMs under the hood?

09:29 Like if I have an app that makes a summary, how can I make sure the summary is reliable?

09:33 And basically get the ball rolling on those building blocks.

09:36 That's what that course is about.

09:37 That's also probably going to be the main bones of this topic as opposed to a dentist

09:41 Bob and his ambitions to make a new tech startup.

09:44 Exactly.

09:45 We're going to grow new teeth if we just could get the software right.

09:47 Now, yeah.

09:48 So how do you do that?

09:49 And we're going to talk about that before.

09:51 I just want to give you a chance before we really dive too far into the details of how

09:56 we make that work is just, you know, give people a sense of what you've been up to lately.

10:00 You've done a lot of cool stuff with CalmCode.io.

10:04 I can see your YouTube game is growing strong here.

10:08 And you've been doing a lot of data science at Marimo.

10:11 Yeah, what's...

10:12 Right.

10:12 Yeah, so we haven't spoken in a year, so maybe we should catch up.

10:16 So long story short, CalmCode is still a project that I maintain.

10:19 It's basically 99% free educational content on Python.

10:24 that thing is just going to maintain itself.

10:27 Super happy to keep maintaining it.

10:29 The main thing I'm doing with that nowadays is just every two months I switch cloud providers

10:32 just because I can, and then I can sort of see what it's like on the other side.

10:36 So that's the thing that I do.

10:37 From Calm Cult, though, I figured I might also start a YouTube channel,

10:40 and that ended up being a YouTube channel where I still talk about Python stuff,

10:43 but I mainly talk about these fancy ergonomic keyboards

10:46 because I had a couple of RSI issues.

10:47 In a year's time, I went from 100 subscribers to 5,000 something,

10:51 so this thing kind of took off.

10:53 That's awesome.

10:53 I actually have sponsors now.

10:55 So I actually have a couple of companies in Asia sending me their custom boards for me to review.

11:00 So that's been a really fun journey.

11:02 But since last time we spoke, I also switched employers.

11:04 So people might have heard of Jupyter before.

11:07 That's an environment in Python where you can do interactive things.

11:10 And I work now for a company that makes Maremo, which is an alternative, a very likable one.

11:15 It does work completely differently.

11:17 One of the main things that attracts people to it is the fact that these kinds of notebooks are Python files under the hood.

11:23 They're also a little bit more interactive.

11:25 You can do more rapid prototyping with UI.

11:27 You can blend Python with it very nicely.

11:30 So I will say like all of my rapid prototyping, especially with LLMs, I do that in Marimo

11:34 nowadays just so you can blend Python with UIs very easily.

11:38 And there's demos on the site that people can go ahead and check out.

11:41 But that's also the second YouTube channel I made this year.

11:45 I do a lot of stuff for Marimo.

11:47 I'm a little bit more on the growth side of things than the hardcore engineering side of things.

11:51 I still make a lot of plugins for Marimo, of course, but that's a little bit more of what I do nowadays.

11:56 Making sure that people learn about Marimo.

12:00 You can do things in a notebook now that you couldn't think of before.

12:03 I write my command line apps in a notebook nowadays because it's actually not just because I can, but because it's convenient too.

12:09 So we'll talk a little bit about that later when we talk about LLMs, but that's effectively the short story of what I've been doing last year.

12:14 Nice. All right. I am a fan of Marimo.

12:18 Happy to hear it.

12:19 Yeah, yeah, yeah. I did a data science course this year, just enough Python and software engineering for data scientists. Like, you know, what could you bring from the software engineering side to like sort of make your, your data engineering, data science stuff a little more reliable. But I was right on the fence of should I, should I use this? Because it's, I think it's clearly better. But at the same time, I also want to use what people are using. So I didn't quite go for it. But I just, the UI is fantastic.

12:47 the reproducibility, reliability of it, where it uses the abstract syntax tree

12:52 or concrete, whatever, to understand relationships across cells

12:56 so they don't get out of sync, potentially.

12:57 Yeah, I think it's really nice.

12:59 I had Akshay on the show, who you work with also,

13:02 to talk about it.

13:03 The one thing in Jupyter that's, I mean, let me start by saying,

13:08 Jupyter is still a really cool project.

13:10 Like if I think back of the last decade, like all the good that that project

13:13 has brought to my career, not just directly as a user,

13:16 but also indirectly, all the algorithms that got invented

13:19 simply because we had a good enough interactive environment suddenly, which we never

13:23 had before, it's done wonders.

13:25 So we should definitely be thankful.

13:27 Yeah, I'm not bashing on it either.

13:29 Yeah, but I do always want to make sure that I give credit where credit is due, because

13:33 the project had a lot of impact.

13:35 But there is this really annoying thing with Jupyter, though.

13:37 Besides, you can be a bit grumpy about the file format, sure, but the one thing that's

13:41 very annoying is you can write down X is equal to 10, and then

13:45 have all sorts of cells below it that depend on X, You could then delete the cell and the notebook will not complain to you about it.

13:51 Even though if anyone else tries to rerun the notebook, X is gone, it won't run, and your notebook is broken and you can't share the thing anymore.

13:58 And nowadays, fast forward like many years later, and we've got stuff like uv now.

14:02 So we can add metadata to a Python file to add dependencies.

14:06 And oh, wait, because Marimo is a Python file, we can add dependencies to the notebook that makes it super reproducible.

14:11 There's just all this stuff that you can rethink now simply because we have a good notebook format that is still a Python file.

14:18 That's really the killer feature here.

14:20 We can talk more about it if you like, but I can talk for ages about this topic, just warning you.

14:25 One final thing, maybe also for the software engineering side of things, because you didn't mention it.

14:29 Just to give an example of something we added recently.

14:31 If you have a cell in Marimo and there's a function in there that starts with test underscore, we will automatically assume it's a pytest.

14:39 So you can actually add unit tests to your notebook as well.

14:41 And then if you say python notebook.py, then pytest will just run all the tests for you,

14:47 even though you can also run the tests in line in your CI CD as well.

14:49 So there's all sorts of these Python specific things that we can add,

14:53 again, because it's just a Python file.

14:54 Yeah, it's sort of the second mover advantage or nth mover advantage

14:58 where n is greater than one, where you see, okay, that was awesome.

15:01 Maybe this was a little bit of a rough edge.

15:02 And what would we do to work around that or smooth it out, right?

15:06 Well, and also we're also lucky that we're born in the age of uv.

15:09 I got to say like a lot of like quality of life improvements to a notebook.

15:15 A lot of that is also due to the fact that uv is around.

15:17 That's definitely a huge help as well.

15:20 This portion of Talk Python To Me is brought to you by Sentry's Seer.

15:24 I'm excited to share a new tool from Sentry, Seer.

15:27 Seer is your AI driven pair programmer that finds, diagnoses and fixes code issues in your Python app faster than ever.

15:35 If you're already using Sentry, you are already using Sentry, right?

15:39 Then using Seer is as simple as enabling a feature on your already existing project.

15:45 Seer taps into all the rich context Sentry has about an error.

15:48 Stack traces, logs, commit history, performance data, essentially everything.

15:53 Then it employs its agentic AI code capabilities to figure out what is wrong.

15:58 It's like having a senior developer pair programming with you on bug fixes.

16:02 SEER then proposes a solution, generating a patch for your code and even opening a GitHub pull request.

16:08 This leaves the developers in charge because it's up to them to actually approve the PR.

16:13 But it can reduce the time from error detection to fix dramatically.

16:18 Developers who've tried it found it can fix errors in one shot that would have taken them hours to debug.

16:24 SEER boasts a 94.5% accuracy in identifying root causes.

16:29 SEER also prioritizes actionable issues with an actionability score, so you know what to fix first.

16:36 This transforms Sentry errors into actionable fixes, turning a pile of error reports into an ordered to-do list.

16:44 If you could use an always-on-call AI agent to help track down errors and propose fixes before you even have time to read the notification, check out Sentry's SEER.

16:54 Just visit talkpython.fm/SEER, S-E-E-R.

16:58 The link is in your podcast player show notes.

17:01 Be sure to use our code, Talk Python.

17:03 One word, all caps.

17:05 Thank you, Dysentry, for supporting Talk Pythonemy.

17:08 I saw on, gosh, where was it?

17:10 Speaking of Reddit, I saw on slash R, learn Python or slash R Python.

17:16 One of the slash R's with a Python substring.

17:19 Someone asks, what Python package manager would you use now?

17:23 And it's just like, how many times can you say UV?

17:27 The feedback comments.

17:29 I mean, it was someone new who wasn't unsure, right?

17:31 They've seen all the different ones.

17:33 And yeah.

17:34 I mean, the coolest comparison I've seen, I think it was a tweet someone posted.

17:38 But like, definitely, like, suppose you're in the data field right now.

17:41 Like, what are the tools 10 years ago?

17:42 What are the tools now?

17:43 It definitely does feel like, okay, before we had pip, now we have uv.

17:47 Before we had Pandas, now we have Polars.

17:50 Before we had Matplotlib, now we have Altair.

17:52 And before we had Jupyter, now we've got Marino.

17:53 You can kind of see a generational shift, not just on the notebook side of things,

17:57 but like on the package manager, on the data frame library,

18:00 we're all learning from the previous generation and it kind of feels like.

18:03 Yeah, absolutely.

18:04 It's an amazing time.

18:06 Every single day in Python is just more exciting than the day before.

18:09 Yeah.

18:10 Although I should also mention, especially with the Polars

18:12 thing, there's a fair bit of rust too.

18:14 That also makes a difference.

18:16 Yeah, yeah, of course.

18:17 So before Flyby, I mean, you talked a bit about your YouTube

18:22 channel, you review the economic keyboards.

18:25 you have for those people who are just listening, not watching your background is super nice.

18:30 Your bona fides are solid here, but you've got a bunch of different ones. And I personally also,

18:35 I don't know, you might even just kick me out of the club if I should do that.

18:38 That's the Microsoft Sculpt, right?

18:40 Microsoft Sculpt ergonomic. And I like it so much because I can take this little bottom thing off

18:44 and it's like razor thin if I could rotate and reverse and jam it in my backpack and take it

18:49 with me. But I've also had RSI issues for 25 years or something. And it was really bad. And I switched

18:55 I used to type them on like laptops and stuff.

18:56 And I switched to just something like this and I could type 10 hours a day and

19:01 there's no problem. Whereas before, if I, if I were to force to type full on, on a laptop,

19:05 I would probably be like unable to work in a week. If not a week.

19:09 So we had an offsite in San Francisco. So I was there like two weeks ago,

19:13 a lot of fun, but like, I'm not going to bring like,

19:16 airport security is going to look at this and wonder what,

19:19 like what kind of weird alien device this is. Right. So I figured, okay,

19:22 I'll leave those at home and just bring my normal laptop. And after a week,

19:25 I'm feeling it in my wrist again.

19:27 It's not good.

19:27 Yeah, it's something that takes serious.

19:29 Although I will also say like because the kid was sick and the wife was sick

19:34 so I wasn't able to do a whole lot of exercise at home.

19:37 Also do exercise.

19:37 That also totally helps.

19:39 If I had to like, sure, buy an ergonomic keyboard if you like.

19:42 Programmatic keyboards, they're great.

19:44 But if you're going to do that as an excuse not to do exercise, you're doing it wrong.

19:47 That's the one thing I do want to add to that.

19:48 There's lots of stretches you can do.

19:50 Taking breaks matters.

19:51 Exercise matters.

19:53 But I think those keyboards are an essential feature.

19:56 They do totally help.

19:58 And also, sometimes it's not so much a healing measure.

20:01 It's more of a preventative measure.

20:02 Yes, 100%.

20:04 And I would like to put this message out just to everyone listening who is young, absolutely indestructible.

20:11 Please.

20:12 I know they're a bit of a pain in the butt with the weird curviness.

20:15 It is so worth it to just say, I've never had any RSI issues.

20:21 I just use this weird keyboard and people make fun of me,

20:23 but I don't care.

20:23 You know, that is a better thing than like, I'm working on it.

20:27 Yeah.

20:27 Oh, my.

20:28 It hurts.

20:28 You know?

20:29 Well, another thing is also, especially now we've got stuff like Claude, right?

20:32 I just want-- if I can point out like one thing on that front.

20:36 So these ergonomic keyboards, you can often program them as you see fit.

20:39 So you can say, like, if I hit this key, it's K.

20:42 But if I hold it down, it becomes command, or whatever you like.

20:45 But it also means that you can map macros or shortcuts.

20:47 So whenever I hit this button, an app called MacWhisperer boots

20:51 That's going to record my voice.

20:51 I love Mac Whisper.

20:53 Yeah.

20:53 And then there's alternative variants for it.

20:55 I'm sure you've got stuff for Linux as well.

20:56 But the main thing it does is just really good speech to text.

20:59 And then whenever I'm done talking, it's just immediately going to paste whatever I'm in.

21:04 So, oh, that's actually very nice, because now the keyboard shortcut is just me holding my thumb down

21:08 instead of some sort of weird claw I got to eject to my keyboard.

21:15 And suddenly, it doesn't necessarily become a convenience thing.

21:17 It also becomes kind of a power user thing.

21:20 Like you can really customize your computer experience

21:22 if you can really map everything to your keyboard just the way that you like.

21:25 It's like having Vim, but for all the apps out there.

21:28 Okay, that's very neat.

21:29 Yeah, I've remapped Caps Lock to my Activate Mac Whisper.

21:34 So if I hit Caps Lock and hold this down, then I can dictate.

21:36 And I know computers have had dictation for a long time,

21:38 but it's been really bad, right?

21:40 The built-in dictation to macOS or Windows isn't great,

21:44 especially when you try to talk acronyms like, hey, do PyPI, and it's like...

21:47 Yeah, Mac Whisper.

21:48 No, but Mac Whisper, it's not like all the way there,

21:51 but I will say it surprised me in quality a few times, definitely.

21:55 Yeah, yeah.

21:56 And that app, you can go in and actually set replacements.

21:58 Like if you ever think you're going to write this, write that instead.

22:02 So I've done that with like Talk Python because it always just turns it into a camel case combined.

22:08 I'm like, no, those are two separate words.

22:10 You know, whatever.

22:10 You can sort of customize it a bit.

22:12 Yeah, and I think my favorite one, I think Mac Whisper actually gets this one wrong still,

22:17 But whenever I write scikit-learn, which is a very popular data science package, it always

22:22 translates it to psychologists learn.

22:26 Which, you know.

22:30 I never want you to ever write this series of words.

22:33 Because if I have to, I'll just type it all in time.

22:36 It's not my common thing.

22:39 It's one of these moments where actually I type scikit-learn often enough where it's

22:43 like almost becoming the issue.

22:44 So I'm like at the verge of adding these rules manually

22:47 for all these weird open source packages that I interact with.

22:49 But yeah, it's take ergonomics serious people.

22:52 That's the one thing I wanna say.

22:53 You don't always have to buy a super expensive keyboard.

22:56 The, if you wanna explore like programmatic keyboards

22:59 because you can customize things, that's an excellent reason.

23:01 But like take a break and do exercises and just, you know, be healthy.

23:05 That's the, that's the, that's how you win a marathon.

23:07 - Yes, that's for sure.

23:08 Now this is not the Vincent's keyboard review marathon,

23:12 But let's wrap it up with, if you could take any keyboard

23:17 hanging around on your wall there, which one would you use?

23:19 - Well, so there's four pairs of these, so it's probably this one.

23:25 So there's a couple of other boards that are great too.

23:27 This board is not the board I would recommend to everyone,

23:29 but if you have serious RSI issues, I do think the GloVe 80 is your best bet.

23:34 It's simple, in terms of the shape, it is probably the most ergonomic shape for most hand sizes.

23:39 - Yeah, okay, awesome.

23:40 All right, well, let's switch over to talking about programming.

23:45 Yes.

23:45 LLMs with LLMs, not programming LLMs.

23:48 And I guess, like you already called out, you wrote this course called

23:52 LLM building blocks for Python.

23:54 Super fun course, really neat, it's pretty short and concise.

23:57 And it really talks about how can you reliably add some kind of LLM into your code.

24:04 I guess what you're talking about in this course really applies regardless of

24:08 whether it's a self-hosted one or it's open AI or Anthropic, right?

24:12 You can have, there's some, some choices you can make on which LLM to use, right?

24:17 So the, like the main idea I had with the course was like,

24:22 an LLM is a building block at some point, but it's very unlike a normal building block when you're dealing with code,

24:27 because normally with code, you put something into a function and like one thing comes out.

24:32 But in this particular case, you put a thing into a function,

24:34 you have no idea upfront what's going to come out.

24:36 And not only that, but you put the same thing in twice

24:38 and something else might come out as well.

24:40 So that means that you're going to want to think about this tool

24:44 a bit more defensively.

24:45 It's a weird building block.

24:46 It's like you have to put a moat around it because otherwise the building block is going to do stuff

24:49 you don't want it to do, almost.

24:52 And some of that is syntax.

24:53 Some of that is how do you think about these Lego bricks.

24:56 And some of it is also just what is good methodology in general

24:59 to statistically test if the thing is doing roughly what you want it to do.

25:03 Nice.

25:04 That's the gist of it.

25:06 Yeah, cool.

25:07 Yeah, I pulled out a couple of ideas, concepts, and tools

25:10 that you talked about throughout the course.

25:12 And you don't have to have taken the course to have these things be useful.

25:15 I just thought it might be fun to riff on some of the things

25:18 you touched on here.

25:20 The main thing I think will be fun is--

25:22 it's been half a year, I think.

25:23 It will be fun how much of it is still intact.

25:25 I think most of it still definitely is.

25:27 But it might be fun to see if we can find anything that might be dated, just to see if the world has moved on

25:33 quickly.

25:33 I think there's only one thing, but I'm just kind of curious.

25:36 Let's see.

25:37 Yeah, all right.

25:37 Well, let's keep our radar up for that.

25:40 It's definitely something that's more changing quicker

25:43 and has a higher likelihood of being dated.

25:46 But I think it holds up pretty well.

25:47 Yeah, okay.

25:49 One of the things I remember emphasizing is you want to do some stuff like caching.

25:53 So let's say you've got a function and you use an LLM for it.

25:56 And let's keep it simple.

25:58 Let's say we're just making summaries.

25:59 So talk Python episode paragraph goes in, single sentence is supposed to come out.

26:04 Something like that.

26:05 Okay, well, you might have a loop and you're going to do maybe one pass, try one LLM with

26:13 one type of setting, try another LLM with different type of settings to generate all

26:16 this data.

26:17 It would be a shame that you're going to use an LLM, which is kind of an expensive compute

26:22 thing, if you put the same input in by accident and then you incur the cost twice.

26:26 That would really stink.

26:27 So one of the things you always want to do is think a little bit about caching.

26:31 And there's a Python library called disk cache that I've always loved to use.

26:34 And I highly recommend people have a look at it.

26:36 I think Michael, you've also used it in one of your courses before.

26:39 The trick is we have to talk about this.

26:40 It's so good.

26:41 It is so good.

26:42 It is SQLite and is so good.

26:45 It is even better than SQLite.

26:47 It is unbelievably good.

26:48 And I have you to think I knew about it, but yeah, whatever.

26:52 And then after I saw you use it, I'm like, genius.

26:54 It is genius.

26:56 Yes.

26:56 No, so it's like having the Eliru disk cache.

27:00 But it's also on disk.

27:01 So if you were to restart Python, you still have everything in cache basically.

27:05 And it's a SQLite database, so you can always inspect all the stuff that's in there.

27:10 If you wanted to, you can also do fancy things like add time to live to every single object.

27:16 And this is something you could do in a Docker container for a web app.

27:19 But the main thing that's always nice when you're dealing with LLMs is you always want

27:23 to be able to say in hindsight, like, okay, how did this LLM compare to that one?

27:27 You want to compare outputs.

27:28 then just writing a little bit of a decorator on top of a function

27:32 is the way to put it in SQLite, and you're just done with that concern.

27:35 That is just amazing.

27:37 And we're using this cache directly.

27:40 If you're using LLM by Simon Willison from the command line,

27:43 there's also a mechanism there so that you can get it into SQLite

27:46 if you wanted to.

27:46 So that's also a feature you could consider.

27:49 But--

27:49 MARK MANDEL: Of course, he's going to put something in SQLite.

27:51 Like, he can write a library that doesn't put something in SQLite,

27:53 given his data set project.

27:55 MARK MANDEL: It's Simon Willison.

27:56 He'll put SQLite in SQLite if it's a Sunday.

27:58 Yeah.

28:01 But if you haven't used Discash before, it definitely feels like one of these libraries

28:06 that because I have it in my back pocket, it just feels like I can tackle more problems.

28:10 That's the short story of it.

28:11 And again, the course uses it in a very sort of basic fashion,

28:15 but knowing that everything you do in an LLM only needs to happen once,

28:20 if you're interested in using it once, that just saves you so much money.

28:24 Yeah, and it's so useful.

28:25 So one of the things you did in the course is you said,

28:27 All right, the key for the value that we're going to store is going to be the model, the model settings and the prompt as a tuple, right?

28:35 Something along those lines.

28:36 And then you use that as the key.

28:38 So if any of those variations change, does the model change?

28:41 Do the settings change?

28:42 Or like anything, that's a totally different request.

28:45 And then you just store the response.

28:46 And then boom, if you ask the exact same question of the same model with the same prompt with the same settings, why do you need to go and wait 10 seconds and burn money and environmental badness?

28:56 when you could literally within a microsecond get the answer back.

29:00 Yeah, and there's also a fancy thing.

29:01 It's a trick you can do on top.

29:02 So sometimes you want to say, well, there's a statistical thing also happening.

29:06 So sometimes I want to have one input and actually store maybe 10 outputs

29:10 so I can look at all these outputs, maybe annotate that later.

29:13 And the way you solve that is you just add an integer to the tuple, basically.

29:17 And then you're also able to store many outputs if you really like.

29:20 The sky's the limit.

29:21 The fourth one, the eighth one, whatever.

29:23 Yeah, that's flexible.

29:25 Yeah.

29:26 And it does a whole bunch of neat, neat things that are really, really wild.

29:29 Like, so it looks like just, Hey, I put this value into it.

29:32 Right.

29:32 And it stores it.

29:33 It's really powerful because across application executions.

29:37 So like maybe if you're caching that response in your notebook and what are

29:40 you doing some testing, if you come back later, you've start the notebook back up

29:44 or restart the kernel or whatever, like it's not like an LRU cache.

29:47 It remembers cause it's stored somewhere in like temporary storage in a local

29:51 SQLite file, which is amazing.

29:53 It also has interesting ideas.

29:56 I'm not sure really where they are, but it has different kinds of caches as well.

30:00 So maybe you're storing a ton of stuff in there.

30:03 And so it'll do basically built-in sharding.

30:06 Oh, yeah.

30:06 Multiple SQLite files.

30:08 And it's really, really good.

30:10 This is a deep library.

30:11 This is not just, oh, yeah.

30:13 It's like LRU, but to desk.

30:15 So the cool thing about that library is it really does go deep.

30:18 But if you really just want to use it as a dictionary, you can.

30:22 That's the thing that I really love about it.

30:23 But for all intents and purposes, you just treat it as a dictionary, and you're good.

30:26 Or use it as a decorator on a function, and again, you're good.

30:31 So again, it's one of those little hacks where, oh, if you just know about the library,

30:36 you just become more productive at typical Python stuff.

30:39 Yeah.

30:39 I'll give you a place where I'm using it, actually.

30:41 I use it on some LLM stuff, like where I'm programming against an LLM, for exactly the

30:46 reason you did.

30:46 Because if you have the same inputs, don't ask the question again.

30:49 You just hear the answer.

30:50 It's real, real fast.

30:51 But if you go over to like Talk Python, let's see here.

30:55 Go over to the guests, for example, right?

30:56 So there's a bunch of guests.

30:58 And here we have Vincent.

30:59 Not that Vincent.

31:00 Not that Vincent.

31:01 There you are, that Vincent.

31:01 All these Vincents.

31:04 They're all over the...

31:05 I have like 560 guests or something.

31:07 There's a lot.

31:08 But in here, you'll notice that if you go into the view of the source on this thing,

31:13 like all over the place, anytime there's a picture, it'll have this little cache-busting ID on the end.

31:20 And that's fine when it's served locally because you can just look at the file

31:23 and go just the file still with the same cache at startup.

31:25 But if it comes from like an S3 blob storage, you know, and the app restarts,

31:30 how do I know what that is?

31:31 Like it has to go, it would have to re-download the entire content of the blob.

31:37 So check this out.

31:39 Yeah.

31:39 So I just added.

31:40 Yeah, yeah, yeah.

31:41 So this feels like there's something between the proper CDN

31:44 and then getting it from S3.

31:45 Like there are these moments when you want to have something that's kind of in between.

31:48 And then disk cache could actually be a good solution for it.

31:51 Exactly.

31:52 So what the site does is it has a disk cache.

31:54 And anytime it says, hey, I want to refer to a resource that's external,

31:59 it'll download it once, compute the hash, and then store it in the disk cache unless you change something behind it.

32:05 And so it's automatically using this, and it makes everything like,

32:09 there's never stale resources, and it's instantly fast,

32:12 even if they're served out of something remote like S3 equivalent.

32:15 And do you also do a time to live?

32:19 Is it also the time to live is also like every day it's allowed to refresh once or something like that, I suppose?

32:24 Yeah, for the S3 stuff, I don't because I've set up all the admin functions that if I ever change one through the admin, it deletes it out of the cache.

32:32 Gotcha.

32:33 So it's like it's internally consistent.

32:35 But for like other things, like if it parses something out of the, say, the description, which is set in the dictionary, that stuff, it's just got a time to live of like a day or something.

32:46 And it's got like, there's a bunch of those.

32:47 I'm using all these different places.

32:49 And wow, it's so good.

32:50 I just wanted to say thank you, because I knew we were going to talk about it today,

32:53 because this is so good.

32:54 Yeah, I know.

32:57 If it was part of the standard library, I would honestly not be surprised.

33:01 That's also the story with that.

33:03 But yeah, SQLite is great.

33:04 DiskCache is great.

33:06 It feels like a dictionary, but it gives you so much more.

33:08 It's great.

33:09 That's the only thing I can say about it.

33:10 So people are probably like, wait, I thought we were talking about LLMs.

33:14 Yeah, we are.

33:14 I think this is one of the interesting things, It's because there's all these interesting tools and it's not even about using agentic AI.

33:20 And it's not like there's really cool libraries and tools that you can just apply to this LLM problem, but also apply everywhere else.

33:28 Right.

33:29 It's one of those.

33:30 So it's one thing I have found with like LLMs in general if you're building software with it.

33:34 On the one end, I think it can be very helpful if you're like a proper like senior engineer kind of a person because then you know about things like, oh, I want a cache.

33:40 And what's the pragmatic cache?

33:41 And you can also pick Redis, by the way.

33:43 If you would have used Redis for this, that could have also worked.

33:45 Just that I think--

33:45 - Sure, it would have been fine.

33:46 - Yeah, it would have been fine.

33:47 - But you know what I like is there's no servers.

33:49 I don't have to deal with the servers.

33:51 Like, it's just, it's a file.

33:53 - It's very true.

33:54 - And it just keeps it simpler.

33:55 - It's totally true.

33:56 But the point I wanted to get out here, like your previous experience as a proper engineer

34:01 will still help you write good LLM software.

34:03 However, from a similar perspective, I also think that we do have this generation

34:07 of like data scientists, and maybe data engineer, data analyst kind of person,

34:11 like thinking analytically being quite critical of the output of an algorithm.

34:15 That's also like a good bone to have in this day and age, I would say.

34:20 Because if I've built a couple of recommenders back in my day, like it's been a decade now,

34:24 but like one of the things you do learn when you're building a recommender is that you're stuck with this problem of,

34:29 hey, I gave this user this recommendation and they clicked it.

34:32 Would they have clicked this other thing if I would have recommended it to them?

34:36 And suddenly you're dealing with a system that's like really stochastic and hard to predict.

34:40 and you have to be kind of strict about the way you test and compare these different algorithms.

34:44 And you want to think twice about the way you A-B test these things.

34:47 And, oh, actually, just like disk cache is useful as a tool,

34:50 having a little bit of methodology statistically in your mind will also help you

34:55 because comparing LLMs, a lot of it is doing evaluations, being kind of strict about that.

35:01 And that's also what I try to do in the course.

35:03 I try to just show you that if you're really strict about evaluations,

35:06 then you can also learn that for some problems, you're still better off using scikit-learn

35:09 because you just evaluate it and then you learn that like the number is better on the side could

35:13 learn side of things yeah that's when you feel like oh maybe i did it wrong where you paid

35:17 a bunch of money to run expensive slow lms and you're like i would just use well so it's funny

35:22 you say that so i'm i've actually been talking to a bunch of people that do lms at companies here in

35:26 the netherlands and you know you you go to a pi data you go to a conference and you give them just

35:31 enough beer so they're honest in that kind of a situation the nda curtain opens just a little or

35:37 or whatever.

35:38 Plus, more deniability is the name of the game in this one, yes.

35:42 But then the stories you hear, which I did find somewhat encouraging,

35:45 is they do all kind of go, well, there's budget now to do AI stuff,

35:49 which means that we try out all the different LLMs,

35:51 and we really can invest in evals and that sort of thing.

35:54 And funnily enough, we also put some base models in there,

35:57 like as a standard benchmark that we should beat.

35:59 And I've heard a story a bunch of times now that because of the hype around LLMs and AI,

36:05 after it was implemented, after they did all the benchmarks,

36:07 It turns out that AI is the reason that scikit-learn is now in production in a bunch of places.

36:10 And it's also the same thing with like spaCy.

36:13 Because what a lot of people do learn is that, hey, if the spaCy model or like the lightweight model, so to say,

36:19 is like somewhat equivalent to the LLM model after you give it like enough training data,

36:24 which you do want to have either anyway, because you need to need that for evaluations.

36:27 Well, typically those models are more lightweight and they will always produce the same output.

36:32 Same thing goes in, same prediction will always come out.

36:34 And you can really fence it off, have it on like a normal VM.

36:37 That's also totally fine.

36:39 Oh, and you know, another benefit, it's super lightweight to run.

36:42 You just need a Lambda function and you're good, as opposed to like a GPU or like a huge cloud bill or something like that.

36:48 So some of the stories that I'm hearing do suggest that,

36:52 okay, these LLMs are also helping out like standard data science work, if it were,

36:57 if only because management now really does want to be serious about the investment

37:00 and really wants to do the experiments.

37:03 portion of Talk Python To Me is brought to you by NordStellar. NordStellar is a threat exposure

37:07 management platform from the Nord security family, the folks behind NordVPN that combines dark web

37:13 intelligence, session hijacking prevention, brand abuse detection, and external attack service

37:19 management. Keeping your team and your company secure is a daunting challenge. That's why you

37:24 need NordStellar on your side. It's a comprehensive set of services, monitoring, and alerts to limit

37:30 your exposure to breaches and attacks and act instantly if something does happen. Here's how

37:35 it works. Nordstellar detects compromised employee and consumer credentials. It detects stolen

37:42 authentication cookies found in InfoStealer logs and dark web sources and flags compromised devices,

37:48 reducing MFA bypass ATOs without extra code in your app. Nordstellar scans the dark web for

37:54 cyber threats targeting your company. It monitors forums, markets, ransomware blogs, and over 25,000

38:01 cybercrime telegram channels with alerting and searchable contexts you can route to Slack or your

38:07 IRR tool. Nordstellar adds brand and domain protection. It detects cyber squats and lookalikes

38:13 via visual, content similarity, and search transparency logs, plus broader brand abuse

38:19 takedowns across the web, social, and app stores to cut the phishing risk for your users.

38:24 They don't just alert you about impersonation, they file and manage the removals.

38:29 Finally, Nordstellar is developer-friendly.

38:31 It's available as a platform and an API.

38:35 No agents to install.

38:36 If security is important to you and your organization, check out Nordstellar.

38:40 Visit talkpython.fm/nordstellar.

38:42 The link is in your podcast player's show notes and on the episode page.

38:45 Please use our link, talkpython.fm/nordstellar, so that they know that you heard about their service from us.

38:52 And you know what time of year it is.

38:53 It's late fall.

38:55 That means Black Friday is in play as well.

38:57 So the folks at Nord Stellar gave us a coupon, BlackFriday20.

39:02 That's BlackFriday, all one word, all caps, 20, two, zero.

39:05 That grants you 20% off.

39:07 So if you're going to sign up for them soon, go ahead and use BlackFriday20 as a code,

39:11 and you might as well save 20%.

39:13 It's good until December 10th, 2025.

39:16 Thank you to the whole Nord security team for supporting Talk Python To Me.

39:21 You could say, we're going to use your mandate is to add AI.

39:24 And then you go user, well, we got to see how well it's working.

39:26 So we tested it.

39:27 And then we have to compare it to something of a benchmark.

39:30 Exactly.

39:32 The benchmark is fast and effectively free and about as good.

39:36 So we're good.

39:38 We're just going to do that.

39:39 You know, I think it's fine.

39:41 Some organizations need like mandate from above in order to get something done.

39:45 And this LLM craze, if nothing else, does seem to have caused the mandate from above.

39:50 I'm sure it is.

39:52 So that's something I would keep in the back of your mind as well, dear listener.

39:56 Like sometimes that mandate can also give you permission to do other things.

40:00 Yeah.

40:01 Not working for a large enterprise type company for quite a while.

40:05 I imagine I'm pretty blind to how that is transforming directives from the top.

40:11 But I'm sure bosses are like, I'm using ChatGPT and it's so much better than our software.

40:15 What are we going to do about that?

40:16 you know yeah it's uh well i mean i'm in the developer tooling space so i still talk to big

40:21 companies and small companies as well you do notice that big companies they work differently

40:25 than small companies that is certainly true um for better or worse like there's also good there's

40:28 also good reasons why bigger companies like if i'm a bank like uh i'm pretty sure it's a good idea to

40:33 also have some rules that i have to abide by which yeah i'm just thinking more of like how disconnected

40:38 is the higher level management from the product and like how much do they just like dictate a thing

40:43 Okay, so let's talk about some of the tools. One of the things you primarily focused on was using

40:50 Simon Willison's LLM library. Tell us about this thing. So the funny thing about that library is

40:56 that it's actually kind of more meant as a command line utility. I think that's kind of the entry

41:00 point that he made. But then he also just made sure there was some sort of a Python API around it.

41:06 And after looking at that library, and also after playing around with all these other libraries,

41:11 and I'm about to say this, but this is a compliment.

41:13 I just found the LLM library by Simon Willison by far to be the most boring.

41:18 And I mean that really in a good way, just unsurprising,

41:22 only does a few things.

41:23 The few things that it does is just in a very predictable way.

41:26 And especially if you're doing rapid prototyping, what I felt was just kind of nice is that

41:30 it does feel like you have building blocks that allow you to build very quickly,

41:33 but it doesn't necessarily feel like you're dealing with abstractions

41:35 that can sort of wall you in at some point.

41:38 So for a hello world getting started, just do some things in a very Pythonic way,

41:43 this boring library really did the trick.

41:46 And I'm also happy to report it's still a library that I use.

41:49 It's still definitely a library in my tool, though.

41:51 Whenever I want to do something real quick, that is definitely a tool that I refer to.

41:56 And one of the things--

41:57 Is it kind of an abstraction across the different LLMs

42:01 and that kind of stuff?

42:02 If I want to talk to Anthropic versus OpenAI, that doesn't matter?

42:06 So the way that this is built is also, I think, with good taste.

42:09 So what you don't want to do as a library is say, like, I'm going to basically support every library under the sun.

42:15 Because as a single maintainer in the case of Simon Willis, you're just going to drown in all these different providers.

42:20 So what he went for instead was a plugin ecosystem.

42:23 Now, the downside of a plugin ecosystem is that then you defer the responsibility of maintaining a plugin for a specific source to another maintainer.

42:30 But you might get someone who works at Mistral to make the Mistral plugin.

42:34 And you might get someone who works at Open Router to make the Open Router plugin, etc.

42:38 So you do distribute the workload in kind of a nice way.

42:42 And all the bigger models are definitely supported.

42:45 So the Anthropic and the OpenAI ones, those are just always in there.

42:50 But you will also definitely find some of the more exotic ones

42:54 that they will also just have a plug in themselves.

42:56 And one thing that also helps under the hood is that OpenAI has a standard under the hood.

43:01 Their SDK has become a bit of a standard across industry.

43:04 So you can also reuse the OpenAI stuff.

43:07 It would not surprise me at all that if you were just

43:09 to change a URL in the setting, that you can also connect to Mistral via the OpenAI objects.

43:15 I would have to double check, but that wouldn't surprise me.

43:17 Yeah, that is a very common way of just like, you know what, we're all going to just adopt Open.

43:23 It's a little like S3.

43:24 Like when I was saying S3 earlier, I was actually talking to DigitalOcean.

43:28 But it doesn't matter.

43:29 I'm just still using Boto3 to talk to it.

43:31 Yeah, it does feel weird like, oh, you want to use DigitalOcean?

43:34 you have to download a SDK from a competing cloud provider and then you can.

43:39 Exactly. It's so weird.

43:41 But I mean, the thing I do find that it's just a little bit funny here is like technically,

43:45 this thing is meant to be used from the command line. It just happens that it's written in Python

43:49 and it just happens that also it has a decent Python API. And that's the thing I actually end up

43:54 using more than stuff from the command line. That's because I do a lot of things in notebooks. So the

43:58 stuff that I tend to use is a little bit more in the Python side.

44:01 Sure, of course. Same here.

44:02 Yeah, I don't think I would use it very much on the command line.

44:05 But, you know, I can see using it as a step in a series of things happening on the command line,

44:11 like some sort of orchestration, like X, Y, ask the LLM, Z, you know?

44:16 Well, the main thing I actually use it for from the command line is you can do a git diff

44:21 on whatever you're working on and then sort of pipe that through to the LLM to make the commit message.

44:26 Like there are like these little moments like that where you do want to have a little automation,

44:29 maybe in CI and like being able to do this from a command line is definitely useful it's just not

44:34 the thing i use it for most or like if i use it that way it's a thing i automate once and then

44:38 never really look at it but the interaction really for me is more in the notebook oh yeah that's a

44:44 very cool librarian i definitely need for um like one thing that might be fun to point out because

44:49 that's something i built i think uh around the same time like if you were to type into google

44:54 uh smart funk and then my github alias koning a k-o-a-n-i-n-g um because it's a little

45:02 okay you already had it open good man i'm really good with google actually i use a start page these

45:07 days but i'm sure i could find it there oh okay here you're well you have an ergonomic keyboard

45:11 so there you go quick typing um already yeah no so like a thing i was able to build on top of uh

45:16 the thing that simon willison made and it's something that appears in the course as well

45:24 that. Yeah, there you go. So basically what this function does is you can add a decorator.

45:30 You just have a normal Python function. The inputs, you just put types on it. So you can say like,

45:35 this input is a string, this input is an integer, or what have you. You then add a decorator that

45:40 says what backend you want to use. So GPD4 or anything that Simon Willis and supports. And then

45:45 the doc string is the prompt you give to the LLM. So you can do something like, hey, I have a function

45:51 that accepts a term called paragraph.

45:53 That's a string.

45:54 And then the doc string of the function says, summarize this.

45:56 And then lo and behold, you now have a Python function

45:59 that can do summaries.

46:00 I see.

46:01 And the doc string can be effectively a Jinja template.

46:05 Yes.

46:05 If you wanted to, it could be a Jinja template.

46:08 Alternatively, what you can also do is you can also say, well, what the function returns

46:12 is the prompt.

46:13 That's also something you can do.

46:15 Got it.

46:16 But this is just a very quick hack for some of the stuff

46:19 that I want to do real quick.

46:20 I don't necessarily recommend other people to use this,

46:23 but this is something you can build on top of the LLM library

46:25 from Simon Willison.

46:26 So if you want to make your own syntactic sugar, your own abstractions, it's definitely something

46:29 that you can also do.

46:31 And also, again, the name of the game here is quick iteration.

46:34 So also feel free to think of your own tools now and again.

46:37 But you want the base layer to be as boring as possible.

46:39 And Simon Willison's library, again, does that in a good way.

46:42 Right, right.

46:43 Just don't make me think about the details.

46:45 Yes.

46:46 Let me swap out the model providers, which is actually

46:48 doing quite a bit, but not conceptually.

46:51 Yeah.

46:52 So again, one thing I do think at this point in time,

46:56 if you've not played around with this software stuff already,

46:59 I do recommend maybe doing that sooner rather than later.

47:02 And I think Simon Willison's approach, if you're already a Python person,

47:04 it's probably the simplest way to get started.

47:06 But I do want to warn people in the sense of these are just tools.

47:10 Tools are useful.

47:11 Tools are great.

47:13 But at some point, it's not so much about the tools that you use,

47:15 but more about how you think about the tools, Like the way you interact with these tools,

47:19 is it not some, like you can use a hammer in many different ways,

47:22 but the way you think about a hammer is like also equally important.

47:25 - Yeah, 100%.

47:26 So speaking of thinking about how this goes, one of the challenges is,

47:30 so let's just take this example here.

47:33 It says, the example on your smartphone, GitHub readme, it says generate summary,

47:38 take some text, it says generate a summary, the following text, colon text.

47:42 - Yes, text.

47:43 - Super straightforward.

47:44 But what if you, what if you said, change the prompt a little bit said, give me all of the

47:48 Python libraries discussed in the following markdown content.

47:52 Right.

47:53 And do you want a novella back, like a novel back when, when really what you want is like,

47:59 I want a list of strings and URLs or whatever.

48:03 Like how do I get programmability instead of just chatability, I guess.

48:07 Yeah.

48:08 So the thing that Simon Willis's library does allow you to do, is you are able to

48:13 say, well, I have a prompt over here, but the output that I'm supposed to get out, well, that has to be

48:18 a JSON object of the following type. And then you can use Pydantic. So you can do something like, well,

48:22 I expect a list of strings to come out over here in the case that you're trying to detect Python

48:25 libraries or Pokemon names or what have you. Or you can say, well, there's more like a classification.

48:31 I believe you can pass a literal with like only these values and that's an also constraint. And

48:35 the reason that you can do that is some of these LLMs are actually trained on specific tasks. One of

48:41 these tasks could be tool calling. Another one of these tasks that are trained for these days is

48:45 that you have to say the right word here, something, something output, structured output,

48:51 so that the LLM can sort of give a guarantee that if you declare that you get a list of strings out,

48:56 that you actually do get a list of strings out. And that's something that these things are

48:59 typically trained for. So that's something you can do. If you're using open source models,

49:05 definitely check them out. They're cool. But quality does vary, is what I will say.

49:10 So that's something they always keep in the back of your mind.

49:12 And also the more complex you make your PyDonic objects,

49:15 like if you make like a nested thing of a nested thing of a nested thing,

49:18 at some point the LLM is going to have a bit of trouble with that.

49:21 Even though PyDonic is of course great, but like you make it harder for the LLM to do the right thing at some point.

49:27 But that's also a thing that definitely you want to use

49:30 if you're doing stuff with software.

49:32 Like the really cool thing about PyDonic is you can say,

49:34 I expect something that looks like this to come out

49:36 and you can force an LLM to also actually do that.

49:39 Like, you will get the right types that come out.

49:41 The contents might still be maybe a bit spooky.

49:43 But at least you get the right types out, which makes it a lot more convenient to write software.

49:47 You can say, I can loop over the things that sent back,

49:49 and you're not into vague text parsing land.

49:53 Well, the main thing you get is--

49:55 I remember when the first OpenAI models sort of started coming

49:58 out, I worked back at Explosion.

50:00 We made this tool called spaCy there.

50:02 And one of the things that we were really bummed out about

50:04 was LLMs can only really produce text.

50:06 So this structural, like if you want to detect a substring in a text, for example,

50:11 which is the thing that spaCy is really good at, then you really want to guarantee, well, I'm going to select a substring that actually did appear in the text.

50:18 I want to know exactly where it starts and exactly where it ends, and that's the span that I want to select.

50:22 So normally in NLP, text comes in and structured information comes out.

50:27 But here come these LLMs and text comes in and text comes out.

50:30 And then you've got to figure out a way to guarantee that structured output actually comes out.

50:34 So that's the thing that the LN providers also acknowledge.

50:36 So that's something that they're actually training these models for.

50:39 But I think it's a lot better these days, but it was definitely a huge part of the problem

50:45 like three to four years ago when these things just happened.

50:47 It was super annoying.

50:48 So if you use Pydantic, you define a Pydantic model that drives from Pydantic.basedModel.

50:54 You put the types, type information, fields, et cetera.

50:58 And then what does it do?

50:59 Does it say, kind of use the schema generation that Pydantic already has?

51:04 Like people are familiar with FastAPI.

51:06 You set the response model, it'll do open API spec, which I believe just comes more or less from Pydantic, right?

51:12 I could be.

51:13 I think it's a JSON standard in the end.

51:16 Like, you know how you can generate JSON schemas as well?

51:18 I think it's that spec that is using under the hood

51:21 that is then also giving off to OpenAI or what have you.

51:24 So I think in the end, it's a JSON spec, but I could be wrong.

51:27 But it's driven by the Pydantic model.

51:29 And then that's given as part of the prompt.

51:31 Like, here's my question.

51:32 your answer will look like this.

51:34 And it uses the Pydanic, something to that effect.

51:36 Yeah, there's one small caveat there in the sense that you can also add custom types

51:42 with Pydanic.

51:42 So you can do something like, well, this is a non-negative integer, for example.

51:46 And you can definitely imagine that, especially with these validations

51:49 that you can write in Python, not everything that you can come up with there

51:52 is going to be supported by this JSON spec.

51:54 So you are going to get some of these weird edge cases

51:56 maybe where Pydanic can do more than what the JSON spec can do on your behalf.

52:00 There is a really cool library, though, called Instructor that actually--

52:04 Yes, I was just thinking about that.

52:06 So what they do is actually kind of cute.

52:09 The impression I have is that they acknowledge that PyDanic can actually do a bit more than what the JSON

52:13 spec can do, especially because the validator that you write

52:16 can really be a custom whatever thing you care about.

52:19 So what they do as a trick is they say, well, if the LLM gives me something back that doesn't get validated

52:25 by PyDanic, we take the error message, together with the thing that we got back,

52:30 make a new prompt to the LLM and just ask it again.

52:33 And see, maybe now with the hint, it is able to give us the proper structure back.

52:37 That's sort of the term.

52:38 It sounds silly, but anyone who's done agentic AI and stuff,

52:41 like, you know, Claude or whatever, that's a lot of the magic where you say,

52:44 it tried this.

52:45 Oh, I see.

52:46 I actually forgot an import.

52:47 Oh, I see.

52:48 I passed the wrong command.

52:49 Let me try again a different way.

52:51 Oh, now it's working.

52:52 And, you know, I think there's a lot of potential with that.

52:54 So, well, definitely true.

52:56 The one thing I think is safe to say at this point,

52:59 although another researcher, right?

53:01 So don't pin me down on this.

53:02 That library did originate from an early era of LLMs

53:05 when the structured stuff wasn't really being trained for yet.

53:09 And it is my impression that LLMs have gotten a fair bit better

53:13 at this structured task at this point.

53:15 So you could argue that maybe that library is solving

53:18 a solved problem with the technique that they've got.

53:21 But I still don't want to discount it, because you inevitably, at some point,

53:25 you might want to go for a very lightweight model that you can run locally that isn't fine-tuned

53:29 structured output task. And in that situation, this technique can actually do a lot of good.

53:33 Like you might be prompting the LLM for like three times, but you might be able to get away

53:37 with that because the model is just so much lighter because it doesn't have to do all those tasks that

53:41 those big LLMs are trained for. Right. Especially if you're running on your machine or you're

53:46 self-hosting the model, you're not like, oh, it's going to cost us so much money. It's just,

53:49 it'll take a little longer, but it's fine. Yeah. So like there's a lot of support for

53:53 instructor. There's also Olama support for the LM library from Simon Willison,

53:57 by the way. But yeah, you can definitely imagine this is still going

54:01 to be relevant with the Olama stuff going forward.

54:04 And also, it's a reasonable trick if you think about it.

54:08 The main thing that we do is we just add a retry mechanic to the structured outputs by using

54:11 the PyDonic validator, effectively.

54:13 I mean, think about what you probably do, certainly what

54:15 I do when I'm working with ChatGPT or something.

54:18 It gives me a wrong answer like, no, this is not what I

54:19 wanted. This is what I wanted. It's like, oh, okay.

54:21 I see.

54:22 And it's kind of automating that, more or less.

54:25 It's certainly an attempt, yeah.

54:27 In my experience, I have also seen moments when it sends the same error back five times,

54:33 and then it just kind of fails.

54:34 So there is also a limit to the retry mechanic.

54:37 Sure, sure.

54:38 It's a little bit like tenacity or stamina.

54:41 Exactly.

54:41 APIs or whatever.

54:43 It uses tenacity under the hood, if I'm not mistaken.

54:45 So there's definitely that mechanism in there.

54:47 Nice.

54:48 I just-- didn't work.

54:49 Try it again.

54:50 It's like turn it off and turn it on again.

54:51 See if it works now?

54:52 Yep, pretty much.

54:53 Television.

54:54 Yeah, a little more smart than that, though, because it does validate and return the error.

54:58 So super cool.

54:59 Yep.

54:59 Okay.

55:00 Have you worked with, I guess, I don't know, higher order programming models?

55:05 So, you know, I saw Samuel Colvin yesterday talking about Pydantic AI.

55:10 I saw on X, I saw Sidney Ruckel talking about some big release, like a 1.0 release

55:17 or something at LangChain.

55:18 What are your thoughts on some of these higher order orchestration libraries?

55:22 I mean, the main thing for me, at least, this was a demo I did do with the Pydanic AI one.

55:29 I do think types are quite powerful when you think of LLMs.

55:34 There's a live stream you can check on the Marimo YouTube

55:38 if you're keen to see a full demo.

55:39 But let me just sketch you a problem that used to be really hard.

55:42 Six years ago, we used to work at this company called Raza.

55:45 We work on chatbot software.

55:47 And a thing that was really hard is you--

55:49 Let's say you're a chat bot, you can order a pizza.

55:51 So you can do things like, yep, that's the one.

55:54 It's like also one of the PyDonic AI engineers, and he's also the creator of Starlet, by the way.

55:58 Cool project, Mariemost built on top of Starlet.

56:01 Main maintainer of that project.

56:02 He deserves to get more high fives from the community,

56:04 so do you want to point it out?

56:07 Well, let's say you're working on a pizza bot, and okay, someone is ordering a pizza.

56:11 Suppose you go to the bot and you say, "Hey, I want to order a pepperoni pizza."

56:15 But you can imagine there's a pizza type, and a pizza type says, "What's the kind of pizza?

56:19 What's the size of the pizza?

56:20 And do you want to have extra toppings?

56:22 And that has to be a list of ingredients, let's say.

56:24 But if I tell you, hey, you want to have a pepperoni pizza,

56:29 but I don't know the size yet, well, then I need the LLM to ask a good follow-up question.

56:33 OK, how are you going to build that logic?

56:35 Because that can get quite gnarly quite quickly, especially if, in this case, the pizza is simple.

56:39 But you can come up with a complicated object, like a complicated form almost.

56:42 And then--

56:43 Right, right.

56:43 I only need to ask this question if they've answered that one

56:46 in that way.

56:47 Right, it could get really tricky, yeah.

56:49 Well, and then the question is, how can you actually get the LLM to do this for you?

56:53 And there's a trick that you can do with a type, because what you can say is,

56:56 well, the response from the LLM has to be of type pizza,

56:59 or bar string.

57:01 And then if the prompt says something along the lines of,

57:03 well, parse the output of the user such that it is a pizza,

57:06 but then if this validation error occurs, oh, how do you express that logic?

57:10 Well, you can express it with a type.

57:11 You just say, this is a string, or it's the type that I'm actually interested in the pizza.

57:15 And then the LLM has to figure out, is the response either going to be the string where I'm asking the user for missing information

57:21 or is it going to be the pizza object and if it's a pizza object then the code will actually talk

57:24 to the back end so you do get to rethink how typing works because we have llms at our disposal now you

57:30 can actually have the llm handle some of that logic as long as your type is like strict about

57:34 it or like well defined i should say um it's it's a bit of a weird advanced feature of types and llms

57:40 but i can definitely imagine uh some of the pydantic ai stuff does have does use this trick in order

57:46 to perform some of the higher order stuff and maybe you don't need a full framework as long as you are

57:51 really um like deliberate about your types is what i that that's kind of the mentality i try to

57:56 have right now like let's interesting let's not worry about like the big framework just yet if we

58:00 can just focus on the right types and make sure that all that stuff is kind of sane then you also

58:04 keep the extractions at bay which is i think also convenient especially early on it sounds a little

58:09 bit like an analogy you might be able to draw with databases like you could have like a really strong

58:15 structured Postgres or relational database where the database thing, aka the AI, is in charge of it.

58:22 Or you could be using an ODM or ORM where the code will only accept responses, right? So like,

58:29 for example, you can get away talking to a document database if you have strongly typed

58:34 models. If you're just passing dictionaries, you're going to end up in a bad place real quick

58:38 because something's not going to match, right? So it's kind of like the Pydanic thing is being a

58:42 check for if it's not quite perfect. I don't know. I feel like there's an analogy there.

58:46 There's a bit of an analogy there. The main thing with the LLM is like, oh, how much of the logic do

58:52 you want to control yourself as a guarantee? And how much would you like the LLM to do it? And

58:57 oh, that's always kind of a struggle. Like what is the right glue for that? And it turns out that

59:02 Pydantic is in a very funny, unique position to say, how about we just use types? And that actually

59:07 It just kind of works.

59:08 How ironic is that that that's such a popular thing coming out of a effectively untyped language?

59:15 Yeah.

59:16 Maybe that's why it's so popular.

59:18 I think it partly is.

59:19 I mean, it's my impression that there's a lot of this stuff also happening on the JavaScript

59:23 side of things, if I'm honest.

59:24 Now, for the JavaScript side of things, the argument is, of course, oh, you can also build

59:27 the web UIs a bit easier and the user experience and that whole story.

59:30 But they also do a lot of this validation stuff.

59:32 I forget the library, but there's also--

59:35 I'm learning a little bit of JavaScript now.

59:37 there's actually pydantic things happening in JavaScript land as well.

59:40 Yeah, it's probably jidantic.

59:44 Just-dantic.

59:46 Just-dantic, yeah, j-s-dantic.

59:49 Or TypeScript or whatever.

59:50 But there's also like ORM libraries for JavaScript that are like type E,

59:54 but not like super TypeScript just yet.

59:56 Yeah, there's a lot of interesting analogies from the JavaScript space for us.

59:59 You know, like the whole TypeScript team just rewrote everything in Go,

01:00:04 whereas we're seeing a lot of stuff in Python being modified.

01:00:06 Rust. Rust. Yeah, exactly. So pretty interesting. But let's, let's close this out a little bit with

01:00:12 like running some of our own models. Okay. So clearly there's Anthropic, there's OpenAI,

01:00:18 there's, you know, all the big, huge foundational models, but some of these things we're talking

01:00:22 about, it's like this identity schema validation type of thing. And so on, you know, maybe we could,

01:00:27 as you said, get away with running a smaller model, maybe small enough that we could run it

01:00:30 ourselves on our servers or on my Mac mini here m2 pro 32 gigs ram i have the um the open ai

01:00:39 20 billion parameter model running as like my my default llm for any my code that i write that

01:00:45 needs to talk to an llm so um yeah so we actually have a video on the maroon channel that one but

01:00:51 go check that out later but basically i've recorded that link we'll put in the show notes yeah no so

01:00:56 I kind of find myself exploring all these different open source LLMs like once every two months or so.

01:01:01 And then Marimo has a couple of interesting integrations that makes it really easy to either hook into Olama or,

01:01:06 and that's like the only thing I might change on the course that we made,

01:01:09 there's now also the service called Open Router.

01:01:11 Like I don't know if you've ever seen that one.

01:01:13 Yes, I've heard of it.

01:01:14 Because I was looking at Klein.bot.

01:01:17 Are you familiar with Klein.bot?

01:01:18 Oh, yeah.

01:01:19 Klein is cool.

01:01:19 Klein is really cool.

01:01:20 I've been using that for a bit.

01:01:22 Tell me what Klein is real quick.

01:01:24 So Klein is basically Copilot for VS Code, but the one thing that they do

01:01:29 is they really let you do any model, like just plug it in.

01:01:32 And they are really strict about plan mode versus like run mode, if it makes sense.

01:01:36 So you really got to, and one, the UI is kind of cool.

01:01:39 So like every time that you're like planning or doing whatever,

01:01:42 you kind of see the progress bar of your context.

01:01:45 And then you also see how much money it costs.

01:01:47 So you're like always aware of the fact that like, okay, this thing is costing me bucks now.

01:01:51 And it's just a refreshing take on GitHub Copilot.

01:01:54 And I will say, you can't do sessions in parallel, because it is, I think, at the moment, still stuck to the VS Code

01:02:00 extension ecosystem.

01:02:01 Could be wrong with that.

01:02:02 They just introduced a CLI.

01:02:04 Ah, there you go.

01:02:05 There you go.

01:02:06 I bet you can do it with that.

01:02:07 Yeah, so that's also--

01:02:09 the main thing that drew me in was the whole, oh, they're very strict about plan mode versus Go mode,

01:02:14 which I think is definitely super useful.

01:02:17 And on top of that, you have some nice UI.

01:02:19 Yeah, it's also real similar to Cursor and others.

01:02:21 And I think it's a big selling point.

01:02:23 open source and they don't charge for inference so you just put in your api key and it just just

01:02:28 passes through but you're right they do have like the as it's thinking it's got a little tick tick

01:02:31 tick price of this opera and it's like oh it's up to three cents five you know it's just interesting

01:02:37 to watch it go you also it's kind of like the fibonacci sequence you kind of realize that it's

01:02:41 additive so it's like oh i've spent like half a dollar now oh and the next problem is going to be

01:02:45 half a dollar plus extra the next one's going to be like that extra exactly it makes you manage your

01:02:51 your context size real well.

01:02:53 So anyway, OK, that's a bit of a diversion.

01:02:54 But I think this is a fun tool to shout out.

01:02:57 No, I agree.

01:02:57 So the reason I brought this up is I saw they were number one on Open Router.

01:03:02 I'm like, wait, what's Open Router?

01:03:03 Yeah, so Open Router is basically you have one API key,

01:03:06 and then they will route to any LLM that you like.

01:03:09 And one thing that's really cool about it is that if there's a new open source model

01:03:13 that you see all at Twitter and YouTube rave about,

01:03:15 it'll be on Open Router.

01:03:17 And then you can just give it a spin.

01:03:18 If you have, let's say, stuff in your cache, and you've got a couple of functions that you use for evaluation,

01:03:24 and you've got a setup for that, then it's just changing a single string to give a new LLM model a try.

01:03:29 And they will make sure everything is nicely routed.

01:03:32 And that's like the whole service also, basically.

01:03:34 I think they add like a 5% fee hike or something like that,

01:03:39 but they've got all sorts of GPU providers competing for the lowest price.

01:03:42 So they do all the routing to all the LLM models, but also the open source ones.

01:03:47 And that's the main convenient bit.

01:03:48 Because I might not have-- so it's partially that I don't have the biggest GPU for the biggest models,

01:03:55 but it's really convenient to just do kind of a for loop.

01:03:57 Like, okay, take the 7B model from Qwenn, the 14B model, and just loop over all of them,

01:04:02 and see if there's like a sweet spot as far as like quality is concerned and also model size.

01:04:07 Like those kinds of experiments become a whole lot simpler if you just have one API that has all the models.

01:04:11 And OpenRouter kind of gets you there.

01:04:13 Yeah, okay, very neat.

01:04:14 And it's also why Klein likes using it.

01:04:16 So they have a direct connection with Klein.

01:04:18 Just in the user interface, you can say, well, I want to have this model from Open Router.

01:04:21 Just click, click, click, and it's all sort of ready to go.

01:04:24 But that's their service.

01:04:25 OK, very neat.

01:04:26 So maybe for people who haven't run their own models,

01:04:30 there's a couple of options.

01:04:32 Sounds like you recommend Olama.

01:04:34 I mean, that's the one that I have used.

01:04:35 You've used LM Studio, I think?

01:04:37 I use LM Studio because LM Studio has this UI for discovering and setting them up

01:04:42 and playing and configuring them.

01:04:43 But then you can just say, run in dev mode, and it just runs, and here we go,

01:04:47 an open AI compatible--

01:04:49 - There you go. - API endpoint on my local network.

01:04:51 And so I can just point anything that can do ChatGPT,

01:04:54 I can point it at this, whatever's running there.

01:04:56 And I was running the GPT OSS 20 billion one last time I tried.

01:05:00 - There you go.

01:05:01 Okay.

01:05:01 So Ollama also has like a very similar thing.

01:05:04 My impression is that LM Studio in the end is like pretty similar,

01:05:07 just has a slightly different UI.

01:05:09 It's a bit more of an app, I suppose.

01:05:10 Like Ollama is definitely more of a command line utility.

01:05:13 Does the job though, I would say, but it doesn't have that much to offer in the UI department.

01:05:18 Yeah, I think if I were ever to try to set it up on my server,

01:05:21 I'd probably just do Olama, maybe inside a Docker or something.

01:05:24 I don't know, something like that.

01:05:25 Yeah, not true.

01:05:27 The one sort of mysterious thing with a lot of these services,

01:05:29 like how do they make money?

01:05:31 And I think I read somewhere that Olama is going to start doing

01:05:35 open router kind of stuff as well, so it wouldn't surprise me if they ended up going in that direction.

01:05:40 But I mean, there's all sorts of vendors, and there's another thing I'm also seeing a lot of nowadays

01:05:45 use like big models that generate a data set such that they can take a super tiny model and fine

01:05:49 tune on the data that was generated by the big model and that way like a future maybe that we

01:05:53 might have is that you have a dutch model that's really good for legal maybe we'll have like a

01:05:58 super bespoke lm for that at some point or like a um like an lm that's like really good at css but

01:06:04 only css like that's also a thing that we might have and then those models can be really tiny and

01:06:09 run on your laptop and who knows this is the age we live in um but i certainly can foresee a world

01:06:14 like that where it's like okay i'm gonna work on a project instead of spinning up a general world

01:06:19 knowledge model i'm gonna just let it on demand go okay i have a css question well what framework

01:06:24 are you using i'm using tailwind okay we're gonna spend up the tailwind llm model and we need to ask

01:06:29 it a question yeah we're gonna shut that you know what i mean like really focus but then have some

01:06:33 kind of orchestrating thing that says now i gotta talk to this now let me let me work with that one

01:06:37 but that could be great yeah and it might also be a solution to the whole oh the time cut off is

01:06:41 January to 2025.

01:06:43 And like everything happened after we have no knowledge about.

01:06:46 Like it could also be a way for us to have a system where we just retrain a thing weekly

01:06:50 and it's just a diff on top or something like that.

01:06:52 Right.

01:06:53 Yeah.

01:06:53 It could also happen.

01:06:54 Yeah.

01:06:54 And if it were small enough, then it would be okay.

01:06:56 Yeah.

01:06:57 It would be kind of like SETI at home.

01:06:58 All of our stuff would just be like kind of have the fan going.

01:07:02 The, I mean, it's a bit of a science fiction thing, but you can imagine at some point every

01:07:06 home will have solar panels and then maybe every home will also just have a GPU and maybe

01:07:10 we won't have huge-ass server farms.

01:07:12 It's just that we have a network, and everyone has a GPU, and if my neighbor suddenly

01:07:16 wants to do math, then he can borrow my GPU.

01:07:18 I don't know. We'll see.

01:07:21 It kind of sounds cool, doesn't it?

01:07:22 Yeah. If people like distributed systems,

01:07:26 might as well go for it.

01:07:29 If you want to add to the unpredictability of

01:07:32 the LLM, mix in a distributed grid computing that you don't know about too much either,

01:07:36 and then you'll get some real...

01:07:38 I do want to give one quick shot on this one front, though, because I do think

01:07:41 it's an idea that's worth pursuing.

01:07:43 So in Holland, we've got this one cloud provider called

01:07:46 LeafCloud, and I think they're still around.

01:07:48 But what they do is they install these server farms,

01:07:51 like what we're discussing.

01:07:52 They have a GPU rack at the bottom of an apartment building.

01:07:57 And any excess heat, which is one of the main cost drivers,

01:08:01 that's actually used to preheat the water for the rest

01:08:03 of the apartment.

01:08:04 Yeah.

01:08:05 So on paper, at least, you do have something that's not only

01:08:09 carbon negative, but also more cost effective because the person, like the waste product of the

01:08:13 of the compute is now a thing that the heat that someone is willing to pay for as well. So

01:08:18 on paper, you can also be cheaper if you play your cards right.

01:08:21 I ran across this and I was really inspired by it. I ended up not using LeafCloud for anything,

01:08:27 but it's a cool idea.

01:08:28 So the one downside of that approach is you do have to be close to a data center because

01:08:33 do you really want to store your private data on disk in an apartment building or in a secure

01:08:37 facility. So what they do is they just have glass fiber cables. Like the disc is in the facility,

01:08:42 but the memory and the GPU and the computer is in the apartment building. That's sort of the way

01:08:45 it's like a mounted volume over fiber, sort of like a network volume over fiber. Sure. I think

01:08:51 that's pretty common. A lot of data center setups, it just happens to be spanning an apartment

01:08:56 complex, which is unusual. Oh, it's also only for certain types of compute loads, right? You're

01:09:00 probably not going to do your web app fire that apartment building. It's probably not going to be

01:09:04 productive but if you're doing like big simulation big like deep learning things like that actually

01:09:09 everyone everyone knows you can do machine training machine learning training and it's all good for

01:09:15 serverless you just send the stuff out it just goes finds an apartment complex runs it serverless and

01:09:20 it comes back right now just kidding so yeah it's a pretty neat idea yeah there's um we'll see what

01:09:25 the future brings like when i did data science the joke i always made is like as a data scientist i

01:09:29 I know for sure that the future is really hard to predict.

01:09:35 Maybe the past is easier to predict, and that's what we do in machine learning.

01:09:38 We try to predict the past, and yeah, we'll see what these LLMs do.

01:09:41 But until then, again, also the repeating lesson from the course,

01:09:46 there's some good habits, but in the end, it's not so much the LLM, it's more the stuff around

01:09:50 and how you think about it.

01:09:51 And also, small shout-out to Marimo.

01:09:53 I do think having some UI in the blend as well, such as you can say, oh, there's a text box here,

01:09:58 another text box there.

01:09:59 you can kind of play around making your own little LLM app. Like the thing that I typically do is I

01:10:04 hook up Gemini to my YouTube videos to make like summaries for my blog. I'd like try to find little

01:10:08 excuses to like improve your life by automating something with these LLM tools. And that's a nice

01:10:13 way to get started. Yeah, there's really, I think we all probably have little opportunities to automate

01:10:18 some low stakes thing, but that we spend a lot of time with or hassle or whatever and let an LLM

01:10:23 have at it and write a little Python to make that happen, right?

01:10:25 Yeah, and also expose yourself to your own slop as well.

01:10:30 The nice thing about building your own apps is also because you then suddenly start to realize when it's also still good to be a human.

01:10:35 Like, I'm not going to have an LLM automate messages to my wife and my kid.

01:10:40 That's just not going to happen because I've seen what these LLMs can output.

01:10:43 Like, that's going to remain on the human side of the spectrum.

01:10:46 Exactly.

01:10:47 But making a summary for my blog on a video, I mean, that feels in the ballpark.

01:10:51 100%.

01:10:52 All right.

01:10:52 Final word for people interested in this topic they want to build.

01:10:56 I want to plug in LLMs as part of that building block.

01:10:59 What do you say?

01:11:00 Open Router is nice and flexible.

01:11:02 LLM by Simon Willison is great.

01:11:04 Disk cache is a thing that you do want to disk cache.

01:11:09 Maybe a final thing that I do maybe want to...

01:11:12 It's also fine if you just want to learn Python as well.

01:11:15 There's some of this talk of people saying, oh, we never have to...

01:11:19 Oh, actually, people from Peru are tuning in.

01:11:21 Peru is amazing.

01:11:22 from seeing people sort of replying the conversation.

01:11:24 Yeah, absolutely.

01:11:25 Oh, sweet.

01:11:26 Best food of my life, Peru.

01:11:28 Also, if you're looking for great excuses to learn Python,

01:11:31 there's plenty.

01:11:31 If you're looking for great excuses to go to Peru, food.

01:11:35 But also, I think in the end, you are going to be better at using LLMs

01:11:41 if you can still do things yourself.

01:11:42 One thing I've started noticing is that it's a bit embarrassing, but like,

01:11:45 oh, what's the CSS property of that thing?

01:11:47 I used to know that.

01:11:48 But nowadays, I just, oh, am I seriously going to ask this, the ChatGPT?

01:11:51 This should be in my memory for Pete's sakes.

01:11:53 Yes.

01:11:54 And sometimes it's like, I'm going to have a whole conversation with ChatGPT

01:11:57 so I can change four characters in a style sheet.

01:11:59 What is this nonsense?

01:12:00 So part of me is also thinking like, definitely expose yourself to LLMs.

01:12:04 And if that inspires you, that's great.

01:12:06 But also try to not overly rely on it either.

01:12:10 It's like I'm building flashcard apps for myself.

01:12:12 So I'm still kind of in the loop, if that makes sense.

01:12:15 So you've got to be a guardian of your own mind.

01:12:19 And yes, so I can see it so easily.

01:12:21 that you just become a machine that just says next okay yes continue and then in six months you're

01:12:28 like gosh i can't program anything i just got to ask the chat and it's it's a problem yeah it's

01:12:33 it's something well and also um and that kind of gets back to the point i made earlier um you want

01:12:38 to be very mindful that if you if you have better tools you should also have better ideas and if

01:12:43 that's not the case then something is wrong because that's then you get into the self-learned

01:12:48 helplessness territory. And that's the one thing I do want to kind of guard against.

01:12:52 Andre Carpathy gave a great interview a little while ago where he mentioned the

01:12:56 worst outcome of humanity. Did you see the movie WALL-E? Remember seeing that?

01:13:00 Oh my gosh. That movie really made me sad.

01:13:04 It's a good movie though. It's a fantastic movie, but as an adult,

01:13:08 it's got a hard undertone to it. Oh my gosh.

01:13:11 And especially now, because for people who never watched the movie, there's a scene in the movie where you see what humanity

01:13:16 comes to and it's basically every everything has to be convenient so people are overweight

01:13:20 they're in these like bikes that move them around they're always able to watch television drink a

01:13:24 milkshake and eat a pizza like every convenience is sort of taken care of but you also just see

01:13:29 that they're not happy and they can't even walk out like they can't they're like just they're so uh

01:13:35 just dependent on the thing it's bad exactly so that's a sweet show it's a very sweet show i

01:13:39 recommend it yeah it's it's a sweet movie but it's like if at some point you notice like hey

01:13:44 learning feels hard, then maybe that's fine because maybe it's just like exercising.

01:13:49 It's okay if that's a bit painful.

01:13:51 In fact, if it's not painful, then maybe just participating in entertainment instead as well.

01:13:54 So that's the thing.

01:13:56 These are thoughts that I'm sort of battling with.

01:13:57 I'm also sort of trying to be really critical of like, okay, I work at Marimo now.

01:14:01 I've accessed all these LLMs and stuff.

01:14:03 And can I build more awesome software that I wouldn't have before?

01:14:06 One thing I made a while ago is I added gamepad support to Marimo Notebooks.

01:14:10 So you can interact with Python with the gamepad and stuff.

01:14:12 So this is just a experiment.

01:14:14 But I would be really disappointed if I'm the only person making packages that I've never made before that are definitely trying to reach new heights.

01:14:23 It does feel a bit lonely because it does feel like more people should do it.

01:14:26 That's like the one sort of hint I want to give to people.

01:14:29 Try to inspire yourself a bit more and do more inspirational stuff so more cool things happen on my timeline.

01:14:34 There's less AI sloth.

01:14:35 There's really cool software being built, people learning and all.

01:14:38 Hear, hear.

01:14:38 I appreciate that.

01:14:39 All right.

01:14:40 Well, Vincent, thanks for being back, sharing all these techniques.

01:14:44 And yeah, I hope people go out there and build using AIs, not just with AIs.

01:14:48 Definitely.

01:14:49 Y'all have a good one.

01:14:50 Yeah, see you later.

01:14:50 Bye.

01:14:51 This has been another episode of Talk Python To Me.

01:14:54 Thank you to our sponsors.

01:14:55 Be sure to check out what they're offering.

01:14:57 It really helps support the show.

01:14:59 Take some stress out of your life.

01:15:00 Get notified immediately about errors and performance issues in your web

01:15:04 or mobile applications with Sentry.

01:15:06 Just visit talkpython.fm/sentry and get started for free.

01:15:11 And be sure to use the promo code Talk Python, all one word.

01:15:15 And it's brought to you by Nordstellar.

01:15:18 Nordstellar is a threat exposure management platform from the Nord security family,

01:15:22 the folks behind NordVPN that combines dark web intelligence,

01:15:26 session hijacking prevention, brand and domain abuse detection,

01:15:31 and external attack surface management.

01:15:33 Learn more and get started keeping your team safe at talkpython.fm/Nordstellar.

01:15:38 If you or your team needs to learn Python, we have over 270 hours of beginner and advanced courses on topics ranging from complete beginners to async code, Flask, Django, HTML, and even LLMs.

01:15:52 Best of all, there's not a subscription in sight. Browse the catalog at talkpython.fm.

01:15:57 Be sure to subscribe to the show. Open your favorite podcast player app. Search for Python. We should be right at the top.

01:16:03 If you enjoy the geeky rap theme song, you can download the full track.

01:16:07 The link is your podcast player show notes.

01:16:09 This is your host, Michael Kennedy.

01:16:11 Thank you so much for listening.

01:16:12 I really appreciate it.

01:16:13 Now get out there and write some Python code.

01:16:25 Talk Python To Me, and we ready to roll Upgrading the code, no fear of getting old

01:16:36 We tapped into that modern vibe, overcame each storm

01:16:40 Talk Python To Me, I-Sync is the norm you