WEBVTT 00:00:00.020 --> 00:00:03.420 Your cloud SSD is sitting there, bored, and it would like a job. 00:00:03.880 --> 00:00:08.320 Today, we're putting into work with DiskCache, a simple, practical cache built on SQLite 00:00:08.320 --> 00:00:12.900 that can speed things up without spinning up Redis or other extra servers. 00:00:13.540 --> 00:00:17.080 Once you start to see what it can do, a universe of possibilities opens up. 00:00:17.720 --> 00:00:20.640 We're joined by Vincent Warmerdom to dive into DiskCache. 00:00:21.120 --> 00:00:25.880 This is Talk Python To Me, episode 534, recorded December 19th, 2025. 00:00:27.100 --> 00:00:29.700 Talk Python To Me, yeah, we ready to roll. 00:00:29.780 --> 00:00:32.540 Upgrading the code, no fear of getting old 00:00:32.680 --> 00:00:35.160 Async in the air, new frameworks in sight 00:00:35.360 --> 00:00:37.200 Geeky rap on deck, Quart crew 00:00:37.420 --> 00:00:38.240 It's time to unite 00:00:38.500 --> 00:00:41.320 We started in Pyramid, cruising old school lanes 00:00:41.560 --> 00:00:43.200 Had that stable base, yeah sir 00:00:43.240 --> 00:00:47.580 Welcome to Talk Python To Me, the number one Python podcast for developers and data scientists. 00:00:48.100 --> 00:00:49.500 This is your host, Michael Kennedy. 00:00:49.880 --> 00:00:53.420 I'm a PSF fellow who's been coding for over 25 years. 00:00:54.060 --> 00:00:55.180 Let's connect on social media. 00:00:55.500 --> 00:00:58.580 You'll find me and Talk Python on Mastodon, Bluesky, and X. 00:00:58.920 --> 00:01:00.760 The social links are all in your show notes. 00:01:01.540 --> 00:01:05.040 You can find over 10 years of past episodes at talkpython.fm. 00:01:05.320 --> 00:01:08.420 And if you want to be part of the show, you can join our recording live streams. 00:01:08.820 --> 00:01:09.260 That's right. 00:01:09.540 --> 00:01:12.700 We live stream the raw uncut version of each episode on YouTube. 00:01:13.320 --> 00:01:17.720 Just visit talkpython.fm/youtube to see the schedule of upcoming events. 00:01:17.910 --> 00:01:21.600 Be sure to subscribe there and press the bell so you'll get notified anytime we're recording. 00:01:22.020 --> 00:01:22.780 Vincent, hello. 00:01:23.380 --> 00:01:24.700 Michael, Michael, we're back. 00:01:25.540 --> 00:01:25.760 Awesome. 00:01:26.130 --> 00:01:27.160 Awesome to be back with you. 00:01:27.500 --> 00:01:32.440 Yeah, this is almost the sequel to the last time you were on the show. 00:01:32.780 --> 00:01:33.960 So it's going to be fun. 00:01:34.240 --> 00:01:36.480 Yeah, so sequel in this case, not the query language, 00:01:36.690 --> 00:01:38.020 like an actual sequel of events. 00:01:38.820 --> 00:01:39.040 Yes. 00:01:39.240 --> 00:01:41.440 Yeah, you can correct me if I'm wrong, 00:01:41.470 --> 00:01:45.120 but I think what happened is you had me on a podcast a while ago 00:01:45.620 --> 00:01:46.760 to talk about a course that I made, 00:01:46.760 --> 00:01:49.780 and a big chunk of the course that we were very enthusiastic about 00:01:49.860 --> 00:01:51.100 was about this tool called DiskCache. 00:01:51.440 --> 00:01:52.780 And then we kind of came to the conclusion, 00:01:52.850 --> 00:01:54.180 well, we had to cap it off. 00:01:54.720 --> 00:01:56.980 Maybe it's fun to do an episode on just DiskCache. 00:01:57.400 --> 00:01:59.100 since we're both pretty huge fans of it. 00:01:59.440 --> 00:02:00.680 I think that's how we got here. 00:02:00.860 --> 00:02:02.540 I think that is how we got here as well. 00:02:02.720 --> 00:02:04.640 And we're going to dive into this. 00:02:05.820 --> 00:02:08.780 Honestly, it's a pretty simple library called DiskCache, 00:02:09.140 --> 00:02:11.520 but what it unlocks is really, really sweet. 00:02:11.880 --> 00:02:14.220 And I'm going to talk about a lot of different angles. 00:02:14.630 --> 00:02:18.040 And now, even though it's just been not that long since you were on the show, 00:02:18.160 --> 00:02:20.340 maybe just give us a quick intro of who you are. 00:02:20.680 --> 00:02:21.400 Hi, my name is Vincent. 00:02:21.680 --> 00:02:24.980 I've done a bunch of data machine learning stuff, mainly in the past. 00:02:25.400 --> 00:02:26.660 That's sort of what a lot of people know me from. 00:02:27.220 --> 00:02:28.940 These days, though, I work for a company called Marimo. 00:02:29.310 --> 00:02:30.160 You might have heard from us. 00:02:30.170 --> 00:02:31.800 We make very modern Python notebooks. 00:02:32.220 --> 00:02:33.680 We took some lessons from Jupyter, 00:02:33.790 --> 00:02:35.100 and we take a new spin of it. 00:02:35.590 --> 00:02:36.600 So that's my day to day. 00:02:37.720 --> 00:02:38.780 But I still like to write notebooks 00:02:39.000 --> 00:02:41.920 and do kind of fun little benchmarks and also stuff 00:02:42.000 --> 00:02:42.520 with LLMs. 00:02:42.810 --> 00:02:45.560 And I've just noticed that for a lot of that work, boy, 00:02:45.800 --> 00:02:46.740 disk cache is amazing. 00:02:47.380 --> 00:02:48.480 And I also use it for web stuff. 00:02:48.650 --> 00:02:49.980 And I think that's also what your use case 00:02:50.060 --> 00:02:50.880 is a little bit more of. 00:02:51.630 --> 00:02:53.760 But yeah, in notebook land, you also 00:02:54.000 --> 00:02:56.599 like to have a very good caching mechanism 00:02:56.620 --> 00:03:00.800 And on the Marimo side of things, we are also working on different caching mechanisms, which I might talk about in a bit. 00:03:01.180 --> 00:03:06.440 But just for me, the bread and butter, the thing I've used for years at this point is disk cache whenever it comes to that territory. 00:03:06.780 --> 00:03:07.480 Yeah, it's funny. 00:03:07.720 --> 00:03:13.040 This was recommended to me for Python Bytes as a news item over there quite a while ago, like years ago. 00:03:13.290 --> 00:03:14.500 And I'm like, oh, that's pretty interesting. 00:03:15.080 --> 00:03:20.760 And then I saw you using it in the LLM Building Blocks course, and it just unlocked for me. 00:03:20.770 --> 00:03:21.620 Like, oh, my. 00:03:22.000 --> 00:03:23.480 Oh, this is something else. 00:03:24.180 --> 00:03:27.420 And so since then, I've been doing a bunch with it, and I'm a big fan. 00:03:27.870 --> 00:03:35.560 I've been on this, like trying to avoid complexity, but still getting really cool responses, performance, et cetera, out of your apps. 00:03:35.610 --> 00:03:47.500 And I think this is a really nice way to add multi-process, super fast caching to your app without involving more servers and more stuff that's got to get connected and keep running and so on. 00:03:47.880 --> 00:03:53.520 But before we get into the details of that, maybe let's just talk about caching in general. 00:03:53.600 --> 00:03:55.140 Like what types of caching is there? 00:03:55.900 --> 00:03:57.580 You know, I sort of give a little precursor there. 00:03:57.720 --> 00:03:58.560 But yeah, dive into it. 00:03:58.700 --> 00:04:01.000 So like in the course, the main example 00:04:01.160 --> 00:04:03.280 I remember talking about was the one-- 00:04:03.600 --> 00:04:05.520 you've got this LLM, and you want to do some benchmarks. 00:04:05.800 --> 00:04:07.560 And it might be the case that, I don't know, 00:04:07.620 --> 00:04:09.280 using an LLM for, let's say, classification, 00:04:09.800 --> 00:04:11.860 like some text goes in, we got to know whether or not 00:04:12.180 --> 00:04:14.100 it's about a certain topic, yes, no, or something like that. 00:04:14.340 --> 00:04:17.519 Then it would be really great if, suppose, the same text came 00:04:17.700 --> 00:04:21.660 by for whatever reason, that we don't run the query on the LLM 00:04:21.680 --> 00:04:23.800 Again, it's like wasted compute, wasted money. 00:04:23.910 --> 00:04:26.940 So it'd be kind of nice if the same text goes in that we then say, 00:04:27.100 --> 00:04:28.940 oh, we know what the answer to that thing is already. 00:04:29.010 --> 00:04:31.020 We cached it, so here you can go back. 00:04:31.520 --> 00:04:34.700 And that's the case when you're dealing with heavy compute ML systems. 00:04:35.240 --> 00:04:36.980 But there's a similar situation that you might have, I guess, 00:04:37.180 --> 00:04:38.800 with expensive SQL queries, 00:04:39.100 --> 00:04:41.360 or you want to reduce the load on a database somewhere. 00:04:41.490 --> 00:04:43.440 Then having some sort of a caching layer that's able to say, 00:04:43.730 --> 00:04:46.100 oh, you're querying for something, but I already know what it is. 00:04:47.020 --> 00:04:48.560 Boom, we can send it back. 00:04:49.040 --> 00:04:51.120 I think the classical thing you would do in Python 00:04:51.120 --> 00:04:53.580 is you have this decorator in functools, I think, right? 00:04:53.820 --> 00:04:57.020 The LRU_cache. 00:04:57.440 --> 00:04:57.860 Yeah, exactly. 00:04:58.300 --> 00:04:58.400 Yeah. 00:04:58.780 --> 00:04:59.840 That's a hell of a world to that. 00:04:59.880 --> 00:05:02.200 But the downside of that thing is that it's all in memory. 00:05:02.420 --> 00:05:04.020 So if you were to reboot your Python process, 00:05:04.220 --> 00:05:05.400 you lose all that caching. 00:05:05.880 --> 00:05:07.420 So that's why people historically, I think, 00:05:07.520 --> 00:05:08.560 resorted to-- 00:05:08.660 --> 00:05:12.480 I think Redis, I think, is the most well-known caching tool. 00:05:12.780 --> 00:05:13.620 It's the one I've always used. 00:05:13.700 --> 00:05:14.640 There's Memcache, I think. 00:05:14.760 --> 00:05:15.260 There's other tools. 00:05:15.360 --> 00:05:17.020 You could use Postgres for some of this stuff as well. 00:05:18.360 --> 00:05:21.400 But recently, especially because disks are just getting quicker 00:05:21.450 --> 00:05:23.520 and quicker, people have been looking at SQLite 00:05:23.950 --> 00:05:25.060 for this sort of a thing as well. 00:05:25.180 --> 00:05:27.220 So that's, I think, the quickest summary 00:05:27.400 --> 00:05:30.920 and also sort of the entryway to how I got started with disk cache. 00:05:31.380 --> 00:05:34.680 Yeah, and so for this example that you highlight in the LLM 00:05:34.740 --> 00:05:38.040 Building Blocks course, it's not a conversation. 00:05:38.400 --> 00:05:41.240 It's like a one-shot situation, right? 00:05:41.340 --> 00:05:43.740 You come up-- you say, I have some code or some documents, 00:05:43.830 --> 00:05:45.580 and I have almost like an API. 00:05:45.820 --> 00:05:51.500 I'm going to send that off to the LLM and ask it, tell me X, Y, and Z about it. 00:05:51.840 --> 00:05:56.280 And sure, it's got some kind of temperature and it won't always give an exactly the same answer, 00:05:56.700 --> 00:05:59.780 but you're willing to, you know, you're willing to accept an answer. 00:06:00.220 --> 00:06:05.420 And at that point, like why ask it again and again and again, which it might take seconds, 00:06:05.800 --> 00:06:06.720 it might cost money. 00:06:06.980 --> 00:06:12.860 Whereas if you just remember through caching somehow, you remember it, it's like, boom, instant. 00:06:13.300 --> 00:06:16.600 Yeah, and it tends to come up a lot in when you're doing benchmarks, for example. 00:06:16.760 --> 00:06:20.640 So you have this for loop, you want to go over your entire data set, try all these different approaches. 00:06:21.220 --> 00:06:23.480 And if you've got a new approach, then you want that to run, of course. 00:06:23.840 --> 00:06:28.800 But if you accidentally trigger an old approach, then you don't want to incur the cost of like going through all those different LLMs. 00:06:29.199 --> 00:06:33.260 I should say, like, even if you just forget about LLMs, let's just say machine learning in general. 00:06:33.740 --> 00:06:36.440 Let's say there's some sort of image classification thing you're using in the cloud. 00:06:36.920 --> 00:06:39.100 There also, you would say, like, file name goes in. 00:06:39.240 --> 00:06:43.240 that's an image and if the same file name goes in we don't want the expensive compute cost to happen 00:06:43.370 --> 00:06:48.440 either so it's definitely more general than llms but llms do feel like it's the zeitgeisty thing to 00:06:48.680 --> 00:06:53.880 worry about yeah i think for two reasons one because they're just the topic du jour and two 00:06:54.040 --> 00:06:59.300 because they're they're i think a part of computing that most people experience that is way slower than 00:06:59.340 --> 00:07:05.500 they're used to yeah well and especially if you're you know if i suppose that you you have a an 00:07:05.520 --> 00:07:09.000 an attic somewhere and you're a dad and you want to do home lab stuff and you're playing with all 00:07:09.030 --> 00:07:14.300 these open source LLM models, then you also learn that, yeah, they're fun to play with, but they also 00:07:14.370 --> 00:07:18.400 take a lot of time to compute things. So then immediately you get the motivation to do it the 00:07:18.480 --> 00:07:25.320 right way. Yeah, I built a couple of little utilities that talk to a local LLM. I think it's 00:07:25.500 --> 00:07:30.840 the OpenAI OpenWeights one, that 20 billion parameter one I have running on my Mac Mini, 00:07:31.210 --> 00:07:35.480 and it's pretty good, a little bit slow, but, you know, it's fine for what it's being used for. And 00:07:35.500 --> 00:07:38.100 put-- use your disk cache technique on it. 00:07:39.100 --> 00:07:41.240 And if I ask it the same question again, it's like, boom. 00:07:41.580 --> 00:07:43.300 You don't need to wait 10 seconds. 00:07:43.680 --> 00:07:43.940 Here's the answer. 00:07:43.960 --> 00:07:44.020 Yeah. 00:07:44.620 --> 00:07:46.860 So that-- and I guess like-- but I guess from your perspective, 00:07:46.900 --> 00:07:49.000 I think your main entry point to this domain 00:07:49.220 --> 00:07:51.540 was a little bit more from the web dev perspective, right? 00:07:51.580 --> 00:07:53.800 Like that's-- and I suppose you're using it a lot 00:07:53.960 --> 00:07:56.680 for preventing expensive queries to go to Postgres, 00:07:57.100 --> 00:07:58.960 or I don't exactly know your backend. 00:07:59.220 --> 00:08:02.200 You know how-- you won't believe how optimized my website is. 00:08:02.400 --> 00:08:05.080 There's not a single query that goes to Postgres, 00:08:05.380 --> 00:08:06.200 because they go to MongoDB. 00:08:06.290 --> 00:08:06.660 I'm just kidding. 00:08:06.800 --> 00:08:07.100 There you go. 00:08:07.920 --> 00:08:10.000 No, but your point is totally valid. 00:08:10.140 --> 00:08:11.260 Go into the database, right? 00:08:11.640 --> 00:08:15.080 Now, I don't actually cache that many requests. 00:08:15.330 --> 00:08:17.260 I don't avoid that many requests going to the database. 00:08:17.430 --> 00:08:19.500 They're really quite quick, and so I'm OK with that. 00:08:19.800 --> 00:08:22.680 But when you think about a feature-rich database, 00:08:23.480 --> 00:08:26.420 feature-rich web app, there's just tons of these little edge 00:08:26.600 --> 00:08:28.560 cases you're like, oh, got to do that thing. 00:08:28.710 --> 00:08:29.880 And it's not a big deal, but we've 00:08:29.880 --> 00:08:31.480 got to do it 500 times in a request. 00:08:31.700 --> 00:08:33.140 Then it is kind of a thing. 00:08:34.260 --> 00:08:34.919 So let me give you an example. 00:08:35.400 --> 00:08:35.960 I'll give you some examples. 00:08:36.340 --> 00:08:40.140 So for example, the good portions of the show notes 00:08:40.780 --> 00:08:43.360 on talkpython.fm are in Markdown. 00:08:43.719 --> 00:08:44.840 I don't want to show people Markdown. 00:08:44.950 --> 00:08:46.840 I want to show them HTML, right? 00:08:47.540 --> 00:08:49.900 So when a request comes in, it'll 00:08:50.100 --> 00:08:53.400 say any fragment of HTML that needs 00:08:53.400 --> 00:08:56.620 to be turned into Markdown instead of just going, oh, 00:08:56.670 --> 00:08:57.700 let me process that. 00:08:57.960 --> 00:09:00.340 It just goes, all right, what is the hash of this 00:09:00.780 --> 00:09:02.680 or some other indicator of the content? 00:09:03.200 --> 00:09:06.400 And then I've already computed that and stored it in disk cache. 00:09:06.660 --> 00:09:08.340 So here's the HTML result. 00:09:08.780 --> 00:09:13.020 Another example is there's a little YouTube icon on each page. 00:09:13.300 --> 00:09:17.480 And that's actually in the show notes, but then the website parses the YouTube ID out 00:09:17.500 --> 00:09:22.200 and then embeds it with an, like, there's a bunch of stuff going on there to keep YouTube 00:09:22.480 --> 00:09:24.540 out of spying on my visitors. 00:09:25.040 --> 00:09:27.480 But stuff happens, YouTube ID is used. 00:09:27.740 --> 00:09:29.060 That could be parsed every time. 00:09:29.420 --> 00:09:32.540 Or I can just say this episode has this YouTube ID. 00:09:33.080 --> 00:09:35.380 That information goes into a cache, right? 00:09:35.410 --> 00:09:38.680 And because it's a disk cache sort of scenario, 00:09:38.950 --> 00:09:41.260 like a file-based one, not an LRU cache. 00:09:42.160 --> 00:09:43.960 It doesn't change the memory footprint 00:09:44.520 --> 00:09:45.780 and it's shared across processes. 00:09:46.040 --> 00:09:47.000 So in like the web world, 00:09:47.120 --> 00:09:48.600 it's really common to have a web garden 00:09:48.650 --> 00:09:50.820 where you've got like two or four processes 00:09:51.260 --> 00:09:53.240 all being like round robin to 00:09:53.780 --> 00:09:56.220 from some web server manager thing, right? 00:09:56.440 --> 00:09:58.700 If you don't somehow out of process that, 00:09:58.860 --> 00:10:02.540 either Redis or SQLite or database or something, 00:10:03.160 --> 00:10:05.720 then all of those things are recreating that, right? 00:10:05.780 --> 00:10:06.940 They can't reuse that, right? 00:10:07.040 --> 00:10:08.600 So there's a lot of interesting components there. 00:10:09.000 --> 00:10:10.420 And I suppose your web deployment, 00:10:10.450 --> 00:10:11.620 you have like a big VM, I suppose, 00:10:11.960 --> 00:10:13.920 and then there's like multiple Docker containers running, 00:10:14.390 --> 00:10:16.760 but they do all have access to the same volume, 00:10:16.830 --> 00:10:18.160 and that's how you access SQLite. 00:10:18.520 --> 00:10:20.120 Bingo, yeah, exactly, exactly. 00:10:21.120 --> 00:10:22.180 And how am I doing? 00:10:22.700 --> 00:10:26.260 Yeah, so what I have done is in the Docker Compose file, 00:10:26.570 --> 00:10:27.720 I have an external, 00:10:28.220 --> 00:10:29.240 This is also important for Docker. 00:10:29.400 --> 00:10:34.580 So I have an external folder on a big hard drive in the big VM that says, here's where 00:10:34.720 --> 00:10:35.520 all the caches go. 00:10:36.000 --> 00:10:39.800 And then depending on which app, it'll pick like a sub directory it can go look at or 00:10:40.000 --> 00:10:40.920 whatever that it's using. 00:10:41.380 --> 00:10:46.640 And so that way, even if I do a complete rebuild of the Docker image, it still retains 00:10:48.120 --> 00:10:51.500 its cache from version to version and all that kind of business. 00:10:51.560 --> 00:10:55.020 You could do that with a persistent VM as well, volume as well. 00:10:55.160 --> 00:10:57.420 But I've just decided-- 00:10:57.420 --> 00:10:58.640 you can go and inspect it a little easier 00:10:58.860 --> 00:11:00.560 and see how big the cache is and stuff like that. 00:11:00.899 --> 00:11:03.060 OK, so we're going to get into the weeds of how 00:11:03.280 --> 00:11:04.420 disk cache works exactly. 00:11:04.720 --> 00:11:06.780 But I'm triggered here because it sounds like you've done 00:11:06.900 --> 00:11:07.420 something clever there. 00:11:07.860 --> 00:11:09.260 Because what you can do in disk cache 00:11:09.480 --> 00:11:11.820 is you can say, look, here's a file that's SQLite. 00:11:11.860 --> 00:11:12.840 And then it behaves like a dictionary, 00:11:13.440 --> 00:11:14.400 but it's persisted on disk. 00:11:14.700 --> 00:11:16.560 But what I just heard you say is that you've got multiple caches. 00:11:16.820 --> 00:11:19.740 So am I right to hear that, oh, for some things that 00:11:19.760 --> 00:11:21.320 need to be cached, let's say the YouTube things, 00:11:21.520 --> 00:11:22.380 that's a separate file. 00:11:22.760 --> 00:11:24.479 And then all the markdown stuff, that's also 00:11:24.500 --> 00:11:27.520 a separate file, and therefore if connections need to be made to either, 00:11:27.700 --> 00:11:28.940 it's also kind of nicely split. 00:11:29.040 --> 00:11:29.920 Is that also the design there? 00:11:30.180 --> 00:11:30.820 Yeah, that is. 00:11:30.870 --> 00:11:33.860 And actually, before, like, we're going to dive into all the details of how it works, 00:11:33.890 --> 00:11:36.320 but I'll just go, I'm just to give people a little glimpse. 00:11:36.390 --> 00:11:38.860 I'll go ahead and show, I've got this whole admin backend here. 00:11:39.170 --> 00:11:41.600 And I've got different caches for different purposes. 00:11:42.050 --> 00:11:45.160 Because they're just SQLite files, you can either say, give me the same one, 00:11:45.190 --> 00:11:47.280 or you can say, this one is named something else, 00:11:47.460 --> 00:11:49.780 and it has a different file name or different folder or whatever. 00:11:50.280 --> 00:11:53.939 Right, so I've got one that stores things like that YouTube ID I talked about 00:11:53.960 --> 00:11:58.500 any markdown, any fragment of markdown anywhere in the web app that it needs to say that needs 00:11:58.540 --> 00:11:59.700 to go to HTML, like just. 00:12:00.400 --> 00:12:03.540 Yeah, and it's like 8,000 items in that thing. 00:12:03.640 --> 00:12:03.880 Yeah. 00:12:04.020 --> 00:12:07.960 In this one, there's 8,970 items, which is nine megs, right? 00:12:08.080 --> 00:12:09.720 I mean, it's not huge, but it's not too bad. 00:12:10.020 --> 00:12:14.080 And you can actually even see where it thinks it lives, but that's not really where it lives 00:12:14.200 --> 00:12:17.000 because there's, you know, the volume redirects and stuff. 00:12:17.560 --> 00:12:22.440 But I've also got stuff for directly about the episodes that it needs to pull back. 00:12:22.660 --> 00:12:25.160 And then I do a lot of HTTP caching. 00:12:25.720 --> 00:12:30.080 And one of the things that I think is really wrong with web development is people say, 00:12:30.220 --> 00:12:33.780 well, that's like a stale image or that's a stale CSS file or JavaScript, you know, 00:12:33.800 --> 00:12:34.500 all that kind of stuff. 00:12:34.920 --> 00:12:41.660 So if you just do like super minor tricks and just put some kind of hash ID on the end 00:12:41.680 --> 00:12:47.140 of your content, it will, and you teach your CDN or whatever, that that's a different file 00:12:47.220 --> 00:12:52.300 if it varies by query string, then you never, ever have to worry about stale content ever. 00:12:52.700 --> 00:12:56.900 right but computing that can be expensive especially for remote stuff like if it's it's on a different 00:12:57.020 --> 00:13:01.460 it's like a s3 thing but you still want to have it do that so i have a special cache for that and 00:13:01.460 --> 00:13:06.520 that takes that's like pretty complicated to build up because it's got to do like almost 700 web 00:13:06.700 --> 00:13:10.540 requests to figure out what those are but once they're done it's blazing fast you don't have to 00:13:10.540 --> 00:13:14.640 do it again right unless it changes then it doesn't change much and so on so there's that's the way 00:13:14.700 --> 00:13:19.599 that i'm sort of using and appreciating disk cache yeah it works well in your setup because you've 00:13:19.620 --> 00:13:21.540 gone for the VM route. I mean, if you go 00:13:21.620 --> 00:13:23.480 for something like Fly.io or maybe even 00:13:24.120 --> 00:13:25.600 DigitalOcean has like a really, I think 00:13:25.620 --> 00:13:27.560 it's a nice like app service, but that 00:13:27.640 --> 00:13:29.500 all revolves around Docker containers that like 00:13:29.590 --> 00:13:31.460 spin up horizontally. And I 00:13:31.640 --> 00:13:33.520 don't think those containers can 00:13:33.550 --> 00:13:35.400 be configured in such a way they share the volume. 00:13:36.220 --> 00:13:37.600 So in that sense, you could still use 00:13:38.240 --> 00:13:39.400 disk cache, but then 00:13:40.080 --> 00:13:41.560 each individual instance of the 00:13:41.620 --> 00:13:43.560 Docker container would have its own cache, which still could 00:13:43.760 --> 00:13:44.360 work out. 00:13:45.520 --> 00:13:47.099 Not going to be as well 00:13:47.120 --> 00:13:50.400 well functional. It's going to be better with your setup, though. 00:13:50.620 --> 00:13:55.040 Yeah, absolutely. I agree, though. You could still do it. Or you could go, I'll take the 00:13:55.220 --> 00:13:59.080 zen of what Vincent and Michael are saying today, and I'll apply that to Postgres, or 00:13:59.080 --> 00:14:02.660 I'll apply that to whatever data. You could pull this off in a database. 00:14:03.190 --> 00:14:07.100 You would just have to do more work. Yeah. I mean, I've had a couple of, I think 00:14:07.100 --> 00:14:11.080 it was like a Django conference talk I saw a while ago. They were also raving about 00:14:11.280 --> 00:14:15.260 disk cache. But the merits of disk cache do depend a little bit on your 00:14:15.220 --> 00:14:15.740 deployment, though. 00:14:15.860 --> 00:14:17.260 That is, I think, one observation. 00:14:17.460 --> 00:14:18.700 Like in your setup, I can definitely imagine it. 00:14:18.830 --> 00:14:18.900 Interesting. 00:14:19.420 --> 00:14:19.560 Yeah. 00:14:19.780 --> 00:14:20.000 Yeah. 00:14:20.520 --> 00:14:22.600 Well, I don't even think we properly introduced this thing 00:14:22.740 --> 00:14:22.980 yet, so. 00:14:23.500 --> 00:14:24.280 But let's maybe go there. 00:14:24.390 --> 00:14:24.460 Yeah. 00:14:24.939 --> 00:14:25.420 Let's start there. 00:14:25.520 --> 00:14:25.900 Let's start there. 00:14:26.280 --> 00:14:26.620 It's time. 00:14:26.840 --> 00:14:26.980 OK. 00:14:27.100 --> 00:14:27.440 It's time. 00:14:27.740 --> 00:14:27.840 Yeah. 00:14:29.060 --> 00:14:31.300 I guess the simplest way I usually describe it, 00:14:31.420 --> 00:14:32.660 it really behaves like a dictionary, 00:14:33.340 --> 00:14:35.020 except you persist a disk and under the hood 00:14:35.100 --> 00:14:35.920 is using SQLite. 00:14:36.050 --> 00:14:38.020 I think that's the-- it doesn't cover everything, 00:14:38.240 --> 00:14:40.340 but you get quite close, if that's the way it is. 00:14:40.540 --> 00:14:42.160 I think there might be-- 00:14:42.160 --> 00:14:43.980 you know, I keep harping on this on the show, 00:14:44.100 --> 00:14:45.820 but there are so many people that are new to Python 00:14:45.920 --> 00:14:46.800 and programming these days. 00:14:47.780 --> 00:14:49.060 Many, many of them, almost half of them. 00:14:49.620 --> 00:14:51.420 I think it's worth pointing out, just like, what is SQLite? 00:14:51.660 --> 00:14:53.820 Like, why is it different than any other database? 00:14:54.320 --> 00:14:56.580 Like, why have I been using the word database or SQLite 00:14:56.660 --> 00:14:57.860 when SQLite is a database, right? 00:14:57.960 --> 00:14:58.220 That's weird. 00:14:58.500 --> 00:15:01.660 - So, I never really took a good database, of course. 00:15:01.780 --> 00:15:03.800 I might be ruining the formalism of it. 00:15:04.240 --> 00:15:06.140 But the main, like, for me at least, 00:15:06.240 --> 00:15:07.840 the way I like to think about it is Postgres, 00:15:08.080 --> 00:15:09.420 that's a thing I can run on a VM, 00:15:09.940 --> 00:15:13.039 and then other Docker containers can connect to it 00:15:13.060 --> 00:15:14.360 because it's running out of process. 00:15:14.660 --> 00:15:16.840 There's some other process that has the database somewhere, 00:15:17.290 --> 00:15:18.200 and I can connect to it. 00:15:18.470 --> 00:15:20.260 And I think the main thing that makes SQLite different 00:15:20.580 --> 00:15:22.400 is that, no, you got to run it on the same machine, 00:15:23.110 --> 00:15:25.700 on the same process where your program is running. 00:15:25.860 --> 00:15:26.740 And that's, I think, the main-- 00:15:26.970 --> 00:15:28.080 and there's all sorts of little details, 00:15:28.440 --> 00:15:30.120 like how the data structures are used internally, 00:15:30.500 --> 00:15:31.920 and SQLite doesn't have a lot of types. 00:15:32.080 --> 00:15:33.280 There's lots of other differences. 00:15:33.760 --> 00:15:34.880 I think that's the main one. 00:15:35.690 --> 00:15:36.780 Unless, Michael, I forgot something. 00:15:36.870 --> 00:15:38.140 MICHAEL LUTH: Yeah, no, I think it's-- 00:15:38.190 --> 00:15:39.360 and it's-- 00:15:40.740 --> 00:15:43.020 operationally, it's a separate thing 00:15:43.040 --> 00:15:49.080 run. It has to have both, it has to be secure because if your data gets exposed, like- 00:15:49.080 --> 00:15:54.360 For Postgres, is it not for SQL? Yes, it's running somewhere. People can SSH in if you're 00:15:54.360 --> 00:15:58.700 not careful. You've got to be mindful of passwords and all that stuff. That's totally true. 00:15:58.900 --> 00:16:02.520 Right. And it can go down. Like it could just become unavailable because you've screwed up 00:16:02.520 --> 00:16:07.360 something or whatever, right? It's a thing you have to manage in the complexity of running your app 00:16:07.360 --> 00:16:10.440 when it's like, well, it used to just be one thing I could run in a Docker container. Well, 00:16:10.520 --> 00:16:14.100 now I got different servers, they got to coordinate and there's firewalls and there's like, it's just, 00:16:14.560 --> 00:16:18.900 it just takes it so much higher in terms of complexity that like SQLite is a file. 00:16:19.290 --> 00:16:19.420 Yes. 00:16:20.030 --> 00:16:22.280 I mean, I do want to maybe defend Postgres a little bit there. 00:16:22.310 --> 00:16:26.620 Cause one thing that's like really nice and convenient in terms of like CI/CD and deployments 00:16:26.620 --> 00:16:31.140 and all that, oh, suppose you want to scale horizontally and there's like Docker containers 00:16:31.460 --> 00:16:34.780 running on the left and there's this one Postgres thing running on the right. 00:16:34.990 --> 00:16:38.260 I mean, you can just turn on and off all those Docker containers as you see fit. 00:16:38.500 --> 00:16:40.120 they're just going to connect to the Postgres instance. 00:16:40.360 --> 00:16:43.800 And I've done this trick for Calmcode a bunch of times 00:16:43.860 --> 00:16:44.980 where I just switch cloud providers, 00:16:45.440 --> 00:16:46.440 because Postgres is running there, 00:16:46.780 --> 00:16:48.380 and I can just move the Docker containers 00:16:48.440 --> 00:16:50.720 to another cloud provider, and it all works fine. 00:16:50.840 --> 00:16:51.720 No migration necessary. 00:16:52.420 --> 00:16:54.620 With SQLite, that aspect is a little bit more tricky. 00:16:54.720 --> 00:16:55.760 You have to be a bit more mindful. 00:16:56.120 --> 00:16:58.740 Although, I should mention, might be worth a Google. 00:16:59.320 --> 00:17:00.760 There's actually this one new cloud provider 00:17:01.000 --> 00:17:02.480 that's very much Python-focused. 00:17:02.760 --> 00:17:05.860 It's called Plash, P-L-A dot S-H, I think. 00:17:06.220 --> 00:17:07.520 Oh, this is new to me. 00:17:07.839 --> 00:17:08.780 Yeah, so I think-- 00:17:08.780 --> 00:17:09.060 Wow, OK. 00:17:09.329 --> 00:17:09.680 Look at this. 00:17:09.920 --> 00:17:12.060 From.py to.com in seconds. 00:17:12.579 --> 00:17:15.740 Yeah, it's the Answer AI, Jeremy Howard and friends. 00:17:15.870 --> 00:17:18.680 I don't know to what extent this is super production ready. 00:17:18.890 --> 00:17:21.620 And SQLite, you've got to be mindful of the production aspect 00:17:22.329 --> 00:17:23.380 for some reasons as well. 00:17:23.520 --> 00:17:25.780 But one thing that is kind of cool about them 00:17:26.160 --> 00:17:29.200 is they give you a persistent SQLite as a database 00:17:29.980 --> 00:17:32.380 and a pipeline process that can just kind of attach to it. 00:17:32.500 --> 00:17:33.940 And they just-- in their mind, that's 00:17:33.960 --> 00:17:35.860 the simplest way that a cloud provider should be. 00:17:36.040 --> 00:17:37.240 take a very opinionated approach. 00:17:38.600 --> 00:17:40.560 So yeah, if you're interested in maybe running this 00:17:40.570 --> 00:17:43.680 as a web service, migrations are a little bit tricky 00:17:43.810 --> 00:17:45.460 in that realm, because you do have 00:17:45.460 --> 00:17:47.440 to download the entire data set due to migration 00:17:47.770 --> 00:17:50.460 and upload it again, I think, if I recall correctly. 00:17:50.490 --> 00:17:52.160 And for some apps, that's no big deal. 00:17:52.330 --> 00:17:53.640 Others, that's a mega deal. 00:17:53.840 --> 00:17:54.740 Depends how big that data is. 00:17:55.510 --> 00:17:56.900 So I'm not suggesting this is going 00:17:56.930 --> 00:17:58.280 to be for everything and everyone, 00:17:58.490 --> 00:18:00.720 but I do think it's cool, which is why I figured I'd mention it. 00:18:00.930 --> 00:18:01.900 Oh, it's new to me. 00:18:03.220 --> 00:18:05.820 I'm going to follow up with a Litestream. 00:18:06.220 --> 00:18:06.740 Have you seen this? 00:18:07.120 --> 00:18:09.900 Yeah, that is also really neat. 00:18:11.300 --> 00:18:13.740 So basically, what if you want to back up your SQLite? 00:18:13.940 --> 00:18:14.840 Like, how could you do that? 00:18:15.360 --> 00:18:17.040 Oh, it might be nice to do that with S3. 00:18:17.280 --> 00:18:21.720 And I think it's like the guy who made the thing works at Fly.io. 00:18:21.920 --> 00:18:23.200 He's doing a bunch of low-level stuff. 00:18:23.580 --> 00:18:25.240 One thing about that open source package 00:18:25.290 --> 00:18:26.560 is also really interesting, by the way, 00:18:26.820 --> 00:18:29.800 is I think he refuses PRs from the outside. 00:18:30.460 --> 00:18:32.620 He just wants to have no distractions whatsoever. 00:18:33.340 --> 00:18:35.120 He has a very interesting way of developing software. 00:18:35.280 --> 00:18:36.400 You can submit issues, of course. 00:18:38.220 --> 00:18:40.740 I think if you scroll down, there used to be a notice that 00:18:40.810 --> 00:18:42.300 basically said, hey, this is a-- 00:18:42.500 --> 00:18:43.400 I'm not running this-- 00:18:43.990 --> 00:18:44.120 Yeah. 00:18:44.520 --> 00:18:44.960 There you go. 00:18:45.880 --> 00:18:48.260 We welcome-- yeah, contribution guide. 00:18:48.380 --> 00:18:49.320 We welcome bug reports. 00:18:51.300 --> 00:18:52.940 Yeah, this is a way where you can basically 00:18:53.300 --> 00:18:54.380 stream updates to S3. 00:18:54.700 --> 00:18:58.100 And the main observation there is S3 is actually really cheap 00:18:58.170 --> 00:18:59.680 if all you do is push stuff into it. 00:18:59.820 --> 00:19:02.280 If you never pull it out, usually getting it out 00:19:02.400 --> 00:19:03.540 is the expensive bit of S3. 00:19:03.960 --> 00:19:06.220 So this is like pennies on the dollar 00:19:06.620 --> 00:19:07.900 for really decent backup. 00:19:08.040 --> 00:19:09.680 And you can also send it to multiple-- 00:19:09.900 --> 00:19:11.720 you can send it to Amazon and also to DigitalOcean, 00:19:11.800 --> 00:19:12.280 if you like. 00:19:12.500 --> 00:19:12.680 Yeah. 00:19:13.060 --> 00:19:15.080 Yeah, because these days, S3 is really 00:19:15.240 --> 00:19:19.520 a synonym for blob storage on almost any hosting platform. 00:19:20.080 --> 00:19:23.320 Like, it used to be S3 might go to literally S3 at AWS. 00:19:23.560 --> 00:19:27.120 But now it's like, or DigitalOcean object spaces, 00:19:27.580 --> 00:19:29.180 or to you name it. 00:19:29.580 --> 00:19:32.160 They've all adopted the API, kind of like OpenAI's API. 00:19:32.920 --> 00:19:35.840 Yeah, I will say it's a little bit awkward that you have to-- 00:19:35.940 --> 00:19:37.580 like, sometimes you go to a cloud provider, 00:19:37.650 --> 00:19:39.820 and they say, you have to download a SDK 00:19:40.070 --> 00:19:41.820 from a competing cloud provider, and then you 00:19:41.900 --> 00:19:43.400 can connect to our cloud bucket. 00:19:44.020 --> 00:19:44.680 I know. 00:19:44.920 --> 00:19:46.220 And it's usually boto3. 00:19:46.410 --> 00:19:48.040 And boto3 is-- 00:19:48.200 --> 00:19:50.580 if you want to cry because you're using a library, 00:19:50.900 --> 00:19:53.080 like, boto3 has a good chance of being the first one 00:19:53.100 --> 00:19:53.560 to make you do it. 00:19:53.620 --> 00:19:55.160 It is so bad for me. 00:19:55.860 --> 00:19:57.659 It's so not custom-- 00:19:57.680 --> 00:19:59.640 It's not built with craft and love. 00:19:59.800 --> 00:20:01.980 It's like auto-generated where you pass these-- 00:20:02.300 --> 00:20:03.680 like, you pass this kind of dictionary, 00:20:03.900 --> 00:20:05.660 and then the other argument takes a separate dictionary 00:20:05.840 --> 00:20:07.220 that relates back-- it's just like, 00:20:07.340 --> 00:20:08.720 could you give me a real API here? 00:20:09.080 --> 00:20:10.200 IAN MCKAYAN: I mean, the one thing 00:20:10.220 --> 00:20:12.440 I can appreciate about Bodo that I do think is honest to mention 00:20:12.500 --> 00:20:14.980 is they do try to just maintain it. 00:20:15.220 --> 00:20:16.760 The backward compatibility of that thing 00:20:17.220 --> 00:20:19.660 also means it can't move in any direction as well. 00:20:19.740 --> 00:20:22.580 And I can't-- there is this meme where Google kills all 00:20:22.600 --> 00:20:24.959 of its products way too early, and Amazon's meme 00:20:24.980 --> 00:20:27.520 that they kill them way too late, sometimes never. 00:20:27.990 --> 00:20:28.080 Right? 00:20:28.280 --> 00:20:30.320 So in that sense, I can appreciate that they just 00:20:30.390 --> 00:20:33.440 try to keep Bodo just not necessarily as user friendly, 00:20:33.550 --> 00:20:34.680 but they do keep it super stable. 00:20:34.960 --> 00:20:36.480 Like, I get there's a balance there. 00:20:36.740 --> 00:20:36.820 Yeah. 00:20:37.180 --> 00:20:39.520 I feel like we still haven't really introduced this cache. 00:20:39.720 --> 00:20:40.520 We've kind of set the stage. 00:20:41.380 --> 00:20:43.920 Anyway, but yeah, SQLite, super cool. 00:20:44.220 --> 00:20:45.320 How does it work under the hood? 00:20:45.400 --> 00:20:46.820 Well, it's really just like a Python dictionary. 00:20:47.070 --> 00:20:49.400 So you can say something like, hey, make a new cache. 00:20:49.800 --> 00:20:52.040 And then you can do things like cache, square brackets, 00:20:52.450 --> 00:20:54.199 string name, equals, and then whatever 00:20:54.220 --> 00:21:00.160 Python object you like can go in. And Python has this serialization method called a pickle. 00:21:00.780 --> 00:21:05.200 Serialization just means, well, you can persist it to disk in some way, and then you can sort of 00:21:05.540 --> 00:21:10.460 get it back into memory again. And that's what disk cache just uses under the hood. So in theory, 00:21:10.740 --> 00:21:16.440 any Python object that you can think of can go into disk cache. The only sort of thing to be 00:21:16.700 --> 00:21:22.039 mindful of is if you have like Python version, if NumPy version 1 in Python 3.6, and you're going 00:21:21.900 --> 00:21:23.760 to inject a whole lot of that into this cache. 00:21:24.140 --> 00:21:26.400 Don't expect those objects to serialize nicely back 00:21:26.500 --> 00:21:29.380 if you're using Python 3.12 and NumPy version 2 or something. 00:21:29.640 --> 00:21:32.680 Right, because pickle is almost an in-memory representation 00:21:33.299 --> 00:21:34.200 of the thing. 00:21:34.220 --> 00:21:36.140 And that may have evolved over time. 00:21:36.440 --> 00:21:38.180 That's also a true statement about your own classes, 00:21:38.540 --> 00:21:38.680 potentially. 00:21:39.020 --> 00:21:41.700 Yeah, so if you're dealing with multiple Python versions 00:21:41.800 --> 00:21:43.420 and multiple versions of different packages, 00:21:43.640 --> 00:21:46.340 there's a little bit of a danger zone to be aware of there. 00:21:47.080 --> 00:21:48.900 That said, for most of the stuff that I do, 00:21:48.980 --> 00:21:50.620 that's basically a non-issue. 00:21:50.740 --> 00:21:56.800 But I do get this nice little object that can just store stuff into SQLite and can get it out. 00:21:56.860 --> 00:21:57.880 And it's very general. 00:21:58.540 --> 00:21:59.740 It's going to try to be clever about it. 00:21:59.820 --> 00:22:03.480 Like if you give it an int, it's going to actually store it as an int and not use the pickle format. 00:22:03.680 --> 00:22:05.360 So there's a couple of clever things that it can do. 00:22:06.260 --> 00:22:07.640 And it's also really like a Python dictionary. 00:22:07.800 --> 00:22:09.840 So you can do the square bracket thing. 00:22:09.980 --> 00:22:15.040 You can also do the delete and then cache square bracket thing to delete a key from the cache. 00:22:15.700 --> 00:22:17.640 Just like a Python dictionary, you have the get method. 00:22:17.820 --> 00:22:19.240 So you can say dot get key. 00:22:19.540 --> 00:22:21.380 And if it's missing, you can pass a default value. 00:22:22.460 --> 00:22:24.400 So it's very much like a dictionary. 00:22:25.660 --> 00:22:27.340 I think Bob's your uncle on that one. 00:22:27.350 --> 00:22:29.460 Unless, Michael, I've forgotten something. 00:22:29.510 --> 00:22:30.820 But I think that's the simplest way to do it. 00:22:30.830 --> 00:22:31.180 Yeah, pretty much. 00:22:31.330 --> 00:22:32.600 Yeah, I think so. 00:22:32.940 --> 00:22:34.420 The difference being it's not in memory. 00:22:34.820 --> 00:22:36.700 It's stored to a file. 00:22:36.970 --> 00:22:38.800 It happens-- it's not always a SQLite file. 00:22:39.020 --> 00:22:42.540 But often, it is a SQLite file as its core foundation 00:22:43.120 --> 00:22:43.780 that it's stored to. 00:22:43.910 --> 00:22:47.220 So it gives you process restart ability, 00:22:47.440 --> 00:22:49.140 where it still remembers the stuff you cached. 00:22:49.300 --> 00:22:50.700 It's not like LRU cache. 00:22:50.700 --> 00:22:52.560 We got to redo it every single time. 00:22:52.960 --> 00:22:55.740 And I think, I don't know where it is in the docs here, 00:22:56.260 --> 00:23:00.760 but the thread safety bit of it and the cross-process safety 00:23:00.850 --> 00:23:03.100 is really nice about, is it persistent? 00:23:03.300 --> 00:23:06.000 You've got this whole table here, things like, is it persistent? 00:23:06.300 --> 00:23:06.400 Yes. 00:23:06.470 --> 00:23:07.100 Is it thread safe? 00:23:07.460 --> 00:23:07.560 Yes. 00:23:07.590 --> 00:23:08.800 Is it process safe? 00:23:08.960 --> 00:23:09.560 Yes. 00:23:10.139 --> 00:23:12.680 Compared against other things people might choose. 00:23:13.490 --> 00:23:17.160 And that, honestly, I think that is the other half of the magic. 00:23:17.780 --> 00:23:21.240 Yeah, so especially for your web stuff, I would say that that's the thing you really want. 00:23:21.520 --> 00:23:23.400 And some of that, of course, is just SQLite itself. 00:23:25.280 --> 00:23:28.640 Historically, one reason why people always used to say, like, use Postgres, not SQLite, 00:23:28.820 --> 00:23:30.720 has to do with precisely this concurrency stuff. 00:23:32.000 --> 00:23:36.940 My impression is that SQLite is really good at reading, but writing can be slow if multiple processes do it. 00:23:37.300 --> 00:23:39.540 Some of that, I think, is related to the disk as well. 00:23:39.740 --> 00:23:41.300 I don't know to what extent that has changed. 00:23:41.380 --> 00:23:44.900 But historically, at least, whenever I was doing Django, hanging out at Django events, 00:23:45.060 --> 00:23:46.460 People are always saying, like, just use Postgres 00:23:46.560 --> 00:23:47.640 because it's better for the web thing. 00:23:48.720 --> 00:23:51.240 But it is safe, the SQLite. 00:23:51.280 --> 00:23:53.880 It might become slower, but it is thread safe if it's-- 00:23:53.880 --> 00:23:54.020 MARK MANDEL: Right. 00:23:54.200 --> 00:23:58.740 There's actually-- they've thought a lot about in this thing 00:23:58.940 --> 00:24:02.900 about transactions, concurrency, and basically dealing with that. 00:24:02.960 --> 00:24:05.080 But it is ultimately, for the most part, 00:24:05.520 --> 00:24:06.860 still SQLite underneath. 00:24:07.260 --> 00:24:10.340 But the thing with a cache is if you're writing it more 00:24:10.540 --> 00:24:12.760 than you're reading it, you probably shouldn't have a cache. 00:24:13.000 --> 00:24:13.060 Yeah. 00:24:13.520 --> 00:24:14.420 I mean, like... 00:24:15.840 --> 00:24:17.040 That beats the purpose. 00:24:18.260 --> 00:24:18.600 Exactly. 00:24:18.770 --> 00:24:20.620 Like, you get no value if you're recreating it. 00:24:21.000 --> 00:24:24.020 You're only probably just doing overhead and wasting memory or disk space. 00:24:24.620 --> 00:24:29.880 So it's inherently a situation where it's going to be pretty read-heavy, 00:24:30.220 --> 00:24:32.340 and SQLite is good at read-heavy scenarios. 00:24:32.720 --> 00:24:36.280 And maybe it's also fair to say, like, the LRU cache that you get with basic Python, 00:24:36.800 --> 00:24:38.400 so also to maybe explain that one, 00:24:38.540 --> 00:24:41.480 so the LRU cache is a little bit different because you decorate a function with it, 00:24:41.780 --> 00:24:44.560 and then given the same inputs, one output goes out, 00:24:44.610 --> 00:24:46.720 you can kind of keep track of a dictionary that's in memory. 00:24:47.100 --> 00:24:49.300 If you don't have a lot of stuff to keep in the back of your mind, 00:24:49.500 --> 00:24:51.480 then maybe you don't have to write to disk, right? 00:24:51.640 --> 00:24:54.380 So there's also maybe a reason to just stick to caching mechanisms 00:24:54.560 --> 00:24:56.260 that use Python memory, because I also think, 00:24:56.700 --> 00:24:58.200 I would imagine it to be quicker too. 00:24:59.860 --> 00:25:01.020 But maybe that's also... 00:25:01.020 --> 00:25:01.040 Probably, yes. 00:25:01.560 --> 00:25:02.460 That should be quicker. 00:25:02.580 --> 00:25:03.820 It's just that if you're capped at memory, 00:25:04.000 --> 00:25:05.120 then you might want to spill to disk, 00:25:05.190 --> 00:25:06.660 and then disk cache becomes interesting too. 00:25:06.960 --> 00:25:10.220 Right, for example, you have literally zero serialization, deserialization. 00:25:11.140 --> 00:25:13.920 What you put in LRU cache is the pointer to the object 00:25:14.260 --> 00:25:15.200 that you're caching, right? 00:25:15.300 --> 00:25:18.360 If you've got a class or a list that's part of the LRU cache. 00:25:18.560 --> 00:25:20.000 The one thing that is good to mention 00:25:20.030 --> 00:25:21.680 is also a really nice feature of disk cache 00:25:21.840 --> 00:25:23.880 is just like LRU cache has a decorator, 00:25:23.930 --> 00:25:26.980 so you can decorate a function, disk cache also has that. 00:25:27.530 --> 00:25:29.100 And it works kind of interestingly, too. 00:25:29.340 --> 00:25:30.880 So when you decorate the function, 00:25:32.320 --> 00:25:34.460 you do have to be a little bit careful if you use that. 00:25:34.630 --> 00:25:35.780 But then disk cache will-- 00:25:36.610 --> 00:25:39.660 I think it will hash the function name and the inputs 00:25:39.800 --> 00:25:40.420 that you pass. 00:25:40.840 --> 00:25:44.920 I don't know if it also hashes the contents of the function. 00:25:45.020 --> 00:25:46.260 Like if you change the function itself, 00:25:46.500 --> 00:25:49.000 I don't know if this cache will actually put that 00:25:49.040 --> 00:25:50.780 in a different slots, if that makes sense. 00:25:51.360 --> 00:25:53.840 Yeah, you can say @cache_memoize, 00:25:54.260 --> 00:25:57.160 which is the design pattern speak for just remember this. 00:25:57.680 --> 00:25:57.740 Yeah. 00:25:57.820 --> 00:25:58.680 It takes the arguments. 00:25:59.080 --> 00:26:00.640 And then it has like the Fibonacci sequence, 00:26:00.860 --> 00:26:02.500 which is the classic example, of course. 00:26:02.940 --> 00:26:05.160 And like there are some extra things you can set there as well. 00:26:05.280 --> 00:26:08.580 So you can say things like, hey, I think you're able to-- 00:26:08.660 --> 00:26:10.080 yeah, you're able to set the expiry. 00:26:10.300 --> 00:26:11.960 So you can say things like, I want to cache this, 00:26:11.990 --> 00:26:13.580 but only for the next five minutes or so, 00:26:14.100 --> 00:26:16.460 which can make a lot of sense if you're doing a front page 00:26:16.600 --> 00:26:17.060 kind of a thing. 00:26:17.690 --> 00:26:18.900 So like the Reddit front page or something 00:26:19.000 --> 00:26:20.760 like that that updates, but not every second. 00:26:20.850 --> 00:26:22.440 It probably updates once every five minutes 00:26:22.510 --> 00:26:23.200 or something like that. 00:26:23.620 --> 00:26:25.040 And then you do want to have something that's cached, 00:26:25.160 --> 00:26:27.100 but then after that, you want the cache 00:26:27.180 --> 00:26:28.320 to maybe basically just reset. 00:26:28.830 --> 00:26:29.860 And that is something you can also 00:26:30.100 --> 00:26:31.200 control with a few parameters here. 00:26:31.420 --> 00:26:32.600 Right, that's interesting. 00:26:32.740 --> 00:26:34.920 There's a couple good use cases that come to mind for me. 00:26:35.040 --> 00:26:37.100 Like one, if I put this on the function that 00:26:37.320 --> 00:26:39.660 generated the RSS feed for Talk Python, 00:26:39.940 --> 00:26:41.300 I could just say every one minute 00:26:41.760 --> 00:26:44.040 and then it might be a little bit expensive to compute 00:26:44.280 --> 00:26:47.880 because it's got a pars, you know, 535 episodes or whatever. 00:26:48.420 --> 00:26:50.780 But then for one minute, all the subsequent requests, 00:26:50.880 --> 00:26:52.440 just here's the answer, here's the answer. 00:26:52.600 --> 00:26:55.080 And then without me managing anything, 00:26:55.340 --> 00:26:58.080 it will just automatically the next minute refresh itself 00:26:58.900 --> 00:27:00.020 by the nature of how it works, right? 00:27:00.200 --> 00:27:01.640 How much traffic do you get on that endpoint? 00:27:01.840 --> 00:27:02.840 Just roughly, like you're asking. 00:27:02.920 --> 00:27:04.660 One terabyte of RSS a month. 00:27:05.700 --> 00:27:06.280 Okay, gotcha. 00:27:06.520 --> 00:27:08.120 Okay, but there you go. 00:27:08.320 --> 00:27:12.960 Like then just doing that like once a minute instead of like many times a minute will be a huge cookie. 00:27:13.480 --> 00:27:16.860 I would say it's probably more than one request a second. 00:27:17.160 --> 00:27:20.820 And the file size, the response size of the RSS feed is over a meg. 00:27:21.100 --> 00:27:24.280 And so it's a non-trivial amount of asking, you know. 00:27:24.380 --> 00:27:24.600 Yeah. 00:27:24.860 --> 00:27:27.380 And then like, and how do you fix that with a whole bunch of infrastructure? 00:27:27.620 --> 00:27:28.560 No, with a decorator. 00:27:28.740 --> 00:27:29.420 Like that feels... 00:27:29.440 --> 00:27:29.720 Exactly. 00:27:30.500 --> 00:27:30.900 Exactly. 00:27:31.140 --> 00:27:33.920 You pretty much summed up all the reasons why I'm so excited about this, 00:27:34.040 --> 00:27:36.700 because it's like you could do all of this complex stuff 00:27:36.820 --> 00:27:38.640 or just like you could just literally 00:27:39.340 --> 00:27:40.740 in such a simple way, 00:27:41.060 --> 00:27:42.880 just not recompute it as often. 00:27:43.400 --> 00:27:43.600 Yeah. 00:27:43.960 --> 00:27:44.520 Here's the danger. 00:27:44.660 --> 00:27:45.680 What if there's a race condition? 00:27:46.260 --> 00:27:48.220 And oh my goodness, two of them sneak in. 00:27:48.520 --> 00:27:49.100 You know what I mean? 00:27:49.180 --> 00:27:51.580 Like, okay, so you've done a little bit extra work 00:27:51.580 --> 00:27:51.980 and you throw it away. 00:27:52.060 --> 00:27:52.380 Who cares? 00:27:52.780 --> 00:27:53.800 I want to use Redis now. 00:27:53.960 --> 00:27:54.700 And Redis is cool, 00:27:54.880 --> 00:27:55.740 but I've never used it before. 00:27:56.060 --> 00:27:56.800 Better buy a book. 00:27:57.060 --> 00:27:57.500 Okay, no. 00:27:58.120 --> 00:27:59.060 With this cache, if you, 00:27:59.460 --> 00:28:01.760 I mean, I'm sure it won't solve everything, 00:28:02.020 --> 00:28:03.960 but this, I make a bit of a joke by saying, 00:28:04.020 --> 00:28:04.740 just use a decorator. 00:28:05.140 --> 00:28:07.280 But it's honestly that feeling that this library really 00:28:07.440 --> 00:28:07.840 does give you. 00:28:08.940 --> 00:28:10.840 You can just use it as a decorator, 00:28:11.020 --> 00:28:12.060 which has a lot of great use cases. 00:28:12.140 --> 00:28:13.240 You can just use it as a dictionary. 00:28:13.540 --> 00:28:15.880 So it still feels like you're writing Python. 00:28:16.220 --> 00:28:18.600 It's just Python with one concern less. 00:28:19.020 --> 00:28:19.600 And that is the magic. 00:28:20.040 --> 00:28:22.600 MARK MANDEL: And it takes on so many of the cool aspects 00:28:22.900 --> 00:28:26.980 of these high-end servers like Postgres or Redis or Valkey. 00:28:27.320 --> 00:28:29.440 Valkey is sort of the shiny new Redis, right? 00:28:29.620 --> 00:28:31.620 I would actually love to do a Redis benchmark. 00:28:31.720 --> 00:28:32.540 I haven't done that yet. 00:28:32.660 --> 00:28:35.900 But one thing I do wonder with disks are getting so much faster. 00:28:36.180 --> 00:28:36.400 Yes. 00:28:36.920 --> 00:28:37.020 Right? 00:28:37.170 --> 00:28:40.780 And so you can actually at some point wonder like how much faster is Redis really going to be 00:28:40.910 --> 00:28:42.940 and how much money are you willing to spend on it? 00:28:43.030 --> 00:28:47.480 Because if your cache is huge and it allows to go in memory in Redis, 00:28:47.950 --> 00:28:50.360 it could be wrong, but Redis is fully in memory, I think, right? 00:28:50.530 --> 00:28:51.140 I believe so. 00:28:51.560 --> 00:28:52.720 There is a database aspect. 00:28:53.040 --> 00:28:54.500 Redis is weird because it could be so many. 00:28:54.820 --> 00:28:55.360 Redis is cool. 00:28:55.590 --> 00:28:56.620 It can do a lot. 00:28:57.820 --> 00:29:00.400 But they do have, they actually have benchmarks here on. 00:29:00.500 --> 00:29:01.280 Ah, there you go. 00:29:01.640 --> 00:29:06.320 Compared against memcached and Redis. 00:29:06.700 --> 00:29:08.360 And it has the get speed and the write speed. 00:29:08.600 --> 00:29:10.520 And this is smaller is better. 00:29:10.870 --> 00:29:11.560 Yeah, look at this. 00:29:11.980 --> 00:29:13.360 Disc cache beats Redis. 00:29:13.840 --> 00:29:16.940 And I imagine that's because of the network hop or something. 00:29:17.420 --> 00:29:17.720 Yeah, exactly. 00:29:17.870 --> 00:29:19.800 I bet it's the network, the network connection. 00:29:20.040 --> 00:29:22.020 So if you're running it on the same machine, 00:29:22.130 --> 00:29:23.320 you would have a different number there. 00:29:24.160 --> 00:29:25.440 Might be good to maybe caveat that. 00:29:25.800 --> 00:29:26.560 Yeah, it might be. 00:29:26.870 --> 00:29:29.620 I mean, but also that I think that I don't know when they ran this benchmark, 00:29:29.900 --> 00:29:31.880 but I just checked on PyPI. 00:29:32.260 --> 00:29:33.740 This project started in 2016. 00:29:34.520 --> 00:29:35.340 So it might- 00:29:35.340 --> 00:29:38.140 I bet this is 2016 data right here. 00:29:38.140 --> 00:29:40.160 If I know how these docs go. 00:29:40.360 --> 00:29:43.640 Yeah, so it could also be that those are old disks 00:29:43.640 --> 00:29:44.980 comparing old memories, right? 00:29:45.419 --> 00:29:47.040 So then this is one of those weird benchmarks. 00:29:47.040 --> 00:29:49.580 You got to really run them every six months or so 00:29:49.580 --> 00:29:51.000 for them to remain relevant. 00:29:51.320 --> 00:29:51.960 Yeah, yeah. 00:29:51.960 --> 00:29:56.540 I mean, the new NVMe, whatever, disks, 00:29:56.540 --> 00:29:58.880 SSD disks are so fast. 00:29:58.880 --> 00:29:59.340 And also they're not memory. 00:29:59.620 --> 00:30:00.680 Memory is expensive nowadays. 00:30:01.600 --> 00:30:02.440 Yes, it is. 00:30:02.590 --> 00:30:04.080 People want to build data centers with them, I've heard. 00:30:04.880 --> 00:30:05.220 Yeah, yeah. 00:30:05.720 --> 00:30:07.640 And on the cloud there, this is a totally, 00:30:07.970 --> 00:30:10.800 this is another really interesting aspect to discuss. 00:30:11.460 --> 00:30:13.580 Probably more of a web dev side of things. 00:30:13.840 --> 00:30:17.260 But if you do LRU caches or even to a bigger degree, 00:30:17.350 --> 00:30:18.720 I run a whole separate server, 00:30:18.930 --> 00:30:20.280 even if it is just a Docker container 00:30:20.440 --> 00:30:21.820 that holds a bunch of this stuff in memory, 00:30:22.280 --> 00:30:24.740 that's going to take more memory on your VM 00:30:25.020 --> 00:30:26.300 or your cloud deployment or whatever. 00:30:26.660 --> 00:30:27.600 And if you just say, well, 00:30:28.000 --> 00:30:33.560 I have this 160 gig hard drive. That's an NVMe high speed drive. Like maybe I could just put a 00:30:33.610 --> 00:30:38.920 bunch of stuff there and you could really thin down your deployments, not just because it's not in 00:30:39.000 --> 00:30:42.680 memory in a cache somewhere, but if you're not having any form of cache, you might be able to 00:30:42.800 --> 00:30:48.380 dramatically lower how much compute you need and avoid them. Right. Like there's layers of how this 00:30:48.460 --> 00:30:53.280 could like shave off. And again, it's one of those things of like, oh, I just, can I pay for disk 00:30:53.560 --> 00:30:56.720 instead? Oh, that's a whole lot cheaper. What else do I got to do? You just got to write a tech 00:30:56.720 --> 00:31:02.920 creator yeah i think i pay five dollars for uh i think i remember exactly but i pay something like 00:31:03.100 --> 00:31:08.940 five dollars for 400 gigs of disc there you go and do you know how much 400 gigs of ram will cost 00:31:09.100 --> 00:31:18.120 on the cloud um well i mean more there goes the college tuition but the exactly sorry kids yeah 00:31:18.600 --> 00:31:22.940 no but like it's um and again like i vividly remember when i started college people were 00:31:23.060 --> 00:31:26.680 always saying oh keep it in memory because it's way faster than disc but i i think we got to let 00:31:26.700 --> 00:31:28.220 a lot of that stuff just go. 00:31:28.880 --> 00:31:30.340 Interesting idea. Yeah, I agree, though. 00:31:30.550 --> 00:31:32.260 I think you're right. But anyway, 00:31:33.080 --> 00:31:34.680 so far, we've mainly 00:31:34.690 --> 00:31:36.400 been discussing the mechanics of it, but 00:31:36.660 --> 00:31:38.680 there's some bills and whistles I think we should maybe also mention. 00:31:38.800 --> 00:31:40.420 The expiry definitely is one of them. 00:31:41.180 --> 00:31:42.440 There's also first in, first out 00:31:42.510 --> 00:31:43.760 kinds of things that you can do. 00:31:45.040 --> 00:31:46.440 So maybe if we could go back to that. 00:31:46.440 --> 00:31:47.940 Yeah, there's a bunch of features, actually, 00:31:48.760 --> 00:31:49.000 there. 00:31:50.620 --> 00:31:52.280 The expiry is interesting already 00:31:53.330 --> 00:31:53.420 because 00:31:54.240 --> 00:31:56.660 if you do regular, say, LRU caching 00:31:56.660 --> 00:31:58.860 like that, you have a natural. 00:31:59.280 --> 00:32:00.540 It's going to go away when the process 00:32:00.760 --> 00:32:02.740 restarts, and that's going to happen eventually. 00:32:03.340 --> 00:32:04.620 Even in a web app, you ship a new 00:32:04.760 --> 00:32:06.600 version, you've got to restart the thing or something. 00:32:07.100 --> 00:32:08.520 But when it goes to the disk, 00:32:08.630 --> 00:32:10.660 it starts to pile up, right? That's why I have this little admin 00:32:10.880 --> 00:32:12.660 page that has, like, how big is this 00:32:12.730 --> 00:32:13.720 and a button to clear it. 00:32:14.440 --> 00:32:16.600 In fairness, that can blow up 00:32:16.600 --> 00:32:18.440 to a lot of Palooza as well, if you're not careful. 00:32:18.620 --> 00:32:20.640 It is a concern. Yeah, it is definitely a concern. 00:32:20.730 --> 00:32:22.560 And so I think a big way to fix it, 00:32:22.800 --> 00:32:24.480 like the expiry, we already talked about why it's 00:32:24.580 --> 00:32:26.460 interesting for stale data. 00:32:26.540 --> 00:32:28.000 You want it to just like auto refresh, 00:32:28.250 --> 00:32:32.020 but it's also just a safeguard of maybe we'll just recompute this once a month. 00:32:32.240 --> 00:32:33.380 It's really quick and easy. 00:32:34.080 --> 00:32:34.280 Yeah. 00:32:34.980 --> 00:32:37.020 Maybe just don't let it linger forever. 00:32:37.440 --> 00:32:39.700 I think what you can also do though, if I'm not mistaken, 00:32:39.710 --> 00:32:41.820 is I think you can also set like a max key size. 00:32:41.960 --> 00:32:45.900 So you can say this cache, this particular disk cache can only have 10,000 keys in it 00:32:46.300 --> 00:32:48.520 and use a first in first out kind of a principle. 00:32:49.080 --> 00:32:51.720 Right, or last accessed or number of times. 00:32:52.000 --> 00:32:54.460 There's a bunch of metrics there actually for how that works. 00:32:54.680 --> 00:32:55.920 Yeah, it's pretty interesting. 00:32:56.800 --> 00:32:59.260 I've never had to fiddle around with them too much, 00:32:59.400 --> 00:33:02.380 but it's one of those things where even if I don't need it right now, 00:33:02.600 --> 00:33:04.660 it is just a relief to see that the feature is there 00:33:04.840 --> 00:33:05.960 in case you might need it later. 00:33:06.760 --> 00:33:07.880 Yeah, yeah, for sure. 00:33:08.300 --> 00:33:09.600 Let me see if I can find out where that is. 00:33:09.600 --> 00:33:11.000 I don't know, keep bouncing around the same spot. 00:33:11.380 --> 00:33:12.620 I've got it, let's just talk through them. 00:33:12.760 --> 00:33:15.680 So I think the tags and expiries are pretty interesting, 00:33:15.940 --> 00:33:18.600 but then there's also, I think, something that surprised me a little bit 00:33:18.600 --> 00:33:20.100 is these different kinds of caches. 00:33:20.460 --> 00:33:22.260 So there's like a fan out cache. 00:33:23.480 --> 00:33:24.060 Have you looked at these? 00:33:24.360 --> 00:33:25.020 These are interesting. 00:33:25.380 --> 00:33:29.020 I remember reading about, I've never used them, but I do remember reading them. 00:33:29.440 --> 00:33:32.140 So let me give you the quick rundown and then you'll understand instantly. 00:33:32.200 --> 00:33:32.680 It's super quick. 00:33:32.760 --> 00:33:35.500 So it uses sharding, which is a database term, right? 00:33:36.000 --> 00:33:40.780 So sharding is like, if I've got a billion records and it's a challenge to put them all 00:33:40.780 --> 00:33:45.320 in the same database entry, database table or server, I could actually have 10 servers 00:33:45.440 --> 00:33:49.400 and decide, well, okay, what we're going to do is if it's a number of the ID of the user, 00:33:49.520 --> 00:33:52.540 if the first number is one, then they go into this database. 00:33:52.680 --> 00:33:54.000 If it's two, then they go in that, right? 00:33:54.100 --> 00:33:57.100 So 2005 goes into the second database and so on. 00:33:57.980 --> 00:33:59.220 So it does that as well. 00:33:59.460 --> 00:34:03.520 This is one of the things it does to try to avoid the issues of multiple writers, I believe. 00:34:04.260 --> 00:34:07.000 So you're less likely to write to the same database. 00:34:07.100 --> 00:34:08.419 So it doesn't have to lock as hard. 00:34:08.700 --> 00:34:12.399 It's kind of what you do where you say, oh, I've got this for the YouTube link. 00:34:12.419 --> 00:34:14.639 And I've got this for the HTML markdown. 00:34:14.720 --> 00:34:19.520 Except those are like different chunks because of their use case. 00:34:19.540 --> 00:34:22.120 But you can also imagine, well, I've got this long list of users. 00:34:22.159 --> 00:34:24.520 but I still want to benefit from having multiple SQLite instances. 00:34:25.060 --> 00:34:26.760 And I suppose that's when you use this, right? 00:34:26.980 --> 00:34:28.399 Yeah, I think that is why. 00:34:28.580 --> 00:34:33.000 So it says, it's built on top of cache, cache fanout, automatically shards. 00:34:33.100 --> 00:34:35.600 Automatically, you just said how many you want, it figures out what that means. 00:34:36.020 --> 00:34:37.960 And it says, while readers and writers don't block each other, 00:34:38.159 --> 00:34:39.000 writers block other writers. 00:34:39.240 --> 00:34:44.260 Therefore, a shard for every concurrent writer suggests it. 00:34:44.379 --> 00:34:45.980 This will depend on your scenario, default is eight. 00:34:46.419 --> 00:34:47.520 So that's pretty cool. 00:34:48.320 --> 00:34:48.700 Yeah, okay. 00:34:48.960 --> 00:34:52.740 And presumably internally does something like hashing to figure out how to like 00:34:53.879 --> 00:34:54.940 send it around the shards. 00:34:55.350 --> 00:34:55.460 Right. 00:34:55.510 --> 00:34:57.800 The keys themselves have to be hashable anyway, probably. 00:34:57.940 --> 00:35:01.680 So just hashes the key and then shards on like the first couple letters or whatever. 00:35:02.840 --> 00:35:03.600 Yeah, this is cool. 00:35:03.650 --> 00:35:05.700 So it avoids the concurrency crashes. 00:35:06.200 --> 00:35:10.320 The one difference for me, the reason I didn't choose fanout cache is because I want to be 00:35:10.330 --> 00:35:13.800 able to say, I want to clear all the YouTube IDs, but I want to keep the really expensive 00:35:13.850 --> 00:35:15.200 to compute hashes. 00:35:15.580 --> 00:35:19.040 I want to be able to clear stuff if I really have to by category. 00:35:19.340 --> 00:35:21.840 And I guess you could also do that with tags, but I'm just not that advanced. 00:35:22.560 --> 00:35:23.780 Well, and also keep it simple, right? 00:35:23.900 --> 00:35:27.020 And again, it's one of those things where it's, oh, it's nice to know that this is in here, 00:35:27.140 --> 00:35:28.200 even if you don't use it directly. 00:35:28.800 --> 00:35:29.160 I do agree. 00:35:29.540 --> 00:35:30.400 This is a really nice feature. 00:35:30.740 --> 00:35:31.000 It is. 00:35:31.140 --> 00:35:34.340 And I probably will never in my life use it, but it's really cool that it's like, you know, 00:35:34.340 --> 00:35:37.920 it's one of those things about a library that when you're thinking about picking it, it's 00:35:38.000 --> 00:35:42.820 like, okay, the core feature is great, but if I outgrow it, what is my next step? 00:35:42.880 --> 00:35:45.080 Do I have to completely switch to something really different 00:35:45.220 --> 00:35:46.120 like Redis or Valkey? 00:35:46.400 --> 00:35:48.880 Or do I just change the class I'm using? 00:35:49.860 --> 00:35:50.960 I had that with this. 00:35:51.020 --> 00:35:52.680 So I actually had that feeling a while ago. 00:35:53.120 --> 00:35:54.800 We're going to get to my example I think a bit later. 00:35:54.960 --> 00:35:57.680 But I was using the Memoize decorator 00:35:57.820 --> 00:36:00.080 to decorate a function to properly cache that. 00:36:00.860 --> 00:36:03.860 But the one issue I had is an input to that function 00:36:04.060 --> 00:36:05.100 was a progress bar. 00:36:05.440 --> 00:36:06.740 It's kind of a remote specific thing. 00:36:06.740 --> 00:36:08.280 I wanted this one progress bar to update 00:36:09.080 --> 00:36:10.120 from inside of this one function. 00:36:10.540 --> 00:36:12.200 But the downside was that every time 00:36:12.220 --> 00:36:14.720 I rerun the notebook, I make a new progress bar object. 00:36:15.280 --> 00:36:16.420 Oh, and that means that the input 00:36:16.610 --> 00:36:17.960 has a different object going in. 00:36:17.960 --> 00:36:19.580 MARK MANDEL: So you never actually hit the cache. 00:36:19.730 --> 00:36:21.560 JOHN MUELLER: So, oh my god, I'm never hitting the cache. 00:36:21.760 --> 00:36:22.600 This is horrible. 00:36:23.100 --> 00:36:25.140 Then it turns out the Memoize decorator 00:36:25.310 --> 00:36:27.200 also allows you to ignore a couple of the inputs 00:36:27.460 --> 00:36:27.860 of the function. 00:36:28.030 --> 00:36:28.440 So you can also-- 00:36:28.440 --> 00:36:28.860 MARK MANDEL: Oh, interesting. 00:36:28.950 --> 00:36:31.060 Like ignore by keyword or something, keyword argument. 00:36:31.270 --> 00:36:31.800 JOHN MUELLER: Precisely. 00:36:31.850 --> 00:36:33.900 And there's a bunch of use cases for it, and this was one. 00:36:34.360 --> 00:36:35.820 And you can just imagine my relief 00:36:36.070 --> 00:36:39.440 after writing the entire notebook to then look at the docs 00:36:39.510 --> 00:36:40.880 and go, oh, sweet. 00:36:42.520 --> 00:36:43.640 Yeah, that's super sweet. 00:36:43.920 --> 00:36:44.080 Yeah. 00:36:44.640 --> 00:36:44.780 Okay. 00:36:45.460 --> 00:36:46.600 I think there's right below this. 00:36:46.680 --> 00:36:47.260 Yeah, there's a couple. 00:36:47.340 --> 00:36:48.380 We can just go down this list here. 00:36:48.500 --> 00:36:48.820 There's some cool. 00:36:48.880 --> 00:36:49.100 Oh, Django. 00:36:49.559 --> 00:36:51.860 So they have a legit Django cache. 00:36:52.320 --> 00:36:52.800 Yeah, yeah, yeah. 00:36:52.940 --> 00:36:53.400 Straight in. 00:36:53.760 --> 00:36:53.920 Sweet. 00:36:54.240 --> 00:36:56.000 Yeah, I recall a little bit. 00:36:56.040 --> 00:36:57.960 It was like a huge Django following for this thing. 00:36:58.260 --> 00:36:59.300 Yeah, and I think this is a part why. 00:36:59.400 --> 00:37:02.300 And when I first saw it, the reason it got sent to me is somebody's like, 00:37:02.340 --> 00:37:07.800 oh, this disk cache, Django cache is a really cool thing to just drop into Django. 00:37:07.960 --> 00:37:08.760 And I'm like, that's cool. 00:37:09.080 --> 00:37:10.860 I'm not using Django, but I admire it. 00:37:11.180 --> 00:37:14.420 That's why I didn't really look into it until I saw your use case outside of Django. 00:37:14.520 --> 00:37:16.880 I'm like, oh, okay, I understand how much this can do. 00:37:17.320 --> 00:37:20.000 So the Django dish cache says it uses the fanout cache, 00:37:20.100 --> 00:37:21.520 which we just discussed with the sharding, 00:37:21.760 --> 00:37:24.380 to provide a Django-compatible cache interface. 00:37:25.080 --> 00:37:26.780 And you just do that in your settings file, 00:37:26.980 --> 00:37:29.760 and you just say the back end is diskcache.django cache, 00:37:29.920 --> 00:37:30.720 and you give it a location. 00:37:31.140 --> 00:37:32.420 Boom, off it goes, right? 00:37:32.600 --> 00:37:33.400 So really, really nice. 00:37:33.720 --> 00:37:33.860 Cool. 00:37:34.140 --> 00:37:36.300 Yeah, and it sounds like you've done more Django than me. 00:37:37.000 --> 00:37:37.740 How's this sit with you? 00:37:37.780 --> 00:37:42.220 I mean, to be very clear, I do think Django is really nice and really mature. 00:37:42.370 --> 00:37:47.420 I do sometimes have a bit of a love-hate relationship with it because Django can go really, really deep. 00:37:47.940 --> 00:37:51.260 And some of that configuration stuff definitely can be a little bit in your face. 00:37:51.980 --> 00:37:57.680 So the main thing I just want to observe is doing everything manually inside of Django can be very time-consuming. 00:37:57.750 --> 00:38:02.320 So it's definitely nice to know that someone took the effort to make a proper Django plugin in the way that Django wants it. 00:38:03.060 --> 00:38:04.500 That's definitely the thing to appreciate here. 00:38:05.200 --> 00:38:07.360 I've never really used this in a Django app, to be honest. 00:38:08.540 --> 00:38:08.660 Yeah. 00:38:09.120 --> 00:38:11.200 You know, it has a lot of nice settings here. 00:38:11.230 --> 00:38:16.720 Like you can set the number of shards, the timeout, so in case there's a write or read 00:38:16.920 --> 00:38:19.440 contention, it can deal with that or it can at least let you know you're failing. 00:38:19.780 --> 00:38:21.500 It even has a size limit. 00:38:21.780 --> 00:38:24.140 Does it say how you can configure what to cache and whatnot? 00:38:24.310 --> 00:38:27.920 Or is like the-- I've never really used caches in Django in general. 00:38:27.990 --> 00:38:31.400 So I don't know if there's a general cache feature in Django itself that it will just 00:38:31.530 --> 00:38:32.700 plug into or if-- 00:38:32.740 --> 00:38:35.620 I think there is a general cache feature in Django. 00:38:35.640 --> 00:38:37.800 The Django people are like screaming silently. 00:38:38.080 --> 00:38:38.400 Yes. 00:38:38.979 --> 00:38:39.700 I apologize. 00:38:39.910 --> 00:38:40.480 I know, I know. 00:38:40.770 --> 00:38:44.780 But I'm pretty sure it's just like a built-in Django cache functionality. 00:38:44.940 --> 00:38:45.300 Exactly. 00:38:45.480 --> 00:38:45.900 Yeah, okay. 00:38:46.040 --> 00:38:47.540 It just routes into this thing. 00:38:47.940 --> 00:38:48.100 Exactly. 00:38:48.250 --> 00:38:50.520 So instead of configuring Redis, you just feed it this and you're good. 00:38:50.780 --> 00:38:51.600 That's the idea. 00:38:51.820 --> 00:38:52.520 Yes, exactly. 00:38:53.190 --> 00:38:53.280 Exactly. 00:38:53.340 --> 00:38:54.020 So there's more. 00:38:54.500 --> 00:38:57.060 The next one, I have to rage against the machine. 00:38:57.160 --> 00:39:00.120 I'm sure it's the way, but DQ, pronounced deck. 00:39:01.520 --> 00:39:01.580 Yeah. 00:39:02.320 --> 00:39:02.980 So I don't know. 00:39:03.060 --> 00:39:04.000 For me, I still say DQ. 00:39:04.030 --> 00:39:04.660 I don't say deck. 00:39:04.900 --> 00:39:06.560 like it's spelled D-E-Q-U-E. 00:39:06.980 --> 00:39:09.000 And I know a lot of computer science people just call that deck, 00:39:09.240 --> 00:39:12.480 but this cache dot DQ or deck, however you want to use it. 00:39:13.040 --> 00:39:14.600 It provides, there's, 00:39:14.660 --> 00:39:18.060 there's a couple of higher order data structures that like operate on what we 00:39:18.240 --> 00:39:21.600 talked about so far, but to give you data structure behavior, right? 00:39:21.940 --> 00:39:24.040 Like what we talked about so far is sort of dictionary, 00:39:24.540 --> 00:39:26.700 but not list or order or any of that. 00:39:26.960 --> 00:39:31.040 But this, you can actually do go and add a thing. 00:39:31.480 --> 00:39:32.260 How do we add one? 00:39:32.630 --> 00:39:34.560 Anyway, you can say like pop left, pop left. 00:39:34.700 --> 00:39:36.800 over and over to get, I guess you just add it. 00:39:37.280 --> 00:39:38.160 It's kind of like a queue. 00:39:38.930 --> 00:39:39.600 Yeah, like a queue, exactly. 00:39:40.310 --> 00:39:42.320 With the goal of taking stuff out of it instead of into it. 00:39:42.620 --> 00:39:44.720 But you don't normally think of a cache as doing that. 00:39:44.770 --> 00:39:47.960 But it'd be a cool way actually to fan out work across processes. 00:39:48.360 --> 00:39:51.080 I was about to say, that's a really good, I think, 00:39:51.540 --> 00:39:54.920 I mean, there's people that have made, I forget the name, 00:39:55.319 --> 00:39:57.800 the Python queuing system, Celery. 00:39:59.360 --> 00:40:01.860 So that one is also built in such a way that you can say like, 00:40:02.020 --> 00:40:04.720 oh, where do you have the list of jobs that still need doing? 00:40:04.980 --> 00:40:06.580 And I think also Redis is used-- 00:40:06.580 --> 00:40:07.620 MARK MANDEL: Yeah, right, Redis and Qum. 00:40:07.860 --> 00:40:08.500 Yeah, exactly. 00:40:08.700 --> 00:40:09.960 Or Rabbit and Qum as well. 00:40:10.160 --> 00:40:11.260 FRANCESC CAMPOY: Yeah, exactly. 00:40:11.360 --> 00:40:14.080 But you can configure SQLite if you want to, though, 00:40:14.240 --> 00:40:15.020 if I recall with those. 00:40:15.180 --> 00:40:16.460 It's just that in this particular case, 00:40:16.580 --> 00:40:19.280 if you don't want to use Celery, you can still kind of roll 00:40:19.320 --> 00:40:20.800 your own by using this cache as well. 00:40:20.980 --> 00:40:22.880 I'm assuming it uses the same pickle tricks, 00:40:22.980 --> 00:40:24.180 so you can do general Python things 00:40:24.280 --> 00:40:26.360 and if the process breaks for whatever reason, 00:40:26.480 --> 00:40:27.840 you still have the jobs that need doing. 00:40:27.900 --> 00:40:29.580 MARK MANDEL: Yeah, we're still going 00:40:29.580 --> 00:40:31.620 to have to talk about this serialization thing, 00:40:31.820 --> 00:40:32.440 These pickles. 00:40:33.020 --> 00:40:33.160 Yes. 00:40:33.680 --> 00:40:34.160 Not yet. 00:40:34.300 --> 00:40:34.760 Let's go through this. 00:40:34.900 --> 00:40:35.680 Let's go through this first. 00:40:35.940 --> 00:40:36.240 Let's first. 00:40:36.680 --> 00:40:39.380 Before we get distracted, because it's a deep. 00:40:40.539 --> 00:40:42.480 So DEC, I guess we'll go DEC. 00:40:44.200 --> 00:40:48.680 DEC provides an efficient and safe means for cross-thread, cross-process communication. 00:40:49.080 --> 00:40:51.520 Like, you would never think you would get that out of a cache, really. 00:40:51.740 --> 00:40:52.640 But it's... 00:40:52.980 --> 00:40:54.080 You would do that in SQLite. 00:40:54.940 --> 00:40:55.460 Yeah, exactly. 00:40:55.960 --> 00:40:57.200 But you would do work to do that, right? 00:40:57.240 --> 00:40:58.220 You would do like transactions. 00:40:58.700 --> 00:40:59.340 And you would do sorting. 00:40:59.400 --> 00:41:00.940 You would figure out, well, what if there's contention? 00:41:01.320 --> 00:41:04.100 I mean, the fact that it's just kind of a pop, it's pretty nice. 00:41:04.360 --> 00:41:05.080 Yeah, that's definitely nice. 00:41:05.320 --> 00:41:06.040 No, that's definitely true. 00:41:06.420 --> 00:41:11.100 Although one thing that makes it easy in this case, though, is again, it is all running in like one process. 00:41:11.280 --> 00:41:16.440 So it's not like we've got SQLite running in one place and there's 16 Docker containers that can randomly interact with it. 00:41:16.680 --> 00:41:17.340 No, that can, though. 00:41:17.660 --> 00:41:18.240 Is that the case? 00:41:18.680 --> 00:41:23.020 Because disk cache itself is already cross-process safe. 00:41:23.240 --> 00:41:24.420 Like, that's why I was so excited about it. 00:41:25.339 --> 00:41:26.920 But it has to be on the same machine, though. 00:41:27.180 --> 00:41:28.060 Like, that's what I do think. 00:41:28.640 --> 00:41:30.040 Yes, it's got to at least be accessible. 00:41:30.360 --> 00:41:31.120 RushRite, that is true. 00:41:31.500 --> 00:41:33.940 because technically there's nothing that says 00:41:33.980 --> 00:41:35.100 you can't put the file anywhere. 00:41:35.220 --> 00:41:37.040 I think there's mega performance issues 00:41:37.100 --> 00:41:38.120 and locking issues on, 00:41:38.180 --> 00:41:40.100 it says basically don't use it on network drives. 00:41:40.480 --> 00:41:41.400 Yeah, so that's the thing. 00:41:42.260 --> 00:41:44.560 Some of this is like, okay, you can do the locking. 00:41:44.680 --> 00:41:45.380 You can do all those things. 00:41:45.400 --> 00:41:45.880 You can do it well, 00:41:45.980 --> 00:41:47.760 but the practicality of the network overhead 00:41:47.860 --> 00:41:50.160 is something that usually causes a lot of confuffle, 00:41:50.440 --> 00:41:51.160 at least in my experience. 00:41:51.600 --> 00:41:51.700 Yeah. 00:41:52.260 --> 00:41:54.020 Okay, another one is disk cache index, 00:41:54.720 --> 00:41:57.480 which creates a mutable mapping and ordered dictionary. 00:41:57.800 --> 00:42:00.000 So if you kind of really want to lay into the dictionary side, 00:42:00.340 --> 00:42:01.180 Yeah, you can do that. 00:42:01.820 --> 00:42:03.660 That one's transactions as well. 00:42:04.070 --> 00:42:06.740 So you can actually, it has like sort of in-place updates 00:42:06.980 --> 00:42:07.840 and other things you can do. 00:42:08.420 --> 00:42:11.560 So you can say, I want to make sure that I'm going to get 00:42:11.820 --> 00:42:13.320 two different things out of the cache. 00:42:13.620 --> 00:42:15.740 And I want to make sure that they're not changed 00:42:15.940 --> 00:42:16.980 while I'm doing that, right? 00:42:17.300 --> 00:42:18.600 Just like you would with threading or something. 00:42:18.960 --> 00:42:20.700 Yeah, so nothing can happen in between. 00:42:20.990 --> 00:42:22.500 So they both have to come out at the same time. 00:42:22.550 --> 00:42:25.040 So the two values that I get, they were, they were, 00:42:26.040 --> 00:42:27.700 they both existed at the same time in the cache 00:42:27.730 --> 00:42:29.160 at the point in time that I was retrieving it. 00:42:29.600 --> 00:42:30.320 Yeah, yeah, exactly. 00:42:30.680 --> 00:42:34.980 So just with cache.transact and you just go to town on it. 00:42:35.060 --> 00:42:36.500 That's pretty straightforward, right? 00:42:36.720 --> 00:42:36.820 Yep. 00:42:37.020 --> 00:42:37.940 Are there any more in here? 00:42:38.740 --> 00:42:43.940 There's a bunch of recipes for like barriers and throttling and probably semaphore-like stuff, 00:42:44.160 --> 00:42:47.120 but I don't really want to talk about. 00:42:47.180 --> 00:42:48.700 But you touched on these eviction policies. 00:42:48.940 --> 00:42:49.780 Here's where I was looking for. 00:42:49.900 --> 00:42:51.780 There's these different ones here that are kind of cool. 00:42:52.180 --> 00:42:52.240 Whoops. 00:42:52.440 --> 00:42:52.900 I didn't go away. 00:42:53.020 --> 00:42:57.300 Yeah, so you can set a maximum to the cache. 00:42:57.640 --> 00:43:00.120 I think you do that by number of items typically in it. 00:43:00.240 --> 00:43:01.440 It could also be the case. 00:43:01.460 --> 00:43:02.600 Size or something, yeah. 00:43:02.900 --> 00:43:04.620 Yeah, or like total disk size, 00:43:04.810 --> 00:43:05.800 maybe we should double check. 00:43:06.180 --> 00:43:08.380 The default for the disk size is one gig. 00:43:08.740 --> 00:43:09.260 Yeah, there you go. 00:43:09.260 --> 00:43:10.920 So there's already a built-in one, yeah. 00:43:11.160 --> 00:43:12.740 Which might catch people off guard. 00:43:14.140 --> 00:43:15.660 Much of the stuff is cash, but not always. 00:43:15.810 --> 00:43:16.460 I don't understand. 00:43:17.080 --> 00:43:18.700 Yeah, you've got to be a little bit mindful of that, 00:43:18.710 --> 00:43:19.000 I suppose. 00:43:19.280 --> 00:43:21.440 But it's the same default, 00:43:21.680 --> 00:43:22.840 not to have it go to infinity. 00:43:24.160 --> 00:43:24.520 Agreed. 00:43:26.020 --> 00:43:29.900 Yeah, I guess small screen on my side, but like, yeah, last recently. 00:43:30.480 --> 00:43:31.080 I'll read them out. 00:43:31.160 --> 00:43:31.820 I'll read them out for you. 00:43:31.820 --> 00:43:31.900 Yeah. 00:43:31.980 --> 00:43:35.320 So we got recent last recently stored as a default, every cache item 00:43:35.500 --> 00:43:39.840 records the time it was stored in the cache and that adds an index to that 00:43:40.000 --> 00:43:40.100 field. 00:43:40.220 --> 00:43:41.420 So it's nice and fast, which is cool. 00:43:41.860 --> 00:43:45.760 We have, this is, there's some other ones that are more nuanced, like least 00:43:46.420 --> 00:43:51.640 recently used, not in terms of time, but we've got one that was accessed a hundred 00:43:51.800 --> 00:43:53.220 times and one that was accessed two times. 00:43:53.680 --> 00:43:55.700 Even if the one is accessed two times was just accessed, 00:43:55.790 --> 00:43:57.560 that one's getting kicked out because it's not as useful. 00:43:57.670 --> 00:43:58.000 I don't know. 00:43:58.500 --> 00:43:59.300 That's a pretty neat feature. 00:43:59.480 --> 00:44:02.360 And then the one people would expect is, I don't know, 00:44:02.720 --> 00:44:03.980 maybe at least recently used. 00:44:04.220 --> 00:44:04.560 How are you? 00:44:04.800 --> 00:44:05.020 Yeah. 00:44:06.500 --> 00:44:07.140 Yeah, exactly. 00:44:07.740 --> 00:44:10.140 And there's also pruning mechanisms, if I'm not mistaken. 00:44:10.340 --> 00:44:11.800 So there's all sorts of fun. 00:44:11.980 --> 00:44:14.560 You can argue there are bells and whistles until you need them. 00:44:15.680 --> 00:44:18.640 And one thing I have always found is every item that I see here, 00:44:19.660 --> 00:44:21.100 you might not need it right now, 00:44:21.150 --> 00:44:23.080 but for every item you see, you do plausibly go, 00:44:23.460 --> 00:44:25.580 oh, but that might be useful later down the line somewhere. 00:44:25.800 --> 00:44:29.640 Like the transaction thing where you retrieve two things at the same time. 00:44:29.720 --> 00:44:33.820 I don't really have a use case for it, but I can imagine it one might, 00:44:33.900 --> 00:44:35.180 where the consistency really matters. 00:44:35.560 --> 00:44:36.340 Yeah, I can. 00:44:36.620 --> 00:44:38.120 I could see using the fan out cache. 00:44:38.420 --> 00:44:38.800 Yeah, definitely. 00:44:39.020 --> 00:44:41.440 But probably not the transaction. 00:44:41.680 --> 00:44:45.280 But I'm already talking to MongoDB, which doesn't have transactions effectively. 00:44:45.660 --> 00:44:46.880 So not really. 00:44:47.640 --> 00:44:48.540 What about performance? 00:44:48.780 --> 00:44:49.680 Should we talk about your graphs? 00:44:50.100 --> 00:44:50.640 You brought pictures. 00:44:51.160 --> 00:44:51.380 Yes. 00:44:51.880 --> 00:44:55.560 So that might be, so when you told me like, hey, let's do an episode on disk cache, 00:44:55.630 --> 00:44:58.020 and I kind of told myself, okay, then I need to do some homework. 00:44:58.160 --> 00:44:59.480 Like, I actually have to use it for something real. 00:44:59.600 --> 00:45:00.140 It's a bit complex. 00:45:01.500 --> 00:45:04.380 So what we're going to try and do is we're looking at a chart right now, 00:45:04.510 --> 00:45:05.980 and I'm going to explain to Michael what it does, 00:45:06.170 --> 00:45:09.380 and I'm going to try to explain it in such a way such that if you're not watching 00:45:09.510 --> 00:45:12.600 but listening, that you're also going to be fairly interested in what you're seeing. 00:45:12.900 --> 00:45:14.660 And I'll link to the chart, of course, people can. 00:45:15.000 --> 00:45:16.700 Yeah, so this is all running on GitHub pages, 00:45:17.120 --> 00:45:20.460 and the charts that you see here definitely needed a bit of disk cache 00:45:20.480 --> 00:45:22.100 to make it less painful. 00:45:23.280 --> 00:45:25.920 So what I've done is I've downloaded a Git repository. 00:45:26.440 --> 00:45:27.240 What you're looking at right now 00:45:27.320 --> 00:45:28.760 is the Git repository for Marimo. 00:45:29.160 --> 00:45:30.500 And then I just take a point in time 00:45:30.920 --> 00:45:33.960 and I say, okay, let's just see all the lines of code. 00:45:34.200 --> 00:45:35.720 And then I take another point in time. 00:45:36.200 --> 00:45:38.720 And then I basically just do kind of a Git blame 00:45:38.920 --> 00:45:40.800 to see if the line got changed in between. 00:45:41.480 --> 00:45:42.640 So what you're looking at here 00:45:42.660 --> 00:45:44.520 is kind of a chart over time 00:45:44.760 --> 00:45:46.720 where it's basically like a bar chart, 00:45:46.840 --> 00:45:48.720 but it changes colors as time moves forward. 00:45:49.160 --> 00:45:53.040 and the shape that you see is that things that happened early on, 00:45:53.400 --> 00:45:54.840 well, there's a nice thick slab, 00:45:54.860 --> 00:45:57.140 but it gets a little bit thinner and thinner as time moves forward 00:45:57.240 --> 00:45:59.080 because some of those lines of code got replaced. 00:45:59.700 --> 00:46:02.340 But in the case of Marimo, you can see that, you know, 00:46:02.580 --> 00:46:05.620 most of the lines of code actually stay around for a long time 00:46:05.680 --> 00:46:10.200 and it's kind of like a smooth sedimentary layer every time we move forward. 00:46:10.960 --> 00:46:12.760 It's compressing a little over time. 00:46:12.940 --> 00:46:15.680 Like there's the weight of the project has sort of compressed it. 00:46:15.880 --> 00:46:16.940 So, yeah, it's pretty interesting. 00:46:17.260 --> 00:46:18.700 So, okay, so that's pretty cool. 00:46:18.820 --> 00:46:20.280 But you can also go to Django. 00:46:21.200 --> 00:46:22.520 So there's a director on top. 00:46:22.520 --> 00:46:24.000 MARK MANDEL: So if I go to Django. 00:46:24.390 --> 00:46:25.960 Oh, you can put this cache in here. 00:46:26.200 --> 00:46:27.480 This is really different. 00:46:27.930 --> 00:46:28.440 FRANCESC CAMPOY: Yes. 00:46:28.590 --> 00:46:29.740 MARK MANDEL: What is this telling us? 00:46:29.890 --> 00:46:31.880 FRANCESC CAMPOY: So what you can see here 00:46:32.020 --> 00:46:33.420 is that at some point in time, there's 00:46:33.420 --> 00:46:34.620 a huge shift in the sediment. 00:46:35.260 --> 00:46:38.000 There's a lot of light sand and a lot of the dark sand goes away. 00:46:38.300 --> 00:46:41.140 There's also a button that allows you to show the version number. 00:46:41.750 --> 00:46:42.980 So I've-- 00:46:42.980 --> 00:46:43.400 yep, there you go. 00:46:43.400 --> 00:46:44.180 MARK MANDEL: There we go, yeah. 00:46:44.350 --> 00:46:46.460 FRANCESC CAMPOY: So you can see that right before a new version, 00:46:46.720 --> 00:46:49.080 a bunch of changes got introduced or right after. 00:46:49.260 --> 00:46:50.640 It's usually around the version number 00:46:50.740 --> 00:46:51.340 that you can see that shift. 00:46:51.340 --> 00:46:55.280 Right, once the feature freeze is lifted, 00:46:55.840 --> 00:46:57.780 some stuff comes in, PRs come in maybe or something. 00:46:58.060 --> 00:46:58.220 Yes. 00:46:59.039 --> 00:47:01.500 And one other thing that's actually kind of fun, 00:47:01.580 --> 00:47:02.500 if you go-- 00:47:02.500 --> 00:47:03.980 there's this project called scikit-lego 00:47:04.050 --> 00:47:05.700 that you can also go ahead and select. 00:47:05.880 --> 00:47:06.700 And folks-- 00:47:06.700 --> 00:47:08.560 I've heard a pretty cool guy maintains that, yeah. 00:47:09.180 --> 00:47:11.280 Well, so the funny thing is you can see that there's 00:47:11.280 --> 00:47:12.620 a massive shift there at some point. 00:47:12.740 --> 00:47:12.960 OK. 00:47:13.420 --> 00:47:14.620 That's when we got the new maintainer. 00:47:16.560 --> 00:47:18.140 Are you on the purple or the green side? 00:47:18.580 --> 00:47:22.160 So there's this dark blue sediment that sort of goes down massively at that point. 00:47:22.360 --> 00:47:27.040 But yeah, no, so but in this case, like the first thing he did is redid all the docs. So we went 00:47:27.200 --> 00:47:31.880 from Sphinx to make docs. And that's like a huge, if you look at the lines of code that changed as 00:47:31.880 --> 00:47:36.840 a result that, you know, that's quite a lot. But if we now start talking about how you make a chart 00:47:36.980 --> 00:47:40.720 like this, you got to imagine like, I take the start of the GitHub history, I take the end of the 00:47:40.760 --> 00:47:46.160 GitHub history, I sample like 100 points in between, and then for every line in every file, 00:47:46.240 --> 00:47:47.260 I do a git blame. 00:47:49.620 --> 00:47:52.620 MARK MANDEL: I think Django is something like 300,000 lines 00:47:52.740 --> 00:47:52.980 of code. 00:47:52.980 --> 00:47:54.120 I mean, that's a lot of-- 00:47:54.120 --> 00:47:54.660 A lot of-- 00:47:54.700 --> 00:47:57.020 MARK MANDEL: So that thing took two hours and 15 minutes 00:47:57.180 --> 00:47:58.440 on my M4 Mac. 00:47:58.480 --> 00:48:00.680 And if you go there, you can actually select it. 00:48:02.380 --> 00:48:04.900 That one-- that was a chunky boy, is what I'll say. 00:48:06.240 --> 00:48:08.360 MARK MANDEL: Yeah, 550,000 lines. 00:48:08.780 --> 00:48:08.840 Yeah. 00:48:09.020 --> 00:48:09.740 MARK MANDEL: There you go. 00:48:09.800 --> 00:48:12.200 But you can see that there's one version change, I think, 00:48:12.240 --> 00:48:13.360 where they made a bunch of changes. 00:48:13.500 --> 00:48:14.320 And it could be that-- 00:48:14.320 --> 00:48:14.420 MARK MANDEL: Yeah. 00:48:15.300 --> 00:48:17.540 That might have been, again, because I checked the docs 00:48:17.640 --> 00:48:19.860 as well on Markdown files, it might have been a big docs 00:48:20.060 --> 00:48:20.120 change. 00:48:20.720 --> 00:48:22.260 But hopefully by just looking at this, 00:48:22.320 --> 00:48:25.000 you can go like, oh yeah, this is probably a notebook somewhere. 00:48:25.680 --> 00:48:28.960 And there's a huge for loop that does threading 00:48:29.060 --> 00:48:31.200 and tries to do as much in parallel as possible. 00:48:31.680 --> 00:48:32.740 And there's a progress bar. 00:48:33.540 --> 00:48:33.660 Right? 00:48:34.260 --> 00:48:34.440 Yeah. 00:48:35.940 --> 00:48:37.060 And we don't-- 00:48:37.400 --> 00:48:39.340 Now I see why you had this problem with the caching. 00:48:39.860 --> 00:48:40.400 That's right. 00:48:41.240 --> 00:48:43.800 But yeah, but here's also where the threading came in. 00:48:43.900 --> 00:48:49.420 Because the moment you say this point in time, now do all the files, do the git blame, that's definitely something that can happen in parallel. 00:48:49.940 --> 00:48:58.180 But then for every file, for every point in time, you do want to have something in the cache that says, okay, if I have to restart this notebook for whatever reason, that number is just known. 00:48:58.420 --> 00:48:58.920 Don't check it again. 00:48:59.040 --> 00:48:59.240 Yeah. 00:48:59.820 --> 00:49:00.500 Yeah, super interesting. 00:49:00.990 --> 00:49:01.100 Okay. 00:49:01.700 --> 00:49:07.940 I wonder if this 4.0 in 2022, is that might be when they switched to async? 00:49:08.020 --> 00:49:09.100 They started supporting async. 00:49:10.140 --> 00:49:11.620 It could be docs as well. 00:49:11.630 --> 00:49:12.080 I'm not sure. 00:49:12.380 --> 00:49:15.160 Well, yeah, so that's kind of the hard thing of some of these charts. 00:49:15.320 --> 00:49:18.780 Like, I could expand these charts by saying things like, 00:49:18.880 --> 00:49:21.800 okay, only the Python files, et cetera. 00:49:22.900 --> 00:49:25.060 But, like, the way that this is hosted, 00:49:25.100 --> 00:49:27.000 this is, like, really using the disk as a cache 00:49:27.120 --> 00:49:28.760 because all these charts are Altair charts, 00:49:28.860 --> 00:49:29.640 and you can save them to disk, 00:49:29.680 --> 00:49:32.300 and then you can easily upload them to GitHub pages. 00:49:32.540 --> 00:49:35.120 So I do everything in disk cache to make sure that if I, 00:49:35.340 --> 00:49:37.020 for whatever reason, the notebook fails, 00:49:37.100 --> 00:49:40.500 I don't have to sort of do anything fancy to get it back up. 00:49:41.640 --> 00:49:44.180 But then once it's time to actually put it on the site, 00:49:44.320 --> 00:49:46.580 I could use disk cache to show the charts, 00:49:46.670 --> 00:49:47.500 but then I would need a server. 00:49:47.780 --> 00:49:51.600 So actually using disk to actually just serve some files 00:49:51.650 --> 00:49:53.820 is also just a pretty fine and good idea. 00:49:54.700 --> 00:49:56.940 There are some things on the Marimo side 00:49:57.140 --> 00:50:00.280 where we are also hoping to maybe give better caching tools 00:50:00.770 --> 00:50:01.480 to the library itself. 00:50:02.360 --> 00:50:03.460 It's just that when I was doing this, 00:50:03.460 --> 00:50:04.940 I actually found a bug in our caching layer, 00:50:05.010 --> 00:50:06.340 so then I switched back to disk cache. 00:50:07.180 --> 00:50:07.820 You know what? 00:50:07.900 --> 00:50:08.480 Look, that's valuable. 00:50:08.880 --> 00:50:10.700 That's maybe not the way you will find, but it's valuable. 00:50:11.000 --> 00:50:15.100 It's-- oh, so one thing you learn is that caching is actually 00:50:15.270 --> 00:50:15.920 hard to get right. 00:50:16.560 --> 00:50:17.000 Oh, it is. 00:50:17.260 --> 00:50:17.860 It is. 00:50:18.210 --> 00:50:18.480 Very hard. 00:50:18.720 --> 00:50:20.540 It's on par with naming things. 00:50:22.160 --> 00:50:24.960 It is one of the two things that goes wrong-- 00:50:25.300 --> 00:50:28.040 naming things, cache invalidation, and off by one errors. 00:50:28.090 --> 00:50:28.780 Yes, exactly. 00:50:29.320 --> 00:50:29.880 It's the middle one. 00:50:30.100 --> 00:50:30.200 Yeah. 00:50:32.200 --> 00:50:33.320 Dad jokes are amazing. 00:50:33.780 --> 00:50:37.900 Anyway, so one thing about this repo, by the way, 00:50:38.020 --> 00:50:38.660 this is all my-- 00:50:38.820 --> 00:50:40.120 we're going to add a link to the show notes. 00:50:40.540 --> 00:50:43.680 There is a notebook, so if you feel like adding your own project 00:50:43.840 --> 00:50:46.480 that you want to just add, feel free to spend your compute 00:50:46.700 --> 00:50:49.680 resources two and a half hours to add a popular project. 00:50:49.770 --> 00:50:50.480 I would love to have that. 00:50:50.760 --> 00:50:52.520 One thing I think will be cool with these sorts of charts 00:50:52.610 --> 00:50:55.260 is to see what will change when LLMs kind of get into the mix. 00:50:55.740 --> 00:50:59.340 Do we see more code shifts happen if more LLMs get used 00:50:59.540 --> 00:50:59.820 for these libraries? 00:50:59.820 --> 00:51:00.680 MARK MANDEL: Very interesting. 00:51:01.230 --> 00:51:02.520 I don't know if more code-- 00:51:02.810 --> 00:51:05.860 old code will get changed, but they are verbose code writers, 00:51:06.020 --> 00:51:06.320 those things. 00:51:06.760 --> 00:51:06.800 Yes. 00:51:07.080 --> 00:51:10.720 So this, assuming we can do this over time 00:51:10.780 --> 00:51:12.280 and we're going to start tracking this, 00:51:13.100 --> 00:51:14.780 I'm calling this code archaeology. 00:51:15.720 --> 00:51:18.060 I do think it will be an interesting chart. 00:51:18.260 --> 00:51:20.260 As is, I think it's already quite interesting to see differences 00:51:20.440 --> 00:51:21.200 between different projects. 00:51:21.640 --> 00:51:23.020 I think if you go to sentence transformers, 00:51:23.020 --> 00:51:25.720 you can also see when the project got moved from an academic lab 00:51:25.860 --> 00:51:26.460 to hugging face. 00:51:26.840 --> 00:51:29.560 So there are interesting things you can see with these charts. 00:51:30.580 --> 00:51:33.300 But you are going through every file, every line, 00:51:33.640 --> 00:51:35.020 get blame 100 times. 00:51:35.460 --> 00:51:36.200 Yeah, a lot. 00:51:36.740 --> 00:51:38.720 You got to do 100 times per line as well. 00:51:39.040 --> 00:51:42.400 Well, so the start of the project with a Git repository, 00:51:42.690 --> 00:51:44.160 and then to make a chart like this, 00:51:44.210 --> 00:51:45.820 you got a sample over the entire timeline. 00:51:46.700 --> 00:51:48.820 And it is a bit cheeky because sometimes you can go like, 00:51:48.850 --> 00:51:51.160 OK, but there's a character that changed because of a linter. 00:51:51.480 --> 00:51:53.160 And then is that really a change? 00:51:53.400 --> 00:51:54.220 Does it really matter? 00:51:55.019 --> 00:51:59.980 It's whoever decided to run the format on the thing or whatever. 00:52:00.240 --> 00:52:01.300 We're looking at the Django chart. 00:52:01.310 --> 00:52:02.960 It could also just be that black just got an update 00:52:03.050 --> 00:52:03.920 or something like that, right? 00:52:03.940 --> 00:52:04.260 Yeah, exactly. 00:52:04.740 --> 00:52:05.460 It's also possible. 00:52:06.420 --> 00:52:07.020 It's very possible. 00:52:07.340 --> 00:52:09.500 It's unlikely, but it's not impossible, let me say. 00:52:09.670 --> 00:52:10.200 But yeah, anyway. 00:52:11.799 --> 00:52:13.340 Yeah, this was one of the benchmarks 00:52:13.510 --> 00:52:16.240 that I did with disk cache that I thought was pretty amusing 00:52:16.630 --> 00:52:17.500 and pretty interesting. 00:52:18.420 --> 00:52:19.520 But there's this one other feature 00:52:19.530 --> 00:52:21.080 that I think we should also talk about, which 00:52:21.180 --> 00:52:24.740 is that if you want to, disk cache actually lets you 00:52:24.840 --> 00:52:26.080 do the serialization yourself. 00:52:26.450 --> 00:52:29.280 So normally, what it would do is it would say, 00:52:29.440 --> 00:52:31.280 like, OK, let's do the pickle thing. 00:52:31.350 --> 00:52:32.360 And it's a bit clever, right? 00:52:32.560 --> 00:52:34.880 So if the thing you're storing is like an integer, 00:52:35.180 --> 00:52:38.620 doesn't go through the whole pickle thing and just stores it as an integer there's these native types 00:52:38.760 --> 00:52:42.320 that sqlite has and then you know it's able to do something clever but right as soon as it becomes 00:52:42.440 --> 00:52:49.340 like a custom class or a list of weird things then yeah then then it's i personally don't like it 00:52:49.400 --> 00:52:53.540 pickling i would i would prefer that it makes me do something it's i think it's weird well so the 00:52:53.540 --> 00:52:58.360 thing is you can write your own disk class and then what you can do is you can pass set disk class 00:52:58.700 --> 00:53:03.780 onto uh the disk cache itself and i'm just kind of wondering like when might it make sense to do 00:53:03.660 --> 00:53:07.000 this sort of a thing and if you go to the docs there's actually like a really good example just 00:53:07.280 --> 00:53:13.220 right there which is jason yeah i i lost your example if i'll get it back if you type disk 00:53:13.540 --> 00:53:18.780 you'll find it so jason has this interesting thing the it is text if you think about it but 00:53:18.840 --> 00:53:23.220 it's text that has a bit of structure and that means that there are these compression libraries 00:53:23.260 --> 00:53:28.060 you can actually run on them and especially if you have like a pattern that repeats itself let's say 00:53:28.060 --> 00:53:32.480 a list of users or something like that and there's always the user key and there's always maybe the 00:53:32.460 --> 00:53:35.560 the email key and those things just repeat themselves all over the place, 00:53:36.020 --> 00:53:37.540 then there is an opportunity to, 00:53:38.770 --> 00:53:41.380 there's this library called zlib where you can just take that string, 00:53:41.670 --> 00:53:45.280 you can compress it and then that compressed representation can go into disk cache instead. 00:53:46.800 --> 00:53:49.320 Yeah, I figured that sounds like a lot of fun. 00:53:49.670 --> 00:53:51.320 You can just grab the implementation there. 00:53:51.880 --> 00:53:55.500 I have this notebooks repository where I have LLMs just write fun little notebooks. 00:53:55.790 --> 00:53:58.080 I always check the results obviously just to be clear on that. 00:53:58.790 --> 00:54:00.580 But one thing that was I think pretty cool to see, 00:54:00.720 --> 00:54:08.260 if you just do the normal data type and you pickle it, then you get a certain size. And if you just 00:54:08.340 --> 00:54:14.180 have a very short, normal Python dictionary basic thing, then it's negligible. You shouldn't use this 00:54:14.260 --> 00:54:19.460 JSON trick. But the moment you get text heavy, there's just a lot of text that you're inputting 00:54:19.560 --> 00:54:23.599 there and there's some repetition of characters. Or if you really do something that's highly 00:54:23.600 --> 00:54:30.120 compressible, it is not unheard of to get like 80%, 90% savings 00:54:30.250 --> 00:54:31.220 on your disk space, basically. 00:54:32.140 --> 00:54:33.220 Now, there is a little bit of overhead 00:54:33.370 --> 00:54:35.700 because you were doing the compression and decompression. 00:54:36.000 --> 00:54:37.560 But if you're doing text-heavy stuff, 00:54:38.480 --> 00:54:40.260 this is something that can actually save you a whole bunch. 00:54:40.270 --> 00:54:42.760 And I can imagine for LLMs, this would also be a win. 00:54:43.060 --> 00:54:43.300 OK. 00:54:43.690 --> 00:54:47.640 So this JSON disk, not only does it serialize to and from JSON, 00:54:47.840 --> 00:54:48.800 which I think is safer. 00:54:49.180 --> 00:54:50.960 It can be a pain if you had date times. 00:54:51.010 --> 00:54:52.020 You've got to do something about that. 00:54:52.400 --> 00:54:53.820 or JSON or something like that. 00:54:53.860 --> 00:54:54.120 You can. 00:54:54.340 --> 00:54:55.060 Yeah, yeah, I think. 00:54:55.600 --> 00:54:56.960 But then it's using Zlib here. 00:54:57.180 --> 00:54:58.980 You know, I just actually did something like this 00:54:59.240 --> 00:55:00.720 for just something in my database. 00:55:00.860 --> 00:55:01.940 It had nothing to do with caching. 00:55:02.780 --> 00:55:06.680 But these records are holding tons of text 00:55:06.920 --> 00:55:09.800 for something sort of tangential to the podcast. 00:55:10.360 --> 00:55:12.900 And I'm like, I don't really want to put 100K of text 00:55:13.380 --> 00:55:15.160 that I'm not going to query against or search 00:55:15.620 --> 00:55:16.580 into the database. 00:55:17.180 --> 00:55:19.960 So I used Python's XZ implementation. 00:55:20.280 --> 00:55:20.760 And like you said. 00:55:21.300 --> 00:55:22.860 There's a bunch of compression algorithms you could use. 00:55:22.860 --> 00:55:23.600 It was way fast. 00:55:23.780 --> 00:55:26.020 So I just store it as bytes now, and it's like a tenth the size. 00:55:26.200 --> 00:55:26.400 It's great. 00:55:26.880 --> 00:55:29.420 So I guess this is the same, but for the cache back end, right? 00:55:29.940 --> 00:55:30.260 Yeah. 00:55:30.460 --> 00:55:32.040 And I think-- well, you can see, I think Zlib 00:55:32.140 --> 00:55:33.000 is being used internally. 00:55:33.220 --> 00:55:33.280 Yeah. 00:55:33.540 --> 00:55:35.760 I mean, it's not XZ, but it's the same idea. 00:55:36.020 --> 00:55:36.900 Yeah, exactly. 00:55:38.079 --> 00:55:39.740 And there's always new compression algorithms. 00:55:40.080 --> 00:55:41.940 Like, feel free to check whatever makes sense. 00:55:42.420 --> 00:55:45.560 But the fact that you have one very cool example 00:55:45.600 --> 00:55:47.460 to add on the docs, because you can just copy and paste it. 00:55:47.520 --> 00:55:48.480 A lot of people benefit from it. 00:55:48.800 --> 00:55:49.880 But why stop here? 00:55:50.480 --> 00:55:52.040 Because this is the company you can do for JSON. 00:55:52.200 --> 00:55:53.060 What else can you do? 00:55:53.380 --> 00:55:55.920 Before we move on, though, if I were writing this, 00:55:55.940 --> 00:56:00.320 I would recommend using microJSON or ORJSON 00:56:00.320 --> 00:56:02.880 or whatever, some of the more high performance versions 00:56:03.140 --> 00:56:03.340 right there. 00:56:03.560 --> 00:56:05.720 Yeah, and I think ORJSON-- 00:56:05.740 --> 00:56:07.060 I mean, performance is cool. 00:56:07.260 --> 00:56:08.920 The reason I use ORJSON a lot more 00:56:09.040 --> 00:56:10.380 has to do with the types that it supports. 00:56:10.620 --> 00:56:12.820 So it can accept the NumPy arrays, for example, 00:56:13.060 --> 00:56:14.060 and just listifies it. 00:56:14.240 --> 00:56:17.000 And I think it has a few things with dates as well. 00:56:17.080 --> 00:56:19.120 It just has a slightly better support for a few things. 00:56:19.500 --> 00:56:20.220 OK, good to know. 00:56:20.360 --> 00:56:21.380 All right, where are we going? 00:56:21.820 --> 00:56:22.120 What's next? 00:56:23.020 --> 00:56:23.820 Numpy arrays. 00:56:24.760 --> 00:56:25.160 OK. 00:56:25.660 --> 00:56:28.760 So a lot of people like to do things with embeddings nowadays. 00:56:29.020 --> 00:56:31.620 So like text thing goes in, some sort of array thing comes out. 00:56:31.700 --> 00:56:33.580 And then hopefully, if two texts are similar, 00:56:33.720 --> 00:56:34.840 then the arrays are also similar. 00:56:35.040 --> 00:56:36.660 So you can do all sorts of fun little lookups. 00:56:37.720 --> 00:56:40.040 And I do a fair share of doing things with embeddings. 00:56:40.120 --> 00:56:43.720 And embeddings are also not notoriously expensive 00:56:43.760 --> 00:56:45.600 to calculate, but still pretty expensive to calculate. 00:56:46.020 --> 00:56:48.760 OK, but you can write your full Python thing in there. 00:56:48.900 --> 00:56:53.960 So if you compare storing NumPy as bytes compared to that 00:56:54.120 --> 00:56:55.740 to a pickle, it's actually even. 00:56:55.960 --> 00:56:57.460 There's very little to gain there. 00:56:57.860 --> 00:57:00.040 But one thing you could do is you could say, well, 00:57:01.040 --> 00:57:02.700 let's maybe bring it down to float 16. 00:57:03.320 --> 00:57:04.160 That's a thing you can do. 00:57:04.240 --> 00:57:05.860 You can sort of say, before we save it, 00:57:05.860 --> 00:57:08.940 we actually make it just a little bit less accurate 00:57:09.080 --> 00:57:10.040 on the numeric part of it. 00:57:10.040 --> 00:57:11.580 But that'll save us a whole bunch of disk space. 00:57:11.700 --> 00:57:12.680 So that's already kind of old. 00:57:13.020 --> 00:57:15.240 Well, you need it to be super precise 00:57:15.900 --> 00:57:16.900 when it's involved in calculations. 00:57:17.700 --> 00:57:19.900 But then in the end, if you're not going to report the numbers 00:57:20.540 --> 00:57:23.440 to great decimal places, maybe going down is good, yeah. 00:57:23.580 --> 00:57:24.880 Yeah, it depends on the use case. 00:57:25.040 --> 00:57:28.340 But typically, you could argue maybe a 1% difference 00:57:28.480 --> 00:57:32.200 in similarity if we have 100x savings on disk. 00:57:32.400 --> 00:57:33.420 That'll be kind of a win. 00:57:34.100 --> 00:57:37.180 So one thing I was sort of focusing on is just this-- 00:57:37.780 --> 00:57:40.880 you can do things like, OK, come up 00:57:40.880 --> 00:57:42.720 with your own little weird data structure where you say, 00:57:42.980 --> 00:57:45.320 OK, let's pretend we're going to quantize the whole thing. 00:57:45.700 --> 00:57:50.120 So we're going to calculate the quantiles of the float values that it can take. 00:57:50.520 --> 00:57:54.760 And we're going to take basically 256 buckets. 00:57:55.240 --> 00:57:56.660 We're going to store the scale. 00:57:57.360 --> 00:57:58.420 We're going to store the mean. 00:57:58.660 --> 00:58:01.140 And then we're going to store in what bucket the number was in. 00:58:01.460 --> 00:58:03.120 And you can turn that into a string representation. 00:58:03.560 --> 00:58:04.980 These things are pretty fun to write. 00:58:05.660 --> 00:58:05.760 Nice. 00:58:06.420 --> 00:58:11.020 And yeah, and then you scroll down into your big notebook and then you find this. 00:58:12.960 --> 00:58:13.640 There you go. 00:58:13.800 --> 00:58:14.760 That's a retrieval time. 00:58:14.860 --> 00:58:20.600 I think I got like a 4x improvement in terms of like disk space being saved. 00:58:20.650 --> 00:58:24.620 It was like a 1% similarity score that I had to give up for doing things like this. 00:58:24.900 --> 00:58:29.280 Mileage can vary, of course, but like, again, these are fun things to sort of start playing with 00:58:29.550 --> 00:58:32.260 because you have access to the way that you write that down. 00:58:32.660 --> 00:58:36.080 So that was also like a fun little exercise to do. 00:58:36.200 --> 00:58:36.280 Yeah. 00:58:36.430 --> 00:58:40.640 Could you save NumPy arrays by just putting up, just converting it to bytes? 00:58:41.060 --> 00:58:42.100 There's probably some efficient. 00:58:42.880 --> 00:58:43.660 You know what? 00:58:43.730 --> 00:58:44.740 What about a parquet file? 00:58:45.100 --> 00:58:47.620 Like in an in-memory Parquet file, then you just say, 00:58:47.800 --> 00:58:48.880 here's the value in bytes. 00:58:50.239 --> 00:58:53.660 So I tried the bytes thing and compared it to the pickle thing, 00:58:53.820 --> 00:58:54.980 and that was basically the same size. 00:58:55.440 --> 00:58:55.600 OK. 00:58:55.740 --> 00:58:57.180 That barely led to anything. 00:58:57.660 --> 00:58:59.960 About the Parquet one, I mean-- 00:59:00.400 --> 00:59:01.180 You do get compression. 00:59:01.540 --> 00:59:03.700 Well, yeah, but I could be wrong on this one. 00:59:03.780 --> 00:59:07.220 But I think Parquet is optimized to be a disk representation. 00:59:07.900 --> 00:59:09.080 And then once you want to have it in memory, 00:59:09.180 --> 00:59:10.580 it becomes an arrow representation. 00:59:11.100 --> 00:59:11.480 I see. 00:59:11.800 --> 00:59:12.140 Yeah, probably. 00:59:12.680 --> 00:59:14.180 So in that sense, what I would do is, OK, 00:59:14.360 --> 00:59:16.700 if you have something in Arrow, you use this cache 00:59:16.760 --> 00:59:18.520 to make sure it's written as parquet. 00:59:18.680 --> 00:59:20.380 But then you have to be-- you kind of have 00:59:20.380 --> 00:59:23.260 to know what you're doing if you're going to make parquet 00:59:23.460 --> 00:59:23.720 files. 00:59:24.220 --> 00:59:25.580 And also, the benefit of a parquet file 00:59:25.640 --> 00:59:27.000 is that you have one huge table. 00:59:27.160 --> 00:59:27.680 Because then-- 00:59:27.680 --> 00:59:28.720 Right, you can scan it, yeah. 00:59:28.980 --> 00:59:29.980 Yeah, it's a columnar format. 00:59:30.200 --> 00:59:32.040 So then if I were a column, boy, would I 00:59:32.100 --> 00:59:33.100 want to have all the rows in me. 00:59:35.800 --> 00:59:38.700 So in that sense, what you-- 00:59:38.760 --> 00:59:40.240 yeah, so in that sense, what I would do instead 00:59:40.620 --> 00:59:43.680 is if you, for whatever reason, you just have a lot of data, 00:59:43.820 --> 00:59:47.360 It's still kind of a cache, but it makes more sense to store all of it in like a huge parquet file. 00:59:47.710 --> 00:59:49.520 In parquet, you can store a partition. 00:59:49.730 --> 00:59:54.420 So you can say this one column that's partitioned, a date would be like a very typical thing to partition on. 00:59:54.690 --> 00:59:58.540 And then if you point polars to like parquet, but you say, I only want to have this date, 00:59:58.710 --> 01:00:01.840 it can sort of do the forward scan and only pick the rows that you're interested in. 01:00:03.240 --> 01:00:09.120 And I would imagine that that would beat anything we might do with this cache, especially if the table is big. 01:00:09.560 --> 01:00:11.500 So you don't always want to cache stuff. 01:00:11.800 --> 01:00:15.720 Like I said, I actually don't avoid hitting the Mongo database 01:00:15.830 --> 01:00:18.580 a lot for my projects because it's like the response time 01:00:18.610 --> 01:00:20.140 is quick enough and might as well. 01:00:20.560 --> 01:00:22.320 I want to take two little avenues here. 01:00:22.320 --> 01:00:25.000 But the first one is, what about DuckDB? 01:00:25.540 --> 01:00:28.640 I know at least on the data science side and the analytics 01:00:28.880 --> 01:00:31.620 side, DuckDB is really popular, really well respected, 01:00:31.970 --> 01:00:32.380 really fast. 01:00:32.520 --> 01:00:33.900 Maybe you don't even cache it. 01:00:33.960 --> 01:00:36.080 Maybe you just use DuckDB as a thing. 01:00:36.450 --> 01:00:37.080 How do you feel about that? 01:00:37.200 --> 01:00:40.220 I mean, DuckDB does solve a very different problem 01:00:40.240 --> 01:00:41.980 than SQLite or Postgres in a way. 01:00:42.160 --> 01:00:44.580 So I don't believe-- to name one thing, 01:00:44.780 --> 01:00:47.680 I believe DuckDB does assume that everything under the hood 01:00:47.860 --> 01:00:48.920 is immutable. 01:00:49.300 --> 01:00:51.180 So it will never be ACID compliant, 01:00:51.280 --> 01:00:52.600 because it doesn't necessarily have to be. 01:00:52.600 --> 01:00:54.280 You can still insert rows, if I'm not mistaken. 01:00:54.600 --> 01:00:56.320 But the use case is just assumed 01:00:56.320 --> 01:00:57.480 to be analytical in general. 01:00:58.500 --> 01:00:58.960 That like-- 01:00:58.960 --> 01:00:59.140 I see. 01:00:59.560 --> 01:01:02.600 --it's really designed to sort of fit that use case. 01:01:03.000 --> 01:01:04.080 You can insert rows, though. 01:01:04.140 --> 01:01:05.280 So like-- 01:01:05.280 --> 01:01:07.280 I mean, you might be caching data science things 01:01:07.380 --> 01:01:08.500 that you're only computing once. 01:01:08.700 --> 01:01:10.920 Like, for example, your charts. 01:01:11.320 --> 01:01:13.660 Once you've computed that, it's not going to change because it's historical. 01:01:14.000 --> 01:01:16.400 I mean, I might want to rerun it a month later or something like that. 01:01:16.560 --> 01:01:17.780 That's something I might want to do. 01:01:18.220 --> 01:01:21.840 And in that particular case, it would be cool if the sampling is the same 01:01:21.900 --> 01:01:24.760 and I just want to add one sample at the end that all those samples I had before, 01:01:25.000 --> 01:01:26.300 that those are in a cache somewhere. 01:01:26.780 --> 01:01:29.160 Or maybe you want faster, better resolution. 01:01:29.360 --> 01:01:31.400 Instead of going 100, you're going to go to 1,000 points, 01:01:31.660 --> 01:01:34.140 but you could do 10% less because those are done, right? 01:01:34.360 --> 01:01:34.980 So stuff like that. 01:01:35.020 --> 01:01:37.880 But then what you would never do with a cache is do a group buy 01:01:37.900 --> 01:01:39.020 and then a mean, for example. 01:01:39.300 --> 01:01:39.860 It's like-- 01:01:41.180 --> 01:01:43.820 It's not-- it's outgrown its use at that point. 01:01:43.940 --> 01:01:44.260 That's for sure. 01:01:45.080 --> 01:01:45.480 Yeah. 01:01:45.740 --> 01:01:47.780 And if it were a part of it, then the docs would say so. 01:01:47.820 --> 01:01:51.320 But like-- no, so in my mind, DuckDB really just 01:01:51.660 --> 01:01:54.460 solves a different problem, similar to like general SQLite 01:01:54.580 --> 01:01:56.300 also solves a different problem than disk cache. 01:01:56.300 --> 01:01:58.960 And also Postgres is also solving a slightly different problem. 01:01:59.200 --> 01:01:59.300 Sure. 01:01:59.820 --> 01:02:00.220 All right, fair. 01:02:00.560 --> 01:02:01.660 Like the other angle-- 01:02:01.800 --> 01:02:01.900 Yeah? 01:02:02.200 --> 01:02:02.780 No, go ahead. 01:02:02.820 --> 01:02:03.260 Finish your thoughts. 01:02:03.500 --> 01:02:06.000 Well, I also really love Postgres, I got to say. 01:02:06.120 --> 01:02:07.420 Like the thing I really like about it 01:02:07.480 --> 01:02:09.640 is that it is like boring, but in a good way, software 01:02:09.740 --> 01:02:12.780 where like I have a Postgres thing running there 01:02:14.400 --> 01:02:17.400 and whatever SSH thing I need, I can just swap the cloud 01:02:17.500 --> 01:02:18.900 provider and it'll just go ahead and still 01:02:19.000 --> 01:02:20.340 run without me having to move the data 01:02:20.400 --> 01:02:21.780 or do any migration or anything like that. 01:02:22.220 --> 01:02:23.600 That is also just like a nice feeling, 01:02:24.380 --> 01:02:25.180 but it solves a different problem. 01:02:25.460 --> 01:02:25.560 Yeah. 01:02:25.780 --> 01:02:25.980 Yeah. 01:02:26.580 --> 01:02:29.260 These would very likely be used together, not instead of-- 01:02:29.460 --> 01:02:31.140 I mean, Postgres can be used instead of disk cache, 01:02:31.260 --> 01:02:33.940 but disk cache, definitely not instead of Postgres. 01:02:34.400 --> 01:02:37.960 So the other one, the other angle I wanted to riff on, have you riff on just a little bit is 01:02:38.240 --> 01:02:43.660 think people, especially people who are maybe new to this idea of caching, they can end up thinking, 01:02:43.840 --> 01:02:47.060 okay, I'm going to store stuff. We talked a lot about like, I get something back from the database. 01:02:47.220 --> 01:02:51.080 I could store that in a cache. So I don't have to query it again or whatever. And those are 01:02:51.220 --> 01:02:55.680 certainly good use cases. But I think a lot of times an even better use case is if you're going 01:02:55.680 --> 01:03:00.579 to get 20 rows back from a database, do some Python and construct them along with a little 01:03:00.580 --> 01:03:02.340 other information into some object. 01:03:03.040 --> 01:03:04.500 And then that's really what you want to work with. 01:03:04.860 --> 01:03:07.420 Store that constructed thing in the cache. 01:03:07.720 --> 01:03:08.100 You know what I mean? 01:03:08.280 --> 01:03:12.360 Like, as far as you can go down the compute layer, like, don't just stop like, well, it 01:03:12.460 --> 01:03:13.900 comes back from the database, so we cache it. 01:03:14.000 --> 01:03:16.880 Like, if there's a way to say it, like, there's a bunch of work after that, think about how 01:03:16.980 --> 01:03:18.240 you might cache at that level. 01:03:18.520 --> 01:03:20.560 Okay, I'm going to pitch you a dream then. 01:03:21.660 --> 01:03:26.600 Imagine you have a Python notebook and, oh, you're running a cell, you're running a cell, 01:03:26.660 --> 01:03:29.460 you're running a cell, and halfway the kernel dies for whatever weird reason. 01:03:29.780 --> 01:03:29.900 Right. 01:03:30.280 --> 01:03:33.740 it'd be nice if I could just reboot the notebook and it would just pick it up again and move further. 01:03:33.960 --> 01:03:37.280 Because again, I'm picking something out of a database and I'm doing something little with it 01:03:37.280 --> 01:03:42.280 and processing, processing, processing. But wouldn't it be nice if maybe every cell had a caching mechanism? 01:03:43.280 --> 01:03:49.660 If only you had some influence. If only we're a company that did this kind of stuff. 01:03:51.080 --> 01:03:55.180 You can imagine these are things that we are thinking about. And again, what I'm about to 01:03:55.400 --> 01:03:59.020 suggest is definitely a dream. This is not something that works right now. Don't pin me on this. 01:03:59.040 --> 01:04:00.600 like we're thinking out loud here. 01:04:01.200 --> 01:04:02.900 You can also imagine this being super useful 01:04:03.140 --> 01:04:05.000 where an entire team can share the cache. 01:04:05.360 --> 01:04:05.440 Yeah. 01:04:05.980 --> 01:04:06.040 Right? 01:04:06.150 --> 01:04:08.160 So if your colleague already calculated something, 01:04:08.280 --> 01:04:09.740 you don't have to recalculate it again. 01:04:10.339 --> 01:04:12.900 There's all sorts of use cases like that as well. 01:04:13.170 --> 01:04:15.020 But there may be, there are these moments 01:04:15.110 --> 01:04:17.680 when you want to have very tight manual control 01:04:17.850 --> 01:04:19.540 over what goes into the cache. 01:04:19.890 --> 01:04:21.020 That makes a lot of sense and it's great. 01:04:21.230 --> 01:04:22.200 But there are also moments 01:04:22.270 --> 01:04:23.920 when you just really don't want to think about it at all. 01:04:24.170 --> 01:04:25.420 And you just want everything to be cached. 01:04:25.550 --> 01:04:27.360 Could I just use this thing as a checkpoint? 01:04:28.080 --> 01:04:30.700 I can autosave as my code runs. 01:04:30.750 --> 01:04:31.260 Yeah, that's cool. 01:04:31.560 --> 01:04:31.640 Yeah. 01:04:31.840 --> 01:04:34.760 And again, doing this right is hard, 01:04:34.860 --> 01:04:36.540 because there's all sorts of weird Python types. 01:04:36.590 --> 01:04:38.080 And I mentioned the progress bar thing. 01:04:38.340 --> 01:04:40.700 And there's all sorts of things that we've 01:04:40.700 --> 01:04:41.660 got to be mindful of here. 01:04:42.420 --> 01:04:44.080 But if you're thinking about really, 01:04:44.860 --> 01:04:45.980 how would you use this in data science 01:04:46.090 --> 01:04:47.880 when you fetch a little bit of data and deal with it, 01:04:48.740 --> 01:04:50.740 to me, it is starting to feel more natural 01:04:50.940 --> 01:04:52.420 than thinking about cells in a notebook, 01:04:52.880 --> 01:04:53.780 maybe cache on that level. 01:04:53.940 --> 01:04:54.880 Yeah, that's pretty interesting. 01:04:55.440 --> 01:04:59.920 just sort of cascade them along as a hash of the hashes 01:05:00.120 --> 01:05:01.680 of the prior cells or-- 01:05:01.750 --> 01:05:04.420 Well, and this is where things become tricky, of course, 01:05:04.540 --> 01:05:05.980 because then, OK, I've got this one cell, 01:05:06.180 --> 01:05:07.620 and I change one function in it. 01:05:07.940 --> 01:05:09.540 Oh, if you're going to cache on the entire cell, 01:05:09.690 --> 01:05:10.820 oh, everything has to rerun. 01:05:10.910 --> 01:05:11.880 And then, oh, if you're not careful, 01:05:11.980 --> 01:05:13.080 your cache is going to be huge. 01:05:13.360 --> 01:05:16.300 So like, OK, how do you do this in a user-friendly way? 01:05:16.420 --> 01:05:17.100 There's all sorts of-- 01:05:17.470 --> 01:05:19.800 it sounds easier than it is the one thing I want to say. 01:05:19.820 --> 01:05:21.100 Yeah, well, I think you've got a better chance 01:05:21.280 --> 01:05:23.540 with Marima than with Jupyter, at least, 01:05:23.640 --> 01:05:25.100 because you have a dependency graph. 01:05:25.460 --> 01:05:28.460 So you can at least say, if this one is invalid, 01:05:28.960 --> 01:05:31.100 that means the following three are also invalid, 01:05:32.020 --> 01:05:33.420 being sort of propagated there. 01:05:33.880 --> 01:05:34.240 Totally. 01:05:34.690 --> 01:05:38.320 But I try to focus a little bit more on the user experience 01:05:38.500 --> 01:05:38.780 side of things. 01:05:38.810 --> 01:05:40.240 And one thing I've really learned from the notebook 01:05:40.480 --> 01:05:42.700 with the progress bar is just there 01:05:42.780 --> 01:05:44.440 were moments when I felt like, oh, I just 01:05:44.560 --> 01:05:45.980 want this entire thing to be automated. 01:05:46.140 --> 01:05:46.980 Don't make me think about this. 01:05:47.460 --> 01:05:49.240 And then there were moments where I thought, oh, it's really 01:05:49.400 --> 01:05:50.580 nice to have tight manual control. 01:05:51.000 --> 01:05:52.060 How do I provide you with both? 01:05:53.420 --> 01:05:53.780 Yeah. 01:05:54.440 --> 01:05:54.980 That's quite tricky. 01:05:55.380 --> 01:05:57.360 But it is a dream, so that's something to keep in mind. 01:05:57.540 --> 01:06:00.700 Yeah, maybe someday there'll be a turn on cash flow checkbox 01:06:01.160 --> 01:06:01.300 something. 01:06:01.740 --> 01:06:03.960 Yeah, or well, at least till then, 01:06:04.680 --> 01:06:06.980 I do think having something that works on disk instead of memory 01:06:07.220 --> 01:06:09.720 these days is also just a boon. 01:06:10.200 --> 01:06:10.380 Right. 01:06:10.480 --> 01:06:12.320 So this works in data science notebooks. 01:06:12.760 --> 01:06:13.840 It works in web apps. 01:06:13.920 --> 01:06:15.080 It works in little TUIs. 01:06:15.340 --> 01:06:16.720 It doesn't care. 01:06:16.920 --> 01:06:18.080 It works with LLMs. 01:06:19.200 --> 01:06:20.780 And if you have a kind of-- 01:06:20.960 --> 01:06:22.580 actually, similar to your set of, 01:06:22.820 --> 01:06:25.180 If you have multiple processes with your web app running on one VM, 01:06:25.340 --> 01:06:27.100 if you have one big VM that you share with your colleagues, 01:06:27.180 --> 01:06:28.880 you can also just share the cache, actually. 01:06:29.240 --> 01:06:29.940 Yeah, that's true. 01:06:30.300 --> 01:06:32.360 So that's the reason you can't just point the same file, yeah. 01:06:32.680 --> 01:06:34.380 Yeah, and especially if you're doing like big experiments 01:06:34.500 --> 01:06:35.980 like grid search results or stuff like that, 01:06:36.120 --> 01:06:38.660 that you really don't want to recalculate the big compute thing. 01:06:39.640 --> 01:06:40.880 That's actually not too unreasonable. 01:06:41.380 --> 01:06:41.880 Yeah, that's cool. 01:06:42.339 --> 01:06:44.420 Yeah, if you have a shared Jupyter server. 01:06:44.820 --> 01:06:46.720 Yeah, and a bunch of universities have that, right? 01:06:47.099 --> 01:06:47.640 Yeah, exactly. 01:06:48.740 --> 01:06:50.760 I don't want to go down this path because we're basically out of time. 01:06:51.480 --> 01:06:52.320 I respect your time. 01:06:52.720 --> 01:06:57.360 However, I do think there's a whole interesting conversation to be had about like, how do you 01:06:57.510 --> 01:07:01.500 choose the right key for your, what goes into the cache? 01:07:01.730 --> 01:07:05.700 Because you can end up with staleness really easy if something changes. 01:07:05.810 --> 01:07:11.340 But if you incorporate the right stuff, you might never run into stale data problems because, 01:07:11.880 --> 01:07:14.980 you know, like for example, I talked about the YouTube ID. 01:07:15.340 --> 01:07:20.340 Basically the cache key is something like episode, episode data, episode YouTube thing, 01:07:20.700 --> 01:07:25.320 colon YouTube, colon the hash of the show notes, right? 01:07:25.460 --> 01:07:27.000 Something like that, where there's no way 01:07:27.070 --> 01:07:28.660 that the show notes are gonna change 01:07:28.790 --> 01:07:31.120 and I'll get the old data because guess what? 01:07:32.659 --> 01:07:34.320 It's constructed out of the source, right? 01:07:34.460 --> 01:07:34.940 Things like that. 01:07:35.660 --> 01:07:38.960 There's probably a lot, especially like in your notebook side. 01:07:39.240 --> 01:07:40.860 There's a lot to consider there, I think. 01:07:41.060 --> 01:07:43.360 - Yeah, I mean, so I remember this one thing 01:07:43.460 --> 01:07:45.820 with the course where I still wanted it to be cached, 01:07:46.260 --> 01:07:48.359 but I wanted to have like text goes in 01:07:48.380 --> 01:07:50.880 And then five responses from the LLM go out. 01:07:51.280 --> 01:07:52.740 And the way you solve that is you just add another key. 01:07:53.180 --> 01:07:54.840 But you have to be mindful of the cache key. 01:07:55.140 --> 01:07:56.920 And you can-- oh, and you can use tuples, by the way. 01:07:57.000 --> 01:07:58.600 That's also something you can totally use as a cache key. 01:07:58.780 --> 01:07:59.260 Right, right. 01:07:59.720 --> 01:08:01.040 So that was easy to fix. 01:08:01.140 --> 01:08:02.240 It's just that you have to be mindful. 01:08:02.740 --> 01:08:04.920 Yeah, that's kind of-- I want to give a quick shout out to that. 01:08:05.420 --> 01:08:06.440 I want to leave-- 01:08:06.920 --> 01:08:08.480 I don't want to leave on a sour note. 01:08:08.560 --> 01:08:11.540 But I think it's necessary to give this shout out, 01:08:11.780 --> 01:08:14.180 or this like call this out, rather, is the way I should say it, 01:08:14.440 --> 01:08:15.780 is I think this project is awesome. 01:08:15.980 --> 01:08:16.540 You think it's awesome. 01:08:17.040 --> 01:08:19.240 Honestly, I think it doesn't need really very much. 01:08:19.700 --> 01:08:22.200 But if you look at last updated, 01:08:22.620 --> 01:08:23.880 if you look at the last updated date, 01:08:24.120 --> 01:08:26.920 it's really, it hasn't got a lot of attention 01:08:27.120 --> 01:08:29.180 in the last six months or something like that. 01:08:29.339 --> 01:08:31.220 Yeah, and if I look at PyPI, 01:08:31.420 --> 01:08:33.319 the last release was 2023, 01:08:34.000 --> 01:08:35.400 which, yeah, a year and a half ago. 01:08:36.200 --> 01:08:37.000 Yeah, and it's okay to, 01:08:37.319 --> 01:08:40.080 I would like to say that it's okay for things to be done. 01:08:40.420 --> 01:08:42.640 It doesn't have to, things don't have to change, 01:08:43.000 --> 01:08:44.839 but there's also a decent amount of, 01:08:45.339 --> 01:08:48.100 like conversations on the issues and they haven't, you know, 01:08:48.240 --> 01:08:52.420 like a couple of days ago, actually someone asked about this, but you know, 01:08:52.670 --> 01:08:56.259 the last change, I believe the guy Grant who works on it, 01:08:57.400 --> 01:09:02.319 started working at OpenAI about the time changes stopped going on. 01:09:02.500 --> 01:09:06.480 I'm not entirely sure. I feel like I could be confused with another project. 01:09:06.720 --> 01:09:09.920 So Grant, if that's not true, I apologize, but I think that it is. 01:09:11.060 --> 01:09:12.540 Pretty sure that is. Do we have LinkedIn? 01:09:13.080 --> 01:09:16.520 I mean, I am comfortable stating is-- 01:09:16.650 --> 01:09:17.240 let me put it this way. 01:09:18.260 --> 01:09:20.980 Yes, my colleague might make a different caching mechanism. 01:09:21.150 --> 01:09:22.819 And yes, I might use that at some point. 01:09:22.960 --> 01:09:24.600 But at least for where I'm at right now, 01:09:25.000 --> 01:09:27.900 this cache needs to break vividly in front of my face 01:09:28.120 --> 01:09:29.520 for me to consider not using it. 01:09:30.200 --> 01:09:33.200 Because it does feel like it's done in a really good way. 01:09:33.380 --> 01:09:35.140 The main thing that needs to happen, I think, 01:09:35.400 --> 01:09:38.580 functionally to make sure this doesn't get deprecated too 01:09:38.799 --> 01:09:40.580 badly is just you got to update the Python version. 01:09:40.779 --> 01:09:41.859 When a new Python version comes out, 01:09:42.000 --> 01:09:43.299 you got to update PyPI to confirm, 01:09:43.350 --> 01:09:44.940 like, OK, we do support this Python version. 01:09:45.540 --> 01:09:47.560 But I mean, most of the-- 01:09:48.359 --> 01:09:49.819 if you look at the area that needs to be covered, 01:09:49.910 --> 01:09:51.440 a lot of that has been covered by SQLite. 01:09:51.470 --> 01:09:54.660 And that thing is definitely still being maintained. 01:09:54.980 --> 01:09:56.240 It's getting mega maintained. 01:09:56.500 --> 01:09:56.800 That's right. 01:09:57.020 --> 01:09:59.060 So I also don't see the problem. 01:09:59.800 --> 01:10:01.140 I'm not going to not use it. 01:10:01.660 --> 01:10:03.720 I just want to put it out there on the radar for people 01:10:04.140 --> 01:10:06.760 for whom my co-look is to go, oh, Michael and Vincent 01:10:06.810 --> 01:10:08.100 were so psyched, and I started to use this. 01:10:08.170 --> 01:10:10.360 And now I'm really disappointed because of whatever. 01:10:10.500 --> 01:10:10.980 I saw this. 01:10:11.900 --> 01:10:14.220 I mean, the only real doom scenario I can come up with 01:10:14.280 --> 01:10:16.500 is if SQLite made like a breaking change. 01:10:16.800 --> 01:10:18.060 That's the only thing I can kind of come up with. 01:10:18.160 --> 01:10:19.760 But the odds of that seem very low. 01:10:20.080 --> 01:10:21.340 Yeah, and it's on GitHub. 01:10:21.420 --> 01:10:21.980 You can fork it. 01:10:22.100 --> 01:10:22.520 I fork it. 01:10:22.580 --> 01:10:23.160 Exactly, exactly. 01:10:23.680 --> 01:10:27.140 So, no, I definitely am still super excited about it. 01:10:27.200 --> 01:10:28.760 I just want to make sure that we put that out there. 01:10:29.080 --> 01:10:32.380 I'd intended to talk about it sooner in the conversation, 01:10:32.620 --> 01:10:33.100 but you know what? 01:10:33.160 --> 01:10:33.880 We were just so excited. 01:10:34.100 --> 01:10:34.360 Yeah, no. 01:10:36.300 --> 01:10:38.760 This is definitely in my top five favorite Python libraries 01:10:38.960 --> 01:10:39.900 outside of the standard lib. 01:10:40.220 --> 01:10:40.440 Awesome. 01:10:40.780 --> 01:10:43.140 Yeah, I've really, really have gotten awesome results out of it as well. 01:10:43.460 --> 01:10:45.800 So remember the way we opened the show and you talked about this, 01:10:45.950 --> 01:10:48.600 like when we talked about the LLM building block stuff 01:10:48.670 --> 01:10:50.360 on the previous time you were on the show, 01:10:50.520 --> 01:10:52.880 it was like, oh, we better not go too deep on this, 01:10:53.280 --> 01:10:55.840 even though we're both so excited because it's going to derail the show. 01:10:56.220 --> 01:10:58.220 We're now one hour and 15 minutes into it. 01:10:58.680 --> 01:11:00.200 We kind of cut ourselves off. 01:11:00.230 --> 01:11:01.500 I think that was accurate. 01:11:02.280 --> 01:11:05.040 Yeah, I mean, you get two dads making dad jokes 01:11:05.090 --> 01:11:06.340 and riffing on tools they both like. 01:11:06.780 --> 01:11:09.320 It's bound to exceed a barbecue. 01:11:10.680 --> 01:11:11.240 Yes, I know. 01:11:11.390 --> 01:11:14.580 I wonder what would happen if sometime we just removed the time limit, 01:11:15.020 --> 01:11:17.400 just got real comfortable and just riffed on something. 01:11:17.410 --> 01:11:18.040 It could be hours. 01:11:18.050 --> 01:11:18.500 It would be fun. 01:11:18.700 --> 01:11:19.840 But maybe not today. 01:11:20.140 --> 01:11:21.540 Two-hour live streams exist, Michael. 01:11:22.320 --> 01:11:22.780 I know. 01:11:24.200 --> 01:11:25.980 I've listened to some podcasts at over three hours. 01:11:26.160 --> 01:11:27.020 I'm like, how is this still going? 01:11:27.160 --> 01:11:27.580 But you know what? 01:11:28.020 --> 01:11:28.120 Yeah. 01:11:28.320 --> 01:11:28.800 It's all good. 01:11:29.380 --> 01:11:31.820 But yeah, this is a good point in time. 01:11:32.740 --> 01:11:33.400 We're both excited. 01:11:33.640 --> 01:11:34.140 That's the summary. 01:11:34.840 --> 01:11:35.480 That is the summary. 01:11:35.670 --> 01:11:39.000 And I think I'm going to let you have the final word on this topic here. 01:11:39.240 --> 01:11:42.620 like maybe speak to people just about caching in general 01:11:42.760 --> 01:11:45.160 and disk cache in particular as we close it out? 01:11:45.320 --> 01:11:48.380 I mean, I guess the main thing that I learned 01:11:48.520 --> 01:11:49.900 with the whole caching thing in the last couple of years, 01:11:49.980 --> 01:11:51.780 I always thought it was kind of like a web thing. 01:11:52.000 --> 01:11:53.700 Like, oh, you know, front page of Reddit, 01:11:54.020 --> 01:11:54.920 that thing has to be cached. 01:11:55.080 --> 01:11:55.800 That's the way you think about it. 01:11:55.800 --> 01:11:55.960 Yeah, of course. 01:11:56.700 --> 01:11:58.320 And thinking about it too much that way 01:11:58.600 --> 01:12:00.260 totally blocked me from considering, like, oh, 01:12:00.400 --> 01:12:02.120 but if you do stuff in notebooks and data science land, 01:12:02.220 --> 01:12:02.980 then you need this as well. 01:12:03.780 --> 01:12:06.820 And I think there's actually a little emerging discovery 01:12:07.200 --> 01:12:09.200 phenomenon happening where people that do things 01:12:09.220 --> 01:12:11.380 LLM's at some point go like, oh, I need a cache. 01:12:11.900 --> 01:12:12.420 And then, oh. 01:12:15.040 --> 01:12:16.800 So that's the main thing I suppose I want to say. 01:12:16.860 --> 01:12:18.160 Like, even if you're doing more data stuff, 01:12:18.300 --> 01:12:19.660 like give this disk cache thing a try. 01:12:19.800 --> 01:12:20.160 It's just good. 01:12:20.700 --> 01:12:23.280 Yeah, it's so easy to adopt and try out. 01:12:23.460 --> 01:12:24.420 Like, you can throw it in there. 01:12:24.760 --> 01:12:25.360 Just add a decorator. 01:12:26.300 --> 01:12:27.260 Exactly, see what you get. 01:12:27.760 --> 01:12:28.300 See what you get. 01:12:28.800 --> 01:12:29.900 All right, Vincent, welcome. 01:12:30.300 --> 01:12:31.300 Oh, thank you for coming back. 01:12:31.360 --> 01:12:32.140 I really appreciate it. 01:12:32.720 --> 01:12:33.340 Thanks for having me. 01:12:33.480 --> 01:12:34.240 Always good to talk to you. 01:12:34.520 --> 01:12:34.580 Yeah. 01:12:34.900 --> 01:12:36.680 And yeah, see you next time when we, again, 01:12:36.900 --> 01:12:38.060 find out there's a cool Python library. 01:12:38.820 --> 01:12:41.680 Yeah, that's going to be the three-hour episode. 01:12:41.920 --> 01:12:42.380 Watch out, y'all. 01:12:43.600 --> 01:12:44.080 Have a good one. 01:12:44.460 --> 01:12:44.560 Later. 01:12:45.840 --> 01:12:47.980 This has been another episode of Talk Python To Me. 01:12:48.360 --> 01:12:49.080 Thank you to our sponsors. 01:12:49.320 --> 01:12:50.620 Be sure to check out what they're offering. 01:12:50.780 --> 01:12:52.140 It really helps support the show. 01:12:52.880 --> 01:12:54.740 If you or your team needs to learn Python, 01:12:54.980 --> 01:12:58.360 we have over 270 hours of beginner and advanced courses 01:12:58.660 --> 01:13:02.100 on topics ranging from complete beginners to async code, 01:13:02.240 --> 01:13:05.020 Flask, Django, HTMX, and even LLMs. 01:13:05.240 --> 01:13:07.520 Best of all, there's no subscription in sight. 01:13:08.160 --> 01:13:09.840 browse the catalog at talkpython.fm. 01:13:10.540 --> 01:13:12.520 And if you're not already subscribed to the show 01:13:12.720 --> 01:13:13.920 on your favorite podcast player, 01:13:14.560 --> 01:13:15.200 what are you waiting for? 01:13:15.880 --> 01:13:17.640 Just search for Python in your podcast player. 01:13:17.800 --> 01:13:18.620 We should be right at the top. 01:13:19.040 --> 01:13:20.580 If you enjoyed that geeky rap song, 01:13:20.670 --> 01:13:21.880 you can download the full track. 01:13:22.030 --> 01:13:23.920 The link is actually in your podcast blur show notes. 01:13:24.760 --> 01:13:26.080 This is your host, Michael Kennedy. 01:13:26.480 --> 01:13:27.540 Thank you so much for listening. 01:13:27.770 --> 01:13:28.540 I really appreciate it. 01:13:28.980 --> 01:13:29.680 I'll see you next time. 01:13:41.580 --> 01:13:44.380 I'm out.