WEBVTT 00:00:00.001 --> 00:00:05.620 There hasn't been a boom like the AI boom since the dot-com days, and it may look like a space 00:00:05.620 --> 00:00:11.280 destined to be controlled by a couple of tech giants. But Ines Montani thinks open source will 00:00:11.280 --> 00:00:16.120 play an important role in the future of AI. I hope you join us for this excellent conversation 00:00:16.120 --> 00:00:23.160 about the future of AI and open source. This is Talk Python To Me, episode 465, recorded May 8th, 00:00:23.160 --> 00:00:23.820 2024. 00:00:23.820 --> 00:00:25.860 Are you ready for your host? 00:00:26.800 --> 00:00:31.820 You're listening to Michael Kennedy on Talk Python To Me. Live from Portland, Oregon, 00:00:31.820 --> 00:00:33.920 and this segment was made with Python. 00:00:33.920 --> 00:00:42.360 Welcome to Talk Python To Me, a weekly podcast on Python. This is your host, Michael Kennedy. 00:00:42.360 --> 00:00:47.480 Follow me on Mastodon, where I'm @mkennedy, and follow the podcast using @talkpython, 00:00:47.480 --> 00:00:53.420 both on fosstodon.org. Keep up with the show and listen to over seven years of past episodes 00:00:53.420 --> 00:00:59.740 at talkpython.fm. We've started streaming most of our episodes live on YouTube. Subscribe to our 00:00:59.740 --> 00:01:05.720 YouTube channel over at talkpython.fm/youtube to get notified about upcoming shows and be part of 00:01:05.720 --> 00:01:10.440 that episode. This episode is brought to you by Sentry. Don't let those errors go unnoticed. Use 00:01:10.440 --> 00:01:16.860 Sentry like we do here at Talk Python. Sign up at talkpython.fm/sentry. And it's brought to you by 00:01:16.860 --> 00:01:22.640 Pork Bun. Launching a successful project involves many decisions, not the least of which is choosing a 00:01:22.640 --> 00:01:29.220 domain name. Get a .app, .dev, or .food domain name at Pork Bun for just $1 for the first year at 00:01:29.220 --> 00:01:36.300 Talk Python.fm slash Pork Bun. Before we jump into the show, a quick announcement. Over at Talk Python, we just 00:01:36.300 --> 00:01:42.420 launched a new course, and it's super relevant to today's topic. The course is called Getting Started with 00:01:42.420 --> 00:01:48.680 NLP and spaCy. It was created by Vincent Warmerdom, who has spent time working directly on spaCy at 00:01:48.680 --> 00:01:53.800 Explosion AI. The course is a really fun exploration of what you can do with spaCy for processing and 00:01:53.800 --> 00:02:00.760 understanding text data. And Vincent uses the past nine years of Talk Python transcripts as the core data for the 00:02:00.760 --> 00:02:06.820 course. If you have text data you need to understand, check out the course today at talkpython.fm/spaCy. 00:02:06.820 --> 00:02:12.340 The link is in your podcast player show notes. Now on to more AI and spaCy with Ines. 00:02:12.340 --> 00:02:14.400 Ines, welcome back to Talk Python. 00:02:14.400 --> 00:02:16.720 Thanks for having me. Yeah, thanks for having me back again. 00:02:16.720 --> 00:02:19.120 You're one of my favorite guests. It's always awesome to have you. 00:02:19.120 --> 00:02:20.740 Oh, thanks. You're my favorite podcast. 00:02:20.740 --> 00:02:28.360 Thank you. We have some really cool things to talk about. spaCy, some of course, but also more broadly, 00:02:28.360 --> 00:02:35.900 we're going to talk about just LLMs and AIs and open source and business models and even monopolies. 00:02:35.900 --> 00:02:42.720 I'm going to cover a lot of things. You've been kind of making a bit of a roadshow, a tour around 00:02:42.720 --> 00:02:45.880 much of Europe talking about some of these ideas, right? 00:02:45.880 --> 00:02:51.760 Yeah, I've gotten invited to quite a few conferences. And I feel like this is like after COVID, the first 00:02:51.760 --> 00:02:56.820 like proper, proper year again that I'm like traveling for conferences. And I was like, why not? 00:02:56.820 --> 00:03:01.540 And I think especially now that so much is happening in the AI space, I think it's actually really nice 00:03:01.540 --> 00:03:06.820 to go to these conferences and connect with actual developers. Because, you know, if you're just 00:03:06.820 --> 00:03:11.440 sitting on the internet and you're scrolling, I don't know, LinkedIn, and sometimes it can be really 00:03:11.440 --> 00:03:16.320 hard to tell like what people are really thinking and what's, do people really believe some of these 00:03:16.320 --> 00:03:22.620 hot, weird takes that people are putting out there. So yeah, it was very, very nice to talk about 00:03:22.620 --> 00:03:28.020 some of these ideas, get them checked against what developers think. So yeah, it's been really cool. 00:03:28.380 --> 00:03:33.480 And there's more to come. Yeah, I know. I'll be traveling again later this month to Italy for 00:03:33.480 --> 00:03:37.660 PyCon for my first time, then PyData London, and who knows what else. 00:03:37.660 --> 00:03:42.040 If you must go to Italy and London, what terrible places to spend time in, huh? 00:03:42.040 --> 00:03:46.860 I'm definitely very open for tips, especially for Italy, for Florence. I've never, I've never been, 00:03:46.860 --> 00:03:49.040 I've never been to Italy ever. So. 00:03:49.040 --> 00:03:54.040 Oh, awesome. I've been to Rome, but that's it. And so I don't have any tips, but London is, 00:03:54.040 --> 00:03:55.540 London is also fantastic. Yeah. 00:03:55.540 --> 00:04:00.960 Cool. So people can check you out, but maybe, I think, do you have a list of that publicly where 00:04:00.960 --> 00:04:03.840 people can see some of your talks? We can put that in the show notes. Yeah. 00:04:03.840 --> 00:04:08.160 Yeah. It's on my website. And then also on the explosion side of our company, we've actually 00:04:08.160 --> 00:04:13.740 added an events page because it came up a lot that like either me, Matt, people from our team 00:04:13.740 --> 00:04:17.880 giving talks. And so we thought like, Hey, let's, and podcasts as well. So let me be collecting 00:04:17.880 --> 00:04:21.200 everything on one page, all the stuff we're doing, which is kind of fun. 00:04:21.260 --> 00:04:26.040 Which is quite a bit actually, for sure. Yeah. Yeah. Well, I know many people know you, 00:04:26.040 --> 00:04:33.280 but let's just talk a little bit about spaCy, Explosion, Prodigy, the stuff that you guys are 00:04:33.280 --> 00:04:35.260 doing to give people a sense of where you're coming from. 00:04:35.260 --> 00:04:41.420 Yeah. So we're basically, we're an open source company and we build developer tools for AI, 00:04:41.420 --> 00:04:46.040 natural language processing specifically. So, you know, you're working with lots of text, 00:04:46.040 --> 00:04:51.180 you want to analyze it beyond just looking for keywords. That's kind of where we started and 00:04:51.180 --> 00:04:56.580 what we've always been focusing on. So spaCy is probably what we're mostly known for, which is a 00:04:56.580 --> 00:05:03.120 popular open source library for really what we call industrial strength NLP. So built for production, 00:05:03.120 --> 00:05:09.740 it's fast, it's efficient. We've put a lot of work into, you know, having good usable, user-friendly, 00:05:09.740 --> 00:05:14.660 developer-friendly APIs. Actually. Yeah. I always set an example. I always like to show in my talks, 00:05:14.660 --> 00:05:19.720 a nice side effect that we never anticipated like that is that ChatGPT and similar models are 00:05:19.720 --> 00:05:25.680 actually pretty good at writing spaCy code because we put a lot of work into all of this stuff like 00:05:25.680 --> 00:05:31.720 backwards compatibility, not breaking people's code all the time, stuff like that. But that happens to 00:05:31.720 --> 00:05:36.860 really help at least for now with these models. It's really nice. It's a good thing you've done to 00:05:36.860 --> 00:05:43.540 make it a really stable API that people can trust. But is it weird to see LLMs talking about stuff 00:05:43.540 --> 00:05:49.280 we all created? It's kind of, it's funny in some way. I mean, it's also, there is this whole other 00:05:49.280 --> 00:05:56.220 site to, you know, doing user support for detecting clearly auto-generated code because 00:05:56.220 --> 00:06:01.360 for spaCy, these models are pretty good for Prodigy, which is our annotation tool, which is also scriptable 00:06:01.360 --> 00:06:07.440 in Python. It's a bit less precise and it hallucinates a lot because there's just less code online and on 00:06:07.440 --> 00:06:12.300 GitHub. So we sometimes get like support requests where like users post their code and we're like, 00:06:12.300 --> 00:06:17.080 this is so strange. How did you find these APIs? They don't exist. And then we're like, ah, this was 00:06:17.080 --> 00:06:23.380 auto-generated. Oh, okay. So that was a very new experience. And also it's, you know, I think everyone 00:06:23.380 --> 00:06:28.220 who publishes online deals with that, but like, it's very frustrating to see all these like auto-generated 00:06:28.220 --> 00:06:34.900 posts that like look like tech posts, but are completely bullshit and completely wrong. Like I saw 00:06:34.900 --> 00:06:41.660 with something on spaCy LLM, which is our extension for integrating large language models into spaCy. 00:06:41.660 --> 00:06:47.060 And they're like some blog posts that look like they're tech blog posts, but they're like completely 00:06:47.060 --> 00:06:52.720 hallucinated. And it's very, very strange to see that about like your own software. And also it 00:06:52.720 --> 00:06:56.860 frustrates me because that stuff is going to feed into the next generation of these models. Right. And 00:06:56.860 --> 00:07:03.260 then they will stop being so good at this because they're full of stuff that they've generated 00:07:03.260 --> 00:07:09.840 themselves on like APIs and things that don't even exist. Yeah. It's just going to cycle around and 00:07:09.840 --> 00:07:14.160 around and around until it just gets worse every time. And then that's interesting. Like it's very 00:07:14.160 --> 00:07:19.240 interesting to see what's going on and where, where these things lead. It is, you know, I just had a 00:07:19.240 --> 00:07:26.000 thought I was, you know, open AI and some of these different companies are, are doing work to try to 00:07:26.000 --> 00:07:31.840 detect AI generated images. And I imagine AI generated content. Yeah. When I heard that, 00:07:31.840 --> 00:07:35.440 my thought was like, well, that's just because they kind of want to be good citizens. And I want to put 00:07:35.440 --> 00:07:40.980 little labels and say this, what if it's just so they don't ingest it twice? I think that's definitely, 00:07:40.980 --> 00:07:46.160 and I mean, in a way it's also good because, you know, we will, it would make these models worse. 00:07:46.160 --> 00:07:50.960 And so from like, you know, from a product perspective for a company like open AI, that's 00:07:50.960 --> 00:07:56.980 definitely very useful. And, I think also, you know, commercially, I think there's definitely, 00:07:56.980 --> 00:08:03.700 you know, a big market in that also for like social networks and stuff to, to detect are these real 00:08:03.700 --> 00:08:08.780 images, are these deep fakes, any money in that too. So it's not, I don't think it's just, yeah, 00:08:08.780 --> 00:08:13.680 being good citizens, but like this, there's a clear product motivated thing, which is fine, 00:08:13.680 --> 00:08:17.940 you know, for a company. Yeah. It is fine. I just, I never really thought about it. Of course. 00:08:17.940 --> 00:08:25.200 Of course. Yeah. Do you think we'll get to some point where in food or art, you hear about artisanal 00:08:25.200 --> 00:08:31.300 handcrafted pizza or, you know, whatever, will there'll be artisanal human created tech that has 00:08:31.300 --> 00:08:36.880 got a special, special flavor to it? Like this was created with no AI. Look at how cool this site is 00:08:36.880 --> 00:08:41.320 or whatever. I think it's already something that like, you see, like, I don't know which product this 00:08:41.320 --> 00:08:45.840 was, but I saw there was some ad campaign. I think it might've been a language learning app 00:08:45.840 --> 00:08:51.320 or something else where they really like put that into one of their like marketing claims. Like, Hey, 00:08:51.320 --> 00:08:57.080 it's not AI generated. We don't use AI. It's actually real in humans because it seems to be, 00:08:57.080 --> 00:09:01.180 you know, what people want. They want, you know, they want to have at least that feeling. So 00:09:01.180 --> 00:09:04.860 I definitely think there's an appeal of that also going forward. 00:09:05.060 --> 00:09:10.300 The whole LLM and AI stuff. It's just, it's permeated culture so much. I was at the motorcycle 00:09:10.300 --> 00:09:15.960 shop yesterday talking to a guy who was a motorcycle salesman. And he was like, do you think that AI 00:09:15.960 --> 00:09:20.000 is going to change how software developers work? Do you think they're still going to be relevant? I'm 00:09:20.000 --> 00:09:24.140 like, you're a sale, you're a motorcycle sales guy. It's amazing. how that you're like, 00:09:24.200 --> 00:09:28.880 really this tuned into it. Right. And that you're like, but you know, you think it's maybe just a 00:09:28.880 --> 00:09:32.700 little echo chamber of us talking about, but it seems to be like these kinds of conversations are 00:09:32.700 --> 00:09:34.280 more broad than maybe you would have guessed. 00:09:34.280 --> 00:09:38.880 Chatty PT definitely, you know, brought the conversation into the mainstream, but on the 00:09:38.880 --> 00:09:43.420 other hand, on the plus side, it also means it makes it a lot easier for us kind of to explain 00:09:43.420 --> 00:09:48.620 our work because people have at least heard of this. And I think it's also for developers working in 00:09:48.620 --> 00:09:53.220 teams. Like on the one hand, it can maybe be frustrating to do this expectation management 00:09:53.220 --> 00:09:57.600 because, you know, you have management who just came back from some fancy conference and 00:09:57.600 --> 00:10:03.880 got sold on like, Ooh, we need like some chatbot or LLM. It's kind of the chatbot hype all over again 00:10:03.880 --> 00:10:07.740 that we already had in 2015 or so. That can be frustrating. 00:10:07.740 --> 00:10:10.800 I forgot all about that. Those were going to be so important. And now what are they doing? 00:10:10.800 --> 00:10:14.300 Nothing. Yeah. But yeah, but it's, it's like, yeah, I see a lot of parallels. It's like, 00:10:14.300 --> 00:10:18.140 if you look at kind of the hype cycle and people's expectations and expectation 00:10:18.140 --> 00:10:22.600 management, it's kind of the same thing in a lot of ways, only that like it actually, 00:10:22.600 --> 00:10:26.420 a lot of parts actually kind of work now, which we didn't really have before. 00:10:26.420 --> 00:10:30.360 Yeah. But yeah, it also means for teams and developers that they at least have some more 00:10:30.360 --> 00:10:34.920 funding available and resources that they can work with. Because I feel like before that happened, 00:10:34.920 --> 00:10:40.060 it looked like that, you know, companies are really cutting their budgets, all these exploratory AI 00:10:40.060 --> 00:10:45.720 projects. They all got cut. It was quite frustrating for a lot of developers. And now at least, 00:10:45.720 --> 00:10:49.920 you know, it means they can actually work again, even though they also have to kind of 00:10:49.920 --> 00:10:56.640 manage the expectations and like work around some of the very wild ideas that companies might have at 00:10:56.640 --> 00:10:57.000 the moment. 00:10:57.000 --> 00:11:01.740 Absolutely. Now, one of the things that's really important, and we're going to get to here, 00:11:01.740 --> 00:11:05.560 give you a chance to give a shout out to the other thing that you all have is, 00:11:05.560 --> 00:11:11.240 is how do you teach these things? Information and how do you get them to know things and, 00:11:11.340 --> 00:11:15.880 and so on. And, you know, for the spaCy world, you have Prodigy and maybe give a shout out to 00:11:15.880 --> 00:11:19.360 Prodigy Teams. That's something you just are just announcing, right? 00:11:19.360 --> 00:11:23.420 Yeah. So that's currently in beta. It's something we've been working on. So the idea of Prodigy has 00:11:23.420 --> 00:11:28.820 always been, hey, you know, support a spaCy, also other libraries. And how can we, yeah, 00:11:28.820 --> 00:11:34.240 how can we make the training and data collection process more efficient or so efficient that companies 00:11:34.240 --> 00:11:39.680 can in-house that process? Like whether it's creating training data, creating evaluation data, 00:11:39.680 --> 00:11:44.420 like even if what you're doing is completely generative and you have a model that does it well, 00:11:44.420 --> 00:11:49.040 you need some examples and some data where you know the answer. And often that's a structured 00:11:49.040 --> 00:11:53.760 data format. So you need to create that. We've, you know, really seen that like outsourcing that 00:11:53.760 --> 00:11:58.520 doesn't work very well. And also now with the newer technologies, you, with transfer learning, 00:11:58.520 --> 00:12:04.400 you don't need millions of examples anymore. This whole, like this big, big data idea for 00:12:04.400 --> 00:12:10.940 task-specific stuff is really dead, dead in a lot of ways. So Prodigy is a developer tool that you can 00:12:10.940 --> 00:12:17.940 script in Python and that makes it easy to really collect this kind of structured data on text images 00:12:17.940 --> 00:12:23.660 and so on. And then Prodigy Teams, that has been a very ambitious project. We've really been, 00:12:23.660 --> 00:12:28.360 we've wanted to ship this a long time ago already, but it's been very challenging because 00:12:28.360 --> 00:12:32.580 we basically want to bring also a lot of these ideas that probably we're going to talk about today a 00:12:32.580 --> 00:12:39.620 bit into the cloud while retaining the data privacy. And so you'll be able to run your own cluster on 00:12:39.620 --> 00:12:44.200 your own infrastructure that has the data that's scriptable in Python. So you can kind of script 00:12:44.200 --> 00:12:50.520 the SaaS app in Python, which is very cool, which you normally can't do. Your data never leaves our 00:12:50.520 --> 00:12:57.060 service. And you can basically also use these workflows like distillation, where you start out with a super easy 00:12:57.060 --> 00:13:05.940 prototype that might use LAMA or some other models to add GPT, GPT-4. Then you benchmark that, see how it does. 00:13:05.940 --> 00:13:12.100 And then you collect some data until you can beat that inaccuracy and have a task-specific model that really 00:13:12.100 --> 00:13:16.900 only does the one extraction you're interested in. And that model can be tiny. Like we've had users 00:13:16.900 --> 00:13:23.020 build models that are under 10 megabytes. Like that's, that is pretty crazy to think about these days. 00:13:23.240 --> 00:13:29.340 And that, that run like 20 times faster. They're entirely private. You can, you know, you don't need 00:13:29.340 --> 00:13:34.040 like tons of compute to run them. And that's kind of really one of the workflows of the future that we 00:13:34.040 --> 00:13:40.140 see as very promising. And it's also, people are often surprised how little task-specific data you 00:13:40.140 --> 00:13:46.220 actually need to say, beat GPT-4 inaccuracy. It's not as much as people think. And it's totally, 00:13:46.220 --> 00:13:51.840 you know, in a single workday, you could often do it. The main idea we've been thinking about a lot is 00:13:51.840 --> 00:13:56.280 basically how can we make that workflow better and more user-friendly, even for people who are, 00:13:56.280 --> 00:13:58.540 who don't have an extensive machine learning background. 00:13:58.540 --> 00:13:58.860 Right. 00:13:58.860 --> 00:14:04.100 Because one thing that like prompting an LLM or prompting a generative model has is that it's 00:14:04.100 --> 00:14:09.920 a very low barrier to entry. And it's very, very use the UX is very good. You just type in a question, 00:14:09.920 --> 00:14:14.580 you talk to it the way you would talk to a human, and that's easy to get started with the workflow. 00:14:14.580 --> 00:14:19.160 That's a bit more involved. Yes. Machine learning developers know how to do that and they know 00:14:19.160 --> 00:14:26.000 when to do it, but it's not as accessible to people who don't have all of that experience. And so that's 00:14:26.000 --> 00:14:28.440 kind of the underlying thing that we're trying to solve. 00:14:29.860 --> 00:14:42.100 This portion of Talk Python To Me is brought to you by Sentry. In the last episode, I told you about how we use Sentry to solve a tricky problem. This time, I want to talk about making your front end and back end code work more tightly together. 00:14:42.700 --> 00:14:57.540 If you're having a hard time getting a complete picture of how your app is working and how requests flow from the front end JavaScript app back to your Python services down into database calls for errors and performance, you should definitely check out Sentry's distributed tracing. 00:14:57.540 --> 00:15:07.380 With distributed tracing, you'll be able to track your software's performance, measure metrics like throughput and latency, and display the impact of errors across multiple systems. 00:15:07.380 --> 00:15:16.320 Distributed tracing makes Sentry a more complete performance monitoring solution, helping you diagnose problems and measure your application's overall health more quickly. 00:15:16.320 --> 00:15:29.360 Tracing in Sentry provides insights such as what occurred for a specific event or issue, the conditions that cause bottlenecks or latency issues, and the endpoints and operations that consume the most time. 00:15:29.360 --> 00:15:32.920 Help your front end and back end teams work seamlessly together. 00:15:32.920 --> 00:15:38.560 Check out Sentry's distributed tracing at talkpython.fm/sentry-trace. 00:15:38.560 --> 00:15:42.400 That's talkpython.fm/sentry-trace. 00:15:42.400 --> 00:15:50.640 And when you sign up, please use our code TALKPYTHON, all caps, no spaces, to get more features and let them know that you came from us. 00:15:50.640 --> 00:15:52.760 Thank you to Sentry for supporting the show. 00:15:53.960 --> 00:15:59.620 You talked about transfer learning and using relatively small amounts of data to specialize models. 00:15:59.620 --> 00:16:00.760 Tell people about what that is. 00:16:00.760 --> 00:16:02.180 How do you actually do that? 00:16:02.180 --> 00:16:08.040 It's actually the same idea that has ultimately really led to these large generative models that we see. 00:16:08.040 --> 00:16:17.320 And that's essentially realizing that we can learn a lot about the language and the world and, you know, a lot of general stuff from raw. 00:16:17.700 --> 00:16:25.740 Like if we just train a model with a language modeling objective on like a bunch of text on the whole internet or parts of the internet or whatever, 00:16:25.740 --> 00:16:31.880 in order to basically solve the task, which can be stuff like predict the next word. 00:16:31.880 --> 00:16:38.220 In order to do that, the model has to learn so much in its weights and in its representations about the language 00:16:38.220 --> 00:16:43.880 and about like really underlying subtle stuff about the language that it's also really good at other stuff. 00:16:43.880 --> 00:16:46.820 That's kind of in a nutshell, the basic idea. 00:16:46.820 --> 00:16:52.560 And that's then later led to, you know, larger and larger models and more and more of these ideas. 00:16:52.560 --> 00:16:59.800 But yeah, the basic concept is if you just train on a lot of raw text and a lot of these models are available, 00:16:59.800 --> 00:17:06.060 like something like BERT, that's already like quite a few years old, but still, if you look at kind of the literature 00:17:06.060 --> 00:17:08.880 and look at the experiments people are doing, it's still very competitive. 00:17:08.880 --> 00:17:13.900 It's like you get really good results, even with the most, one of the most basic foundation models. 00:17:13.900 --> 00:17:19.240 And you can use that, initialize your model with that, and then just train like a small task network on top, 00:17:19.240 --> 00:17:22.720 instead of training everything from scratch, which is what you had to do before. 00:17:23.060 --> 00:17:29.100 Or it's like, if you imagine hiring a new employee, it's like, yes, you expect, you know, you don't, 00:17:29.100 --> 00:17:34.560 you can raise them from birth or you can sort of have them, which is like a very creepy concept, 00:17:34.560 --> 00:17:36.480 but it's really similar. 00:17:36.480 --> 00:17:38.200 Teach them everything. 00:17:38.200 --> 00:17:41.100 You were born to be a barista, let me tell you. 00:17:41.100 --> 00:17:44.080 Yeah, and then you teach them English and you teach them. 00:17:44.080 --> 00:17:47.100 Yeah, I mean, yeah, it's a lot of work. 00:17:47.100 --> 00:17:50.280 And I guess you know this more than me because you have, you have kids, right? 00:17:50.400 --> 00:17:56.780 So, yeah, so, and you know, it's understandable that like, okay, this, this, this made a lot 00:17:56.780 --> 00:18:00.780 of these ML projects really hard, but now you actually have the employee come in and they 00:18:00.780 --> 00:18:02.360 can, they know how to talk to people. 00:18:02.360 --> 00:18:06.360 They speak the language and all you have to teach them is like, hey, here's how you make 00:18:06.360 --> 00:18:07.360 a coffee here. 00:18:07.360 --> 00:18:07.900 Exactly. 00:18:07.900 --> 00:18:08.240 Yeah. 00:18:08.240 --> 00:18:14.100 You basically lean on the school system to say they know the language, they know arithmetic, 00:18:14.100 --> 00:18:16.260 they know how to talk to people. 00:18:16.260 --> 00:18:19.200 I just need to show them how this Expresso machine works. 00:18:19.600 --> 00:18:20.760 Here's how you check in. 00:18:20.760 --> 00:18:22.980 Please take out the trash every two hours. 00:18:22.980 --> 00:18:27.800 Like, yeah, very little bit of specialized information, but you, the sort of general working 00:18:27.800 --> 00:18:30.500 human knowledge is like the base LLM, right? 00:18:30.500 --> 00:18:31.220 That's the idea. 00:18:31.220 --> 00:18:36.480 And also transfer learning, it's still, it's just one technology and in context learning, 00:18:36.480 --> 00:18:38.820 which is, you know, what we have with these generative models. 00:18:38.820 --> 00:18:41.040 That's also just another technique. 00:18:41.040 --> 00:18:46.160 Like it's, you know, it's not the case that transfer learning is sort of outdated or has 00:18:46.160 --> 00:18:47.820 been replaced by in context learning. 00:18:47.820 --> 00:18:51.320 It's two different strategies and you use them in different contexts. 00:18:51.320 --> 00:18:51.820 So. 00:18:52.200 --> 00:18:54.120 One thing I want to touch on for people. 00:18:54.120 --> 00:18:59.900 I know some people, probably everyone listening is more or less aware of this, but in practice, 00:18:59.900 --> 00:19:05.400 a lot of folks out there listening, certainly the ones who are not in the ML or developer 00:19:05.400 --> 00:19:05.980 space. 00:19:06.120 --> 00:19:10.980 They just go to chat or they go to somewhere and they're like, this is the AI I've gone 00:19:10.980 --> 00:19:11.540 to, right? 00:19:11.540 --> 00:19:13.900 Maybe they go to Bard. 00:19:13.900 --> 00:19:14.480 I don't know. 00:19:14.480 --> 00:19:15.780 Gemini or whatever they call it. 00:19:15.780 --> 00:19:17.840 But there's a whole bunch. 00:19:17.840 --> 00:19:22.980 I mean, many, many, many open source models with all sorts of variations. 00:19:22.980 --> 00:19:30.080 One thing I really like is LM studio in more introduced this to me, introduced me to it a 00:19:30.080 --> 00:19:30.800 couple of months ago. 00:19:30.800 --> 00:19:35.660 And basically it's a UI for exploring hugging face models and then downloading them and running 00:19:35.660 --> 00:19:38.240 them with like a chat interface just in a UI. 00:19:38.240 --> 00:19:42.900 And the really cool thing is they just added Llama 3, but a lot of these are open source. 00:19:42.900 --> 00:19:44.680 A lot of these are accessible. 00:19:44.680 --> 00:19:45.820 You run them on your machine. 00:19:45.820 --> 00:19:48.840 You get 7 billion parameter models run easily on my Mac mini. 00:19:48.840 --> 00:19:49.480 Yeah. 00:19:49.480 --> 00:19:52.320 What do you think about some of these models rather than the huge ones? 00:19:52.320 --> 00:19:57.100 Yeah, no, I think it's, and also a lot of them are like, you know, the model itself is 00:19:57.100 --> 00:20:02.180 not necessarily much smaller than what a lot of these chat assistants deploy. 00:20:02.180 --> 00:20:07.140 And I think it's also, you know, these are really just the core models for everything that's 00:20:07.140 --> 00:20:11.200 like proprietary and sort of in-house behind like an API. 00:20:11.200 --> 00:20:15.560 There's at least one open source version that's very similar. 00:20:15.560 --> 00:20:20.780 Like I think it's the raw model is really based on academic research. 00:20:20.960 --> 00:20:23.500 A lot of the same data that's available. 00:20:23.500 --> 00:20:29.680 And I think the most important differentiation we see is then around these chat assistants and 00:20:29.680 --> 00:20:31.900 how they work and how the products are designed. 00:20:31.900 --> 00:20:36.540 So I think it's also, this is a, it's kind of a nice exercise or a nice way just to look at this 00:20:36.540 --> 00:20:42.020 distinction between the products versus the machine facing models. 00:20:42.020 --> 00:20:46.680 Because I think that's AI or like the, you know, these products are more than just a model. 00:20:46.680 --> 00:20:49.500 And I think that's like a super important thing to keep in mind. 00:20:49.500 --> 00:20:54.180 It's really relevant for this conversation because you have a whole section where you talk about 00:20:54.180 --> 00:20:58.800 regulation and what is the thing, what is the aspect of these things that should or could be 00:20:58.800 --> 00:20:59.260 regulated? 00:20:59.260 --> 00:21:00.140 We'll get to that in a minute. 00:21:00.240 --> 00:21:04.720 A lot of the confusion, confusion that people have around like, Ooh, are we like, is all 00:21:04.720 --> 00:21:07.220 AI going to be locked away behind APIs? 00:21:07.220 --> 00:21:10.080 And what, how do these bigger, bigger models work? 00:21:10.080 --> 00:21:14.800 I think it kind of stems from the fact that like, it's not always the distinction between 00:21:14.800 --> 00:21:16.980 like the models and products isn't always clear. 00:21:16.980 --> 00:21:20.840 And, you know, you could even argue maybe some companies that are in this business, you know, 00:21:20.840 --> 00:21:23.620 it benefits them to call everything the AI. 00:21:23.620 --> 00:21:24.140 Yeah. 00:21:24.220 --> 00:21:25.120 That really doesn't help. 00:21:25.120 --> 00:21:27.540 So to here, you really see the models. 00:21:27.540 --> 00:21:28.060 Yeah. 00:21:28.060 --> 00:21:32.120 And just sorry to talk over you, but to give people a sense, even if you search for Lama 00:21:32.120 --> 00:21:40.720 3 in this thing, there's 192 different configured, modified, et cetera, ways to work with the Lama 00:21:40.720 --> 00:21:42.660 3 model, which is just crazy. 00:21:42.660 --> 00:21:46.300 So there's a lot of, a lot of stuff that maybe people haven't really explored, I imagine. 00:21:46.300 --> 00:21:47.060 Yeah, it's very cool. 00:21:47.060 --> 00:21:47.400 Yeah. 00:21:47.400 --> 00:21:47.620 Yeah. 00:21:47.620 --> 00:21:51.660 One other thing about this, just while we're on it, is it also comes with a open AI API. 00:21:51.920 --> 00:21:56.300 So you could just run it and say, turn on a server API endpoint if I want to talk to it. 00:21:56.300 --> 00:21:56.860 Very fun. 00:21:56.860 --> 00:22:01.480 But let's talk about some of the things you talked about in your talk. 00:22:01.480 --> 00:22:03.900 The AI revolution will not be monopolized. 00:22:03.900 --> 00:22:07.440 How open source beats economies of scale, even for LLMs. 00:22:07.440 --> 00:22:07.960 I love it. 00:22:07.960 --> 00:22:09.780 It's a great title and a great topic. 00:22:09.780 --> 00:22:10.540 No, thanks. 00:22:10.540 --> 00:22:12.500 No, I'm very, it's something I'm very passionate about. 00:22:12.500 --> 00:22:17.460 I was like, I was very, you know, happy to be able to say a lot of these things or to, 00:22:17.460 --> 00:22:18.920 you know, also be given a platform. 00:22:19.220 --> 00:22:24.280 You and I, we've spoken before about open source and running successful businesses in 00:22:24.280 --> 00:22:25.900 the tech space and all sorts of things. 00:22:25.900 --> 00:22:27.480 So it's a cool follow on for sure. 00:22:27.480 --> 00:22:27.820 Yeah. 00:22:27.820 --> 00:22:32.840 I think one of the first parts that you talked about that was really interesting and has nothing 00:22:32.840 --> 00:22:40.440 to do specifically with LLMs or AI is just why is open source a good choice and why are 00:22:40.440 --> 00:22:41.220 people choosing? 00:22:41.220 --> 00:22:44.620 Why is it a good thing to base businesses on and so on? 00:22:44.620 --> 00:22:44.880 Yeah. 00:22:44.880 --> 00:22:50.280 Also often when I give this as a talk, I've, I ask like for a show of hands, like, Hey, 00:22:50.280 --> 00:22:54.680 who, who uses open source software or who works for a company that depends on open source 00:22:54.680 --> 00:22:55.020 software? 00:22:55.020 --> 00:22:56.240 Who's contributed before? 00:22:56.240 --> 00:23:01.720 And usually I think most people raise their hand when I ask like who works for a company 00:23:01.720 --> 00:23:03.100 that relies on open source software. 00:23:03.400 --> 00:23:08.580 So, you know, I often feel like, Hey, I don't even have to like explain like, Hey, it's a 00:23:08.580 --> 00:23:08.900 thing. 00:23:08.900 --> 00:23:11.100 It's more about like, you know, collecting these reasons. 00:23:11.100 --> 00:23:16.520 And I do think a lot of it is around like, you know, the transparency, the extensibility, 00:23:16.520 --> 00:23:17.700 it's all kind of connected. 00:23:17.700 --> 00:23:22.760 Like you're not locked in, you can run it in house, you can fork it, you can, you know, 00:23:22.760 --> 00:23:23.800 program with it. 00:23:23.800 --> 00:23:27.660 Like those are all important things for companies when they adopt software. 00:23:28.080 --> 00:23:32.240 And you also often, you have these small teams running the project that can accept PRs that 00:23:32.240 --> 00:23:33.060 can move fast. 00:23:33.060 --> 00:23:37.500 There's a community around it that can basically give you a sense for, Hey, is this a thing? 00:23:37.500 --> 00:23:38.800 Should I adopt it? 00:23:38.800 --> 00:23:40.520 And all of this I think is important. 00:23:40.520 --> 00:23:45.120 And, you know, I also often make a point to like, yes, I've always mentioned, Hey, it's 00:23:45.120 --> 00:23:49.200 also free, which is what people usually associate with open source software. 00:23:49.200 --> 00:23:52.860 And it's kind of the first thing that comes to mind, but I actually don't think this is 00:23:52.860 --> 00:23:56.520 for companies, the main motivation why they use open source software. 00:23:56.520 --> 00:23:57.860 I absolutely agree. 00:23:57.860 --> 00:24:04.420 I mean, even though we have FOSS, free and open source software, this is not really why 00:24:04.420 --> 00:24:05.880 companies care about it, right? 00:24:05.880 --> 00:24:06.460 I'm sure. 00:24:06.460 --> 00:24:12.060 Certainly some people do, some people don't, but companies, they often see that as a negative, 00:24:12.060 --> 00:24:14.820 I think almost like, well, who do we sue if this goes wrong? 00:24:14.820 --> 00:24:17.020 Where's our, our service level agreement? 00:24:17.020 --> 00:24:18.500 Who's going to help us? 00:24:18.500 --> 00:24:20.460 Who's legally obligated to help us? 00:24:20.460 --> 00:24:21.580 We've definitely also seen that. 00:24:21.580 --> 00:24:24.820 Or like we have, you know, companies who are like, well, who can we pay? 00:24:24.820 --> 00:24:30.640 Or can we pay to like, I don't know, get, you know, some guarantee or like some support 00:24:30.640 --> 00:24:36.100 or, or, you know, can you like confirm to us that, Hey, if there is a critical vulnerability, 00:24:36.100 --> 00:24:41.220 that's like really directly affecting our software, which, you know, has never happened, but are 00:24:41.220 --> 00:24:42.000 you going to fix it? 00:24:42.000 --> 00:24:43.740 We're like, yes, we can say that. 00:24:43.740 --> 00:24:44.980 That's what we've been doing. 00:24:45.100 --> 00:24:47.960 But if you want that guarantee, we can give that to you for money. 00:24:47.960 --> 00:24:48.300 Sure. 00:24:48.300 --> 00:24:51.980 But like, you can pay us, we'll, we'll promise to do what we already promised to do, but 00:24:51.980 --> 00:24:53.840 we'll really double, double promise to do it. 00:24:53.840 --> 00:24:54.040 Right. 00:24:54.040 --> 00:24:54.760 That's definitely a thing. 00:24:54.760 --> 00:24:58.020 And also it's kind of to, you know, to go back up to the business model thing, it's what 00:24:58.020 --> 00:25:03.140 we've seen with Prodigy, which, you know, we offer kind of, it's, it really as a tool, 00:25:03.140 --> 00:25:04.520 it follows the open source spirit. 00:25:04.520 --> 00:25:06.180 Like you don't, you pip install it. 00:25:06.180 --> 00:25:10.820 It's a Python library, you work with it, but we decided to kind of use that as a stepping 00:25:10.820 --> 00:25:15.620 stone between our free open source offering and like the SaaS product that we're about 00:25:15.620 --> 00:25:17.380 to launch soon, hopefully. 00:25:17.380 --> 00:25:20.040 And it's kind of in the middle and it's paid. 00:25:20.040 --> 00:25:25.460 And we've definitely, you know, not found that this is like a huge disadvantage for companies. 00:25:25.460 --> 00:25:26.260 Like, yes, sure. 00:25:26.260 --> 00:25:30.740 You always have companies with like no budget, but those are also usually not the teams that 00:25:30.740 --> 00:25:34.980 are really doing, you know, a lot of the high value work because, you know, it is quite 00:25:34.980 --> 00:25:37.900 normal to have a budget or like software tools. 00:25:37.900 --> 00:25:39.400 Companies pay a lot for this. 00:25:40.060 --> 00:25:43.720 Like, you know, if you, if you want to buy Prodigy, like that, that costs less than, 00:25:43.720 --> 00:25:45.680 I don't know, getting a decent office chair. 00:25:45.680 --> 00:25:50.200 Like, you know, in a, in a commercial context, these, these scales are all a bit different. 00:25:50.200 --> 00:25:54.520 So yeah, I do think companies are happy to pay for something that they need and that's 00:25:54.520 --> 00:25:54.740 good. 00:25:54.740 --> 00:25:55.020 Yeah. 00:25:55.020 --> 00:25:59.640 And the ones who wouldn't have paid, there's a group who said, well, maybe I'll use the 00:25:59.640 --> 00:26:03.980 free one, but they're not serious enough about it to, to actually pay for it or actually 00:26:03.980 --> 00:26:04.740 make use of it. 00:26:04.740 --> 00:26:08.220 You know, I think of sort of analogies of piracy, right? 00:26:08.280 --> 00:26:11.780 Like though they stole our app or they stole our music, like, well, because they, it was 00:26:11.780 --> 00:26:15.360 a link, they clicked it, but they, they wouldn't have bought it or used it at all. 00:26:15.360 --> 00:26:18.260 It's not like you lost a customer because they were not going to be customers. 00:26:18.260 --> 00:26:18.780 They just have. 00:26:18.780 --> 00:26:19.160 Yeah, exactly. 00:26:19.160 --> 00:26:23.920 I mean, it's like, I always tell the story of like, when I was, you know, a teenager, I 00:26:23.920 --> 00:26:27.280 did download a crack version of Adobe Photoshop. 00:26:27.660 --> 00:26:31.420 And because I was a teenager, I would have never been able to like back then they had, 00:26:31.420 --> 00:26:32.580 they didn't have a SaaS model. 00:26:32.580 --> 00:26:35.540 Like, I don't know what Photoshop costs, but like, it's definitely not something I would 00:26:35.540 --> 00:26:38.020 have been able to afford as a 13, 14 year old. 00:26:38.020 --> 00:26:40.020 So I did find that online. 00:26:40.020 --> 00:26:40.820 I downloaded it. 00:26:40.820 --> 00:26:44.720 I'm pretty sure if Adobe had wanted, they could have come after me for that. 00:26:44.720 --> 00:26:48.800 And I do think like, I don't know, maybe I'm giving them too much credit, but I do think 00:26:48.800 --> 00:26:52.540 they might've not done that because they're like, well, what it's not like we lost a customer 00:26:52.540 --> 00:26:52.800 here. 00:26:52.880 --> 00:26:57.180 And now I'm an adult and I'm, I'm proficient at Photoshop and now I'm paying for it. 00:26:57.180 --> 00:26:58.140 Yeah, exactly. 00:26:58.140 --> 00:27:02.180 And I think there was this whole generation of teenagers who then maybe went into creative 00:27:02.180 --> 00:27:04.500 jobs and came in with Photoshop skills. 00:27:04.500 --> 00:27:08.440 Like I wasn't even like, compared to all these other teenagers I was hanging out with on the 00:27:08.440 --> 00:27:13.060 internet, like all these, like mostly, mostly girls, I wasn't even that talented at Photoshop 00:27:13.060 --> 00:27:13.600 specifically. 00:27:13.600 --> 00:27:17.920 So maybe, maybe there was someone smart who thought about this as like a business strategy 00:27:17.920 --> 00:27:20.640 that these teenagers have our professional tools. 00:27:20.640 --> 00:27:21.060 Exactly. 00:27:21.060 --> 00:27:22.740 It's, it's almost marketing. 00:27:22.740 --> 00:27:30.380 Another aspect here that I think is really relevant to LLMs is runs in house, AKA, we're 00:27:30.380 --> 00:27:36.520 not sending our private data, private source code, API keys, et cetera, to other companies 00:27:36.520 --> 00:27:40.960 that may even use that to train their models, which then regurgitate that back to other people 00:27:40.960 --> 00:27:42.540 who are trying to solve the same problems. 00:27:42.540 --> 00:27:42.800 Right. 00:27:42.800 --> 00:27:47.780 That's also, we're definitely seeing that companies are becoming more and more aware of this, which 00:27:47.780 --> 00:27:48.500 is good. 00:27:48.580 --> 00:27:51.820 Like in a lot of industries, like I wouldn't want, I don't know, my healthcare provider 00:27:51.820 --> 00:27:57.260 to just upload all of my data to like whichever SaaS tool they decide to use at the moment. 00:27:57.260 --> 00:27:59.060 Like, you know, of course not. 00:27:59.060 --> 00:28:00.660 So I think it's, you know, it's, it's good. 00:28:00.660 --> 00:28:06.400 And then also with, you know, more data privacy regulations, that's all that's really on people's 00:28:06.400 --> 00:28:06.700 minds. 00:28:06.700 --> 00:28:12.060 And people don't want it's like often we have, we have companies or users who actually 00:28:12.060 --> 00:28:15.080 have to run a lot of their AI stuff on completely air gap machines. 00:28:15.080 --> 00:28:20.200 So they can't even have internet access or it's about, you know, financial stuff. 00:28:20.200 --> 00:28:24.420 We're actually working on a case study that we're hoping to publish soon where even the 00:28:24.420 --> 00:28:26.320 financial information can move markets. 00:28:26.320 --> 00:28:28.060 It's even segregated in the office. 00:28:28.440 --> 00:28:31.820 So it needs to be 100% in-house. 00:28:31.820 --> 00:28:32.240 Yeah. 00:28:32.240 --> 00:28:32.900 That makes sense. 00:28:32.900 --> 00:28:37.260 And I think open source software, it's great because you can do that and you can build your 00:28:37.260 --> 00:28:42.600 own things with it and really decide how you want to host it, how it fits into your existing 00:28:42.600 --> 00:28:43.100 stack. 00:28:43.100 --> 00:28:44.640 That's another big thing. 00:28:44.640 --> 00:28:50.240 People will already use some tools and, you know, you don't want to change your entire workflow 00:28:50.240 --> 00:28:53.280 for every different tool or platform you use. 00:28:53.280 --> 00:28:57.120 And I think especially people have been burned by that so many times by now, and there's 00:28:57.120 --> 00:29:00.640 so many like, you know, unreliable startups things you have. 00:29:00.640 --> 00:29:03.920 There's a company that really tries to convince you to build on their product. 00:29:03.920 --> 00:29:08.380 And then two months later, they close everything down or, you know, it doesn't even have to 00:29:08.380 --> 00:29:10.180 be startup, you know, Google. 00:29:10.180 --> 00:29:14.160 I'm still mad at Google for shutting down Google reader. 00:29:14.160 --> 00:29:17.320 And I don't know, it's been over 10 years, I'm sure. 00:29:17.320 --> 00:29:19.340 And I'm still angry about that. 00:29:19.340 --> 00:29:21.300 I actually had a, had this, we did it. 00:29:21.360 --> 00:29:26.260 We were invited to give a talk at Google and I needed a text example to visualize, you 00:29:26.260 --> 00:29:27.300 know, something grammatical. 00:29:27.300 --> 00:29:30.180 And that text I made, Google shut down Google reader. 00:29:30.180 --> 00:29:32.360 That's a quiet protest. 00:29:32.360 --> 00:29:34.040 Oh, that's amazing. 00:29:34.040 --> 00:29:34.680 Yeah. 00:29:34.680 --> 00:29:39.060 We're going to run sentiment analysis on this article here. 00:29:39.060 --> 00:29:44.660 Sure, open source projects can become unmaintained and that sucks, but like, you know, you can fork 00:29:44.660 --> 00:29:44.860 it. 00:29:44.860 --> 00:29:47.160 It's, it's there and you can, you can have it. 00:29:47.160 --> 00:29:48.960 So there is this, this is motivating. 00:29:48.960 --> 00:29:53.000 And I think we've always called it like, you can reinvent the wheel, but don't reinvent 00:29:53.000 --> 00:29:57.540 the road, which is basically, you can, you can build something, reinventing the wheel. 00:29:57.540 --> 00:30:03.380 I don't think it's bad, but like, you don't want to make people follow like, you know, your 00:30:03.380 --> 00:30:05.140 way of doing everything. 00:30:05.140 --> 00:30:06.540 And yeah, that's interesting. 00:30:06.740 --> 00:30:06.980 Yeah. 00:30:06.980 --> 00:30:07.020 Yeah. 00:30:07.020 --> 00:30:08.660 Like we have electric cars now. 00:30:08.660 --> 00:30:09.080 All right. 00:30:09.080 --> 00:30:15.520 So give us a sense of some of the open source models in this AI space here. 00:30:15.520 --> 00:30:18.340 I've kind of divided it into sort of three categories. 00:30:18.340 --> 00:30:22.080 So one of them is what I've called task specific models. 00:30:22.080 --> 00:30:27.680 So that's really models that we're trying to do one specific or some specific things. 00:30:27.680 --> 00:30:30.100 It's kind of what we distribute for spaCy. 00:30:30.100 --> 00:30:37.460 There's also a lot of really cool community projects like size spaCy for scientific biomedical 00:30:37.460 --> 00:30:38.280 techs. 00:30:38.280 --> 00:30:40.740 Stanford also publishes their stanza models. 00:30:40.740 --> 00:30:45.260 And yeah, if you've been on the hugging face hub, there's like tons of these models that 00:30:45.260 --> 00:30:50.620 were really fine tuned to predict like a particular type of categories, stuff like that. 00:30:50.620 --> 00:30:54.180 And so that's been around for quite a while, quite established. 00:30:54.180 --> 00:30:55.940 A lot of people use these in production. 00:30:55.940 --> 00:31:00.840 And it was so quite, especially nowadays, but today's standards, they quite small, cheap, 00:31:00.840 --> 00:31:03.080 but of course they do one particular thing. 00:31:03.080 --> 00:31:05.040 So they don't generalize very well. 00:31:05.040 --> 00:31:07.300 So that's kind of the one category. 00:31:07.300 --> 00:31:14.520 You probably used to think of them as large and now you see how giant, how many gigabytes those 00:31:14.520 --> 00:31:15.520 models are, you know? 00:31:15.520 --> 00:31:15.880 Yeah. 00:31:15.880 --> 00:31:20.120 When deep learning first kind of came about and people were sort of migrating from 00:31:20.120 --> 00:31:22.140 linear models and stuff. 00:31:22.140 --> 00:31:26.380 Like I've met people complaining that the models were too, were so big and slow. 00:31:26.380 --> 00:31:32.960 And that was before we even, you know, used much transfer learning and transformer models 00:31:32.960 --> 00:31:33.980 and BERT and stuff. 00:31:33.980 --> 00:31:38.240 And even when that came about, it was also first a challenge like, Hey, these are significantly 00:31:38.240 --> 00:31:38.580 bigger. 00:31:38.580 --> 00:31:44.480 We do have to change a lot around it or even, you know, Google who published BERT, they had 00:31:44.480 --> 00:31:49.480 to do a lot of work around it to make it work into their workflows and ship them into production 00:31:49.620 --> 00:31:53.500 and optimize them because they were quite different from what was there before. 00:31:54.880 --> 00:31:58.920 This portion of Talk Python To Me is sponsored by porkbun.com. 00:31:58.920 --> 00:32:03.800 Launching a successful project involves many decisions, not the least of which is choosing 00:32:03.800 --> 00:32:04.800 a domain name. 00:32:04.800 --> 00:32:09.200 And as your project grows, ownership and control of that domain is critical. 00:32:09.200 --> 00:32:11.760 You want a domain registrar that you can trust. 00:32:11.760 --> 00:32:16.300 I recently moved a bunch of my domains to a new provider and asked the community who they 00:32:16.300 --> 00:32:17.440 recommended I choose. 00:32:17.440 --> 00:32:19.280 Porkbun was highly recommended. 00:32:20.120 --> 00:32:25.780 Porkbun specializes in domains that developers need like .app, .dev, and .foo domains. 00:32:25.780 --> 00:32:31.280 If you're launching that next breakthrough developer tool or finally creating a dedicated website 00:32:31.280 --> 00:32:34.560 for your open source project, how about a .dev domain? 00:32:34.560 --> 00:32:40.020 Or just show off your kung.foo programming powers with a domain there. 00:32:40.380 --> 00:32:43.020 These domains are designed to be secure by default. 00:32:43.020 --> 00:32:50.240 All .app and .dev domains are HSTS preloaded, which means that all .app and .dev websites will 00:32:50.240 --> 00:32:53.080 only load over an encrypted SSL connection. 00:32:53.080 --> 00:32:56.120 This is the gold standard of website security. 00:32:56.120 --> 00:33:02.780 If you're paying for whois privacy, SSL certificates, and more, you should definitely check out Porkbun. 00:33:03.040 --> 00:33:05.480 These features are always free with every domain. 00:33:05.480 --> 00:33:07.900 So get started with your next project today. 00:33:07.900 --> 00:33:14.760 Lock down your .app, .dev, or .food domain at Porkbun for only $1 for the first year. 00:33:14.760 --> 00:33:16.200 That's right, just $1. 00:33:16.200 --> 00:33:19.340 Visit talkpython.fm/porkbun. 00:33:19.340 --> 00:33:22.260 That's talkpython.fm/porkbun. 00:33:22.260 --> 00:33:24.580 The link is in your podcast player's show notes. 00:33:24.580 --> 00:33:26.800 Thank you to Porkbun for supporting the show. 00:33:28.300 --> 00:33:34.280 Another one in this category of task-specific models is SciSpacee, which is kind of cool. 00:33:34.280 --> 00:33:35.100 What's SciSpacee? 00:33:35.100 --> 00:33:39.300 Yeah, so SciSpacee, that's for scientific biomedical text. 00:33:39.300 --> 00:33:42.320 That was published by Allen AI researchers. 00:33:42.320 --> 00:33:49.280 And yeah, it's really, it has like components specific for working with that sort of data. 00:33:49.280 --> 00:33:54.140 And it's actually, it's definitely, if that's kind of the domain, yeah, any listeners are working 00:33:54.140 --> 00:33:55.300 with, definitely check it out. 00:33:55.300 --> 00:34:00.820 They've also done some pretty smart work around like, A, training components, but also 00:34:00.820 --> 00:34:07.700 implementing like hybrid rule-based things for, say, acronym expansion. 00:34:07.700 --> 00:34:08.200 Right. 00:34:08.200 --> 00:34:12.680 They're like cool algorithms that you can implement that don't necessarily need much machine learning, 00:34:12.680 --> 00:34:13.800 but that work really well. 00:34:13.800 --> 00:34:19.540 And so it's basically the suite of components and also models that are more tuned for that 00:34:19.540 --> 00:34:20.280 domain. 00:34:20.280 --> 00:34:23.560 You mentioned some, but also encoder models. 00:34:23.560 --> 00:34:26.680 What's the difference between the task-specific ones and the encoder ones? 00:34:26.680 --> 00:34:31.280 That's kind of also what we were talking about earlier, actually, with the transfer learning, 00:34:31.280 --> 00:34:32.500 foundation models. 00:34:32.500 --> 00:34:37.820 These are models trained with a language modeling objective, for example, like Google Spurt. 00:34:37.820 --> 00:34:41.780 And, you know, that can also be the foundation for task-specific models. 00:34:41.920 --> 00:34:44.900 That's kind of what we're often doing nowadays. 00:34:44.900 --> 00:34:51.040 Like you start out with some of these pre-trained weights, and then you train like this task-specific 00:34:51.040 --> 00:34:56.880 network on top of it that uses everything that is in these weights about the language and the 00:34:56.880 --> 00:34:57.200 world. 00:34:57.200 --> 00:35:03.520 And yeah, actually by today's standards, these are still relatively small and relatively fast. 00:35:03.700 --> 00:35:08.440 And they generalize better because, you know, they're trained on a lot of raw text that has 00:35:08.440 --> 00:35:14.060 like a lot of, yeah, a lot of that intrinsic meta knowledge about the language and the world 00:35:14.060 --> 00:35:16.020 that we need to solve a lot of other tasks. 00:35:16.020 --> 00:35:16.520 Absolutely. 00:35:16.520 --> 00:35:23.920 And then you've used the word, the term large generative models for things like LAMA and 00:35:23.920 --> 00:35:25.060 Mestral and so on. 00:35:25.140 --> 00:35:29.200 One thing that's very unfortunate when, you know, talking about these models is that like 00:35:29.200 --> 00:35:33.660 everything we've talked about here has at some point been called an LLM by someone. 00:35:33.660 --> 00:35:36.960 And that makes it like really hard to talk about it. 00:35:36.960 --> 00:35:41.940 And, you know, so like, you can argue that like, well, all of them are kind of large language 00:35:41.940 --> 00:35:43.000 models, right? 00:35:43.000 --> 00:35:45.860 And then there's also, you know, the marketing confusion. 00:35:45.860 --> 00:35:52.420 Like, you know, when LLMs were hot, everyone wants to, you know, have LLMs. 00:35:52.600 --> 00:35:57.100 And so by some definition of LLMs, we've all been running LLMs in production for years. 00:35:57.100 --> 00:36:02.240 But basically, yeah, I've kind of decided, okay, I want to try and avoid that phrase as 00:36:02.240 --> 00:36:04.640 much as possible because it really doesn't help. 00:36:04.640 --> 00:36:09.820 And so large generative models kind of captures that same idea, but it makes it clear, okay, 00:36:09.820 --> 00:36:15.380 these generate text, text goes in, text comes out, and they're large, and they're different 00:36:15.380 --> 00:36:17.320 from the other types of models, basically. 00:36:17.320 --> 00:36:22.160 Question on the audience is, Mr. Magnetics, I'd love to learn how to develop AI. 00:36:22.160 --> 00:36:25.080 So maybe let me rephrase that just a little bit and see what your thoughts are. 00:36:25.080 --> 00:36:28.880 Like, if people want to get more foundational, this kind of stuff, like, what areas should 00:36:28.880 --> 00:36:31.200 they maybe focus in to learn? 00:36:31.200 --> 00:36:32.680 What are your thoughts there? 00:36:32.680 --> 00:36:35.580 It depends on really, you know, what it means. 00:36:35.580 --> 00:36:40.680 Like, if you really, you know, there is a whole path to, okay, you really want to learn more 00:36:40.680 --> 00:36:44.860 about the models, how they work, you know, the research that goes into it. 00:36:44.860 --> 00:36:50.880 I think there's a lot of actually also academic resources and courses that you can take that 00:36:50.880 --> 00:36:54.600 are similar to, you know, what you would learn in university if you started... 00:36:54.600 --> 00:36:56.240 ML course. 00:36:56.240 --> 00:36:56.860 Yeah. 00:36:56.940 --> 00:36:56.980 Yeah. 00:36:56.980 --> 00:36:57.500 Like, ML. 00:36:57.500 --> 00:37:01.780 And also, I think some universities have made some of their, like, beginner's courses public. 00:37:01.780 --> 00:37:03.220 I think Stanford has. 00:37:03.220 --> 00:37:03.800 Yeah. 00:37:03.800 --> 00:37:03.920 Right. 00:37:03.920 --> 00:37:07.660 I thought Stanford, I think there's someone else, but like, there's definitely also a lot 00:37:07.660 --> 00:37:08.680 of stuff coming out. 00:37:08.680 --> 00:37:14.200 So you can kind of, you know, go in that direction, really learn, okay, how, what goes into this? 00:37:14.280 --> 00:37:15.580 What's the theory behind this? 00:37:15.580 --> 00:37:18.040 And there are some people who really like that approach. 00:37:18.040 --> 00:37:20.700 And then there's a whole more practical side. 00:37:20.700 --> 00:37:21.060 Okay. 00:37:21.060 --> 00:37:26.180 I want to build an application that uses the technology and it solves the problem. 00:37:26.180 --> 00:37:29.460 And often it helps to have like an idea of what you want to, what you want to do. 00:37:29.460 --> 00:37:32.960 Like, if you don't want to develop AI for the sake of it, then it often helps. 00:37:32.960 --> 00:37:36.660 Like, hey, you have, even if it's just your hobby, like you're into football and you, 00:37:36.660 --> 00:37:39.620 you come up with like some fun, fun problem. 00:37:39.620 --> 00:37:45.200 Like you want to analyze football news, for example, and analyze it for something you care 00:37:45.200 --> 00:37:45.460 about. 00:37:45.460 --> 00:37:49.380 Like, I don't know, like often, often really helps to have this hobby angle or something 00:37:49.380 --> 00:37:50.380 you're interested in. 00:37:50.380 --> 00:37:50.940 Yeah, it does. 00:37:50.940 --> 00:37:51.140 Yeah. 00:37:51.140 --> 00:37:54.740 And then you can start looking at tools that go in that direction. 00:37:54.740 --> 00:37:59.340 Like start with some of these open source models, even, you know, try out some of these 00:37:59.340 --> 00:38:01.360 generative models, see how you go. 00:38:01.360 --> 00:38:05.620 Try out, if you want to do information extraction, try out maybe something like spaCy. 00:38:05.620 --> 00:38:07.480 There's like really a lot there. 00:38:07.560 --> 00:38:12.560 And it's definitely become a lot easier to get started and build something these days. 00:38:12.560 --> 00:38:16.380 Another thing you talked about was economies of scale. 00:38:16.380 --> 00:38:17.660 And this one's really interesting. 00:38:17.660 --> 00:38:24.600 So basically we've got Gemini and OpenAI, where they've just got so much traffic. 00:38:24.600 --> 00:38:28.900 And kind of a little bit back to the question, actually, is if you want to do this kind of 00:38:28.900 --> 00:38:33.100 stuff, you want to run your own service doing it, you know, even if you had the equivalent 00:38:33.100 --> 00:38:33.560 stuff. 00:38:33.680 --> 00:38:38.120 It's tricky because even just the way you batch compute, you maybe want to talk about 00:38:38.120 --> 00:38:38.500 that a bit? 00:38:38.500 --> 00:38:44.360 The idea of economies of scale is basically, well, as the companies produce more output, 00:38:44.360 --> 00:38:46.620 the cost per unit decreases. 00:38:46.620 --> 00:38:50.520 And yeah, there's like all kinds of, you know, basically it gets cheaper to do more stuff. 00:38:50.520 --> 00:38:56.300 And, you know, there are like a lot of more boring, like business-y reasons why it's like 00:38:56.300 --> 00:38:56.560 that. 00:38:56.560 --> 00:39:02.520 But I think for machine learning specifically, the fact that GPUs are so parallel really makes 00:39:02.520 --> 00:39:03.220 a difference here. 00:39:03.220 --> 00:39:07.720 And because, you know, you get the user text in, you can't just arbitrarily chop up that 00:39:07.720 --> 00:39:09.220 text because the context matters. 00:39:09.220 --> 00:39:10.300 You need to process that. 00:39:10.780 --> 00:39:15.740 So in order to make the most use of the compute, you basically need to batch it up. 00:39:15.740 --> 00:39:19.840 So either, you know, kind of need to wait until there's enough to batch up. 00:39:19.840 --> 00:39:25.560 And that means that, yes, that favors a lot of those providers that have a lot of traffic 00:39:25.560 --> 00:39:27.620 or, you know, you introduce latency. 00:39:28.500 --> 00:39:33.580 So that's definitely something that at least looks like a problem or, you know, something 00:39:33.580 --> 00:39:38.000 that can be discouraging because it feels like, hey, if you, you know, if supposedly the only 00:39:38.000 --> 00:39:42.620 way you can kind of participate is by running these models and either you have to run them 00:39:42.620 --> 00:39:46.580 yourself or go via an API, like then you're kind of doomed. 00:39:46.580 --> 00:39:52.340 And does that mean that, okay, only some large companies can, you know, provide AI for us? 00:39:52.340 --> 00:39:56.720 So that's kind of also the, you know, the point and, you know, the very legit, like worry 00:39:56.720 --> 00:40:00.960 that some people have, like, does that lead to like monopolizing AI basically? 00:40:00.960 --> 00:40:05.620 It's a very valid concern because even if you say, okay, look, here's the deal. 00:40:05.620 --> 00:40:08.460 Open AI gets to run on Azure. 00:40:08.460 --> 00:40:12.040 I can go get a machine with a GPU stuck to it and run that on Azure. 00:40:12.040 --> 00:40:13.080 Well, guess what? 00:40:13.080 --> 00:40:20.000 They get one of those huge ARM chips that's like the size of a plate and they get the special 00:40:20.000 --> 00:40:26.600 machines and they also get either wholesale compute costs or they get just, we'll give 00:40:26.600 --> 00:40:31.280 you a bunch of compute for some ownership of your company, kind of like Microsoft and open 00:40:31.280 --> 00:40:31.640 AI. 00:40:31.640 --> 00:40:32.120 Yeah. 00:40:32.120 --> 00:40:34.880 That's a very difficult thing to compete with on one hand, right? 00:40:34.880 --> 00:40:35.240 Yes. 00:40:35.240 --> 00:40:42.440 If you want to, you know, run your own, like, you know, LLM or generative model API services, 00:40:42.440 --> 00:40:45.580 that's definitely, you know, a disadvantage you're going to have. 00:40:45.840 --> 00:40:51.320 But on the other hand, I think one thing that leads to this perception that I think is not 00:40:51.320 --> 00:40:55.780 necessarily true is the fact that you want to do anything you need, basically larger and 00:40:55.780 --> 00:41:00.000 larger models that, you know, if you want to do something specific, the only way to get 00:41:00.000 --> 00:41:05.320 there is to turn that request into arbitrary language and then use the largest model that 00:41:05.320 --> 00:41:07.840 can handle arbitrary language and go from there. 00:41:07.840 --> 00:41:11.700 Like if you, and I know this is like something that, you know, maybe a lot of LLM companies 00:41:11.700 --> 00:41:14.180 want to tell you, but that's not necessarily true. 00:41:14.180 --> 00:41:20.060 And you don't, yeah, for a lot of things you're doing, you don't even need to depend on a large 00:41:20.060 --> 00:41:21.080 model at runtime. 00:41:21.080 --> 00:41:26.840 You can distill it and you can use it at development time and then build something that you can run 00:41:26.840 --> 00:41:27.400 in-house. 00:41:27.400 --> 00:41:32.760 And these calculations also look, look very, very different if you're using something at 00:41:32.760 --> 00:41:38.780 development time versus in production at runtime, and then it can actually be totally fine to just 00:41:38.780 --> 00:41:40.120 run something in-house. 00:41:40.120 --> 00:41:45.600 And the other point here is actually some, you know, if we're having a situation where, Hey, 00:41:45.600 --> 00:41:52.180 you're paying a large company to provide some service for you, provide a model for you by an API. 00:41:52.180 --> 00:41:56.340 And there are lots of companies and kind of the main differentiator is who can offer it for 00:41:56.340 --> 00:42:01.200 cheaper. That's sort of the opposite of a monopoly at least, right? That's like competition. 00:42:01.200 --> 00:42:07.820 So this actually, I feel like economies of scale, this, this idea does not prove that, Hey, we're 00:42:07.820 --> 00:42:14.660 heading into, we're heading into a monopoly. And it's also not true because it's not, if you realize 00:42:14.660 --> 00:42:20.480 that, Hey, it's not, you know, you don't need the biggest, most arbitrary models for everything 00:42:20.480 --> 00:42:24.080 you're doing, then the calculation looks very, very different. 00:42:24.080 --> 00:42:26.280 Yeah, I agree. I think there's a couple of thoughts. 00:42:26.280 --> 00:42:31.620 I also have here is one, this LM studio I was talking about, I've been running the Llama 3, 00:42:31.620 --> 00:42:37.820 7 billion parameter model locally instead of using chat these days. And it's been, I would say, 00:42:37.820 --> 00:42:44.240 just as good. And it's, it runs about the same speed on my Mac mini as a typical request does over 00:42:44.240 --> 00:42:49.040 there. I mean, can't handle as many, but it's just me. It's my computer, right? I'm fine. And then the 00:42:49.040 --> 00:42:55.020 other one is if you specialize one of these models, right? You feed it a bunch of your data sets from your 00:42:55.020 --> 00:43:00.800 companies. It might not be able to write you something in the style of Shakespeare around, 00:43:00.800 --> 00:43:05.980 you know, a legal contract or some weird thing like that, but it can probably answer really good 00:43:05.980 --> 00:43:11.440 questions about what is our company policy on this? Or what is, what are engineering reports about this 00:43:11.440 --> 00:43:14.840 thing say? Or, you know, stuff that you actually care about, right? You could run that. 00:43:14.920 --> 00:43:18.700 That's kind of what you want. Like you actually want to, if you're talking about, if you're going 00:43:18.700 --> 00:43:23.040 to like some of the risks or things people are worried about, like a lot of that is around what 00:43:23.040 --> 00:43:27.620 people refer to like, oh, the model going rogue or like the model doing stuff it's not supposed to do. 00:43:27.620 --> 00:43:27.940 Yeah. 00:43:27.940 --> 00:43:33.260 If you're just sort of wrapping ChatGPT and you're not careful, then when you're giving it access to 00:43:33.260 --> 00:43:39.160 stuff, there's a lot of unintended things that people could do with it. If you're actually running 00:43:39.160 --> 00:43:44.240 this and once you expose it to users, there's like a lot of risks there and yeah, writing something in 00:43:44.240 --> 00:43:49.900 the style of Shakespeare is like probably the most harmless outcome that you can get. But like, 00:43:49.900 --> 00:43:54.420 that is kind of a risk. And you basically, you know, you're also, you're paying and you're putting 00:43:54.420 --> 00:43:59.440 all this work into hosting and providing and running this model that has all these capabilities 00:43:59.440 --> 00:44:04.080 that you don't need. And a lot of them might actually be, you know, make it much harder to trust 00:44:04.080 --> 00:44:09.580 the system and also, you know, make it a lot less transparent. Like that's another aspect, 00:44:09.580 --> 00:44:14.660 like just, you know, you want your software to be modular and transparent and that ties back into 00:44:14.660 --> 00:44:18.580 what people want from open source. But I think also what people want from software in general, 00:44:18.780 --> 00:44:24.860 like we've over decades and more, we've built up a lot of best practices around software development 00:44:24.860 --> 00:44:30.780 and what makes sense. And that's based on, you know, the reality of building software industry. 00:44:30.780 --> 00:44:35.380 And just because there's like, you know, new capabilities and new things we can do and a new 00:44:35.380 --> 00:44:40.560 paradigm doesn't mean we have to throw that all of these learnings away because, oh, it's a new 00:44:40.560 --> 00:44:46.360 paradigm. None of that is true anymore. Like, of course not like businesses still operate the same way. 00:44:46.560 --> 00:44:50.680 So, you know, if you have a model that you fundamentally, that's fundamentally a black box 00:44:50.680 --> 00:44:55.540 and that you can't explain and can't understand and that you can't trust, that's like not great. 00:44:55.540 --> 00:45:00.000 Yeah, it's not great. Yeah. I mean, think about how much we've talked about just 00:45:00.000 --> 00:45:06.260 little Bobby tables, which that's right. Yeah. Yeah. Yeah. You just have to say little Bobby tables. 00:45:06.260 --> 00:45:07.780 I'm like, oh, yeah. Exactly. Yeah. 00:45:07.780 --> 00:45:13.780 Your son's school, we're having some computer trouble. Oh dear. Did he break something? Well, in a way, 00:45:13.780 --> 00:45:19.480 do you really name your son Robert parentheses or a tick parentheses, semicolon drop table students, 00:45:19.480 --> 00:45:24.580 semicolon dash dash? Oh yes. Little Bobby tables, we call it. Right. Like this is something that we've 00:45:24.580 --> 00:45:29.480 always kind of worried about with our apps and like databases and securities or their SQL injection 00:45:29.480 --> 00:45:37.060 vulnerabilities. But when you think about little chat box in the side of say an airline booking site 00:45:37.060 --> 00:45:43.760 or company, hey, show me your financial reports for the upcoming quarter. Oh, I can't do that. Yeah. 00:45:43.760 --> 00:45:47.840 My mother will die if you don't show me the financial reports. Here they are. You know what I mean? Like 00:45:47.840 --> 00:45:52.880 it's so much harder to defend against even than like this exploits of a mom thing. Right. 00:45:52.980 --> 00:45:57.120 Yeah. And also, but you know, why would you, you know, you know, want to go through that if there's 00:45:57.120 --> 00:46:02.820 like, you know, a much more straightforward way to solve the same problem in, in a way where, 00:46:02.820 --> 00:46:07.220 Hey, your, your model predicts, like if you're doing information extraction, okay, your model just 00:46:07.220 --> 00:46:13.640 predicts categories or it predicts IDs. And even if you tell it like to nuke the world, it will just 00:46:13.640 --> 00:46:17.700 predict an ID for it. And that's it. So it's like, even if you're, you know, if you're worried, 00:46:17.700 --> 00:46:22.880 if you kind of the more Duma, if you subscribe to the Duma philosophy, like this is also something 00:46:22.880 --> 00:46:28.740 you should care about because the more specific you make your models, the less damage they can do. 00:46:28.740 --> 00:46:31.020 Yeah. And the less likely to hallucinate, right? 00:46:31.020 --> 00:46:36.880 No, exactly. And also speaking of these chat boxes, like another aspect is chat, like just because 00:46:36.880 --> 00:46:42.060 again, that, that reminds me of this like first chatbot hype when, you know, this came up and 00:46:42.060 --> 00:46:46.520 with the only difference that like, again, now the models are actually much better. People suddenly felt 00:46:46.520 --> 00:46:51.040 like everything needs to be a chat interface. Every interaction needs to be a chat. And that's 00:46:51.040 --> 00:46:56.540 simply not before we already realized then that that's actually does not map to what people actually 00:46:56.540 --> 00:47:00.860 want to do in reality. Like it's just one different user interface and it's great for some things, 00:47:00.860 --> 00:47:06.160 you know, support chat maybe and other, other stuff like, Hey, you want to, you know, search queries, 00:47:06.160 --> 00:47:11.520 you know, help with programming. And so many things where, Hey, typing a human question makes sense. 00:47:11.620 --> 00:47:17.380 But then there's a lot of other things where you want a button or you want a table and you want like, 00:47:17.380 --> 00:47:22.240 and it's just a different type of user interface. And just because, you know, you can make something a 00:47:22.240 --> 00:47:26.860 chat doesn't mean that you should. And sometimes, you know, it just adds like, it adds so much 00:47:26.860 --> 00:47:30.560 complexity to an interaction. There could just be a button click. 00:47:30.720 --> 00:47:34.740 The button click is a very focused prompt or whatever, right? Yeah, exactly. 00:47:34.740 --> 00:47:38.980 Yeah. Even if it's about like, Hey, your earnings reports or something, you wanted to see a table of 00:47:38.980 --> 00:47:45.500 stuff and sum it up at the end. You don't want your model to confidently say 2 million. That's not, 00:47:45.500 --> 00:47:50.140 you know, solving the problem. If you're a business analyst, like you want to see stuff. So, 00:47:50.140 --> 00:47:55.080 yeah. And that actually also sort of ties into, yeah. Another point that I've also had in the talk, 00:47:55.120 --> 00:47:59.880 which is around like actually looking at what are actually the things we try to solve in industry and 00:47:59.880 --> 00:48:05.520 how have these things changed? And while there is new stuff you can now do like generating text and 00:48:05.520 --> 00:48:12.460 that finally works. Yay. There's also a lot of stuff around text goes in, structured data comes out and 00:48:12.460 --> 00:48:16.900 that structured data needs to be machine readable, not human readable, like needs to go into some other 00:48:16.900 --> 00:48:22.880 process. And a lot of industry problems, if you really think about it, have not changed very much. 00:48:22.960 --> 00:48:27.880 They've only changed in scale. Like we started with index cards. Well, there's just kind of limit of 00:48:27.880 --> 00:48:32.520 how much you can do with that and how many projects you could do at the same time. But this was always, 00:48:32.520 --> 00:48:37.580 even since before computers, this has always been bringing structure into unstructured data has always 00:48:37.580 --> 00:48:41.900 been the fundamental challenge. And that's not going to just magically go away because we have new 00:48:41.900 --> 00:48:44.620 capacity capacities and new things we can do. 00:48:44.620 --> 00:48:50.040 Let's talk about some of the workflows here. So you have an example where, you know, you take a large 00:48:50.040 --> 00:48:58.260 model and do some prompting and this sort of iterative model assisted data annotation. Like what's that look like? 00:48:58.260 --> 00:49:03.240 You start out with this model, maybe, you know, one of these models that you can run locally, an API 00:49:03.240 --> 00:49:11.020 during development time, and you prompt it to produce some structured output, for example, or some answer. 00:49:11.360 --> 00:49:16.320 You know, we also have like, for example, you can use something like spaCy LLM that lets you, you know, 00:49:16.320 --> 00:49:21.460 plug in any model in the same way you would otherwise, you know, train a model yourself. 00:49:21.460 --> 00:49:26.900 And then you look at the results, you can actually get a good feel for how is your model even doing. 00:49:27.340 --> 00:49:33.360 And you can also, before you really get into distilling a model, you can create some data to evaluate it. 00:49:33.360 --> 00:49:37.460 Because I think that's something people are often forgetting because it's kind of not, it's not, 00:49:37.460 --> 00:49:41.980 maybe not the funnest part, but it's really, you know, it's like writing tests. 00:49:41.980 --> 00:49:45.840 It's like writing tests can be frustrating. I remember when I kind of started out, like, 00:49:45.840 --> 00:49:51.240 tests are frustrating because they actually kind of, they turn up all of these edge cases and mistakes 00:49:51.240 --> 00:49:52.940 that you kind of want to forget about. 00:49:53.140 --> 00:49:54.920 Right. Oh, I forgot to test for this. Whoops. 00:49:54.920 --> 00:49:59.240 Yeah. Yeah. And then like, oh, if you start writing tests and you suddenly see all the stuff 00:49:59.240 --> 00:50:03.100 that goes wrong and then you have to fix it and it's like, it's annoying. So you better just 00:50:03.100 --> 00:50:09.300 not have tests. I can see that. But like evaluation is kind of like that. And it's ultimately a lot of 00:50:09.300 --> 00:50:15.320 these problems, you, you have to know what you want and here's the input, here's the expected output. 00:50:15.320 --> 00:50:20.120 You kind of have to have to define that. And that's not something any AI can help you with because, 00:50:20.120 --> 00:50:23.180 you know, you are trying to teach the machine. You're teaching the AI. 00:50:23.180 --> 00:50:27.480 You want to, yeah, you want to build something that does what you want. So you kind of need examples 00:50:27.480 --> 00:50:32.320 where you know the answer. And then you can also evaluate like, hey, how does this model do out of the box 00:50:32.320 --> 00:50:39.460 for like some easy tasks? Like, hey, you might find something like GPT-4 can give you 75% accuracy 00:50:39.460 --> 00:50:43.840 out of the box without, without any work. So that's, that's kind of good or even higher. 00:50:44.120 --> 00:50:48.760 Sometimes it's like, if it's a bit harder, you'll see, oh, okay, you went like 20% accuracy, 00:50:48.760 --> 00:50:54.020 which is kind of, which is pretty bad. And the bar is very low, but that's kind of the ballpark that 00:50:54.020 --> 00:50:58.560 you're also looking to beat. And then you can look at examples that are predicted by the model. All you 00:50:58.560 --> 00:51:04.440 have to do is look at them. Yes. Correct. If not, you make a small correction and then you go through 00:51:04.440 --> 00:51:07.520 that and you do basically do that until you've beat the baseline. 00:51:07.520 --> 00:51:11.700 The transfer learning aspect, right? Yeah. And then you use transfer learning in order, 00:51:11.700 --> 00:51:16.560 you know, to give the model like the solid foundation of knowledge about the language and 00:51:16.560 --> 00:51:21.600 the world. And you can end up with a model that's much smaller than what you started with. And you 00:51:21.600 --> 00:51:26.760 have a model that's really has a task network that's only trained to do one specific thing. 00:51:26.760 --> 00:51:32.300 Which brings us from going from prototype to production, where you can sort of try some of 00:51:32.300 --> 00:51:36.820 these things out, but then maybe not run a giant model, but something smaller, right? 00:51:36.820 --> 00:51:40.540 Yeah. Yeah. And you can take all these aspects basically that you're interested in, 00:51:40.540 --> 00:51:46.500 in the larger model and train components that do exactly that. And another thing that's also 00:51:46.500 --> 00:51:53.020 good or helpful here is to have kind of a good path from prototype to production. I think that's also 00:51:53.020 --> 00:51:58.240 where a lot of, yeah, machine learning projects in general often fail because it's all, you know, 00:51:58.240 --> 00:52:02.600 you have this nice prototype and it all looks promising and you've hacked something together in your 00:52:02.600 --> 00:52:08.880 Jupyter notebook and that's all looking nice. You maybe have like a nice streamlit demo and you can 00:52:08.880 --> 00:52:14.060 show that, but then you're like, okay, can we ship that? And then if your workflow that leads to the 00:52:14.060 --> 00:52:19.160 prototype is completely different from the workflow that leads to production, you might find that out 00:52:19.160 --> 00:52:25.720 exactly at that phase. And that's kind of where projects go to die. And that's sad. And yeah, so that's, 00:52:25.760 --> 00:52:29.560 that's actually something we've been thinking about a lot. And also what we've kind of been trying to 00:52:29.560 --> 00:52:34.240 achieve with spaCy LLM where you have this LLM component that you plug in and it does exactly the 00:52:34.240 --> 00:52:40.920 same as the components would do at runtime. And it really just slots in and then might use GPT-4 behind 00:52:40.920 --> 00:52:47.380 the scenes to create the exact same structured object. And then you can swap that out. Or maybe, 00:52:47.520 --> 00:52:52.440 you know, you, there are a lot of things you might even want to swap out with rules or no AI at all. 00:52:52.440 --> 00:52:58.500 Like, you know, like a ChatGPT is good at recognizing US addresses and it's great to build a prototype, 00:52:58.500 --> 00:53:04.400 but instead of asking it to extract US addresses, for example, you can ask it, give me spaCy rules, 00:53:04.400 --> 00:53:10.080 match your rules for US addresses. And it can actually do that pretty well. And then you can bootstrap from there. 00:53:10.080 --> 00:53:10.600 Oh, nice. 00:53:10.600 --> 00:53:14.260 There's a lot of stuff like that that you can do. And there might be cases where you find that, 00:53:14.640 --> 00:53:21.040 yeah, you can totally beat any model accuracy and have a much more deterministic approach if you just 00:53:21.040 --> 00:53:24.500 write a regex. Like that's still true. 00:53:24.500 --> 00:53:25.380 It does still work. 00:53:25.380 --> 00:53:29.900 Yeah. It's still something, it's easy to forget because, you know, again, if you look at research 00:53:29.900 --> 00:53:34.660 and literature, nobody's talking about that because this is not an interesting research question. 00:53:34.660 --> 00:53:40.640 Like nobody cares. You know, you can take any benchmark and say, I can beat ChatGPT accuracy with 00:53:40.640 --> 00:53:45.600 two regular expressions. And that's like, that's true. Probably in some cases. 00:53:45.600 --> 00:53:45.880 Yeah. 00:53:45.880 --> 00:53:48.960 It's like nobody cares. Like that's not, that's not research. 00:53:48.960 --> 00:53:54.800 For sure. But you know, what is nice to do is to go to ChatGPT or LM studio or whatever and say, 00:53:54.800 --> 00:54:00.480 I need a Python based regular expression to match this text and this text. And I want to capture group for 00:54:00.480 --> 00:54:04.740 that. And I don't want to think about it. It's really complicated. Here you go. Oh, perfect. 00:54:05.020 --> 00:54:06.280 Now I run the regular expression. 00:54:06.280 --> 00:54:10.240 Yeah. That's actually, that's a good use case. I've still been sort of hacking around on like 00:54:10.240 --> 00:54:15.140 these, you know, interactive regex editors because I'm, I'm not particularly good at regular expressions. 00:54:15.140 --> 00:54:15.740 Neither am I. 00:54:15.740 --> 00:54:20.640 Like on the scale, like I can do it, but like, I know people who really, I think my co-founder, 00:54:20.640 --> 00:54:25.320 Matt, he, he worked through it. Like he's more the type who really approaches these things very 00:54:25.320 --> 00:54:29.960 methodically. And he was like, now he, he wants read this one big book on regular expressions. 00:54:29.960 --> 00:54:36.080 And like, he really did it like the hardcore way, but like, that's why he's obviously much better than I am. 00:54:36.080 --> 00:54:40.840 I consider regular expressions kind of write only. Like you can write them and make them do stuff, 00:54:40.840 --> 00:54:42.240 but then reading them back is tricky. 00:54:42.240 --> 00:54:42.580 Yeah. 00:54:42.580 --> 00:54:48.400 At least for me. All right, let's wrap this up. So what are the things that you did here at the end 00:54:48.400 --> 00:54:53.020 of your presentation, which I want to kind of touch on is you brought back some of the same ideas 00:54:53.020 --> 00:54:58.380 that we had for like, what are the values of open source or why open source, but back to 00:54:58.380 --> 00:55:02.100 creating these smaller focused models. Talk us through this. 00:55:02.100 --> 00:55:05.880 How specific components. Yeah. I mean, if you kind of look at, Hey, what are the, you know, 00:55:05.880 --> 00:55:11.360 the advantages of sort of approach that we talked about of distilling things down of creating these 00:55:11.360 --> 00:55:17.060 smaller models, a lot of it comes down to it being like it's modular. You, again, you're not locked in 00:55:17.060 --> 00:55:22.260 to anything. You own the model. Nobody can take that away from you. It's easier to write tests. 00:55:22.260 --> 00:55:26.900 You have the flexibility. You can extend it because, you know, it's cold. You can program 00:55:26.900 --> 00:55:31.940 with it because often it very rarely you do machine learning for the sake of machine learning. It's 00:55:31.940 --> 00:55:36.340 always like there is some other process. You populate a database, you do some other stuff 00:55:36.340 --> 00:55:41.100 with your stack. And so you want to program with it. It needs to be affordable. You want to understand 00:55:41.100 --> 00:55:46.640 it. You need to be able to say, why is it doing what it is? Like, what do I do to fix it? It again, 00:55:46.640 --> 00:55:51.840 runs in-house it's entirely private. And then, yeah, when I, you know, was kind of thinking 00:55:51.840 --> 00:55:56.480 about this, I realized like, oh, actually, you know, it like this really maps exactly 00:55:56.480 --> 00:56:01.120 the reasons why that we saw it. We talked about earlier why people choose open source or companies. 00:56:01.120 --> 00:56:06.320 And that's obviously not a coincidence. It's because ultimately these are principles that 00:56:06.320 --> 00:56:12.000 we have come up with over a long period of time of, yeah, that's good software development. 00:56:12.000 --> 00:56:17.840 And ultimately AI is just another type of software development. So, of course, it makes sense that 00:56:17.840 --> 00:56:23.600 the same principles make sense and are beneficial. And that, you know, just having a workflow where 00:56:23.600 --> 00:56:29.120 everything's a black box and third party, this can work for prototyping, but it's not. 00:56:29.120 --> 00:56:35.040 That kind of goes against a lot of the things that we've identified as very useful in applied settings. 00:56:35.040 --> 00:56:37.520 Absolutely. All right. So we have to answer the question. 00:56:37.520 --> 00:56:38.400 Oh, the... 00:56:38.400 --> 00:56:40.720 Will it be monopolized? 00:56:40.720 --> 00:56:46.160 And your contention is no, that open source wins even for LLMs. Yeah. 00:56:46.160 --> 00:56:51.760 Open source means there's no monopoly to be gained in AI. You know, I've kind of broken it down into 00:56:51.760 --> 00:56:56.640 some of these strategies, which, you know, how do you get to a monopoly? And these are like, you know, 00:56:56.640 --> 00:57:00.880 this is not just some big stuff. These are things like a lot of companies are actively thinking about. 00:57:00.880 --> 00:57:05.440 If you were in a business where, you know, it's winner takes all, like you want to, you know, 00:57:05.440 --> 00:57:10.960 get rid of like all of that competition that companies hate, that investors hate. And there are ways to do 00:57:10.960 --> 00:57:12.640 that and companies really actively think about this. 00:57:12.640 --> 00:57:14.560 Those pesky competitors, let's get rid of them. 00:57:14.560 --> 00:57:19.040 There are different ways to do that. Like one is having this compounding advantage. 00:57:19.040 --> 00:57:22.800 So that's stuff like network effects, like, you know, if your social network, of course, 00:57:22.800 --> 00:57:27.040 that makes a lot of sense. Everyone's on it. If you could kind of have these network effects, 00:57:27.040 --> 00:57:33.680 that's good. And economies of scale. But as we've seen, like economies of scale is a pretty lame mode 00:57:33.680 --> 00:57:38.880 in that respect. Like that has a lot of, you know, a lot of limitations. It's not even fully true. 00:57:38.880 --> 00:57:41.840 It's kind of the opposite of a monopoly in some ways. 00:57:41.840 --> 00:57:43.520 Yeah. Especially in software. 00:57:43.520 --> 00:57:49.200 Yeah. In software. Exactly. So it's like, I don't think that's, that's not really the way to go. 00:57:49.200 --> 00:57:54.880 One example that comes to mind, at least for me, maybe I'm seeing it wrong, but Amazon, 00:57:54.880 --> 00:58:01.040 amazon.com. Just, you know, how many companies can have massive warehouse with everything at by every 00:58:01.040 --> 00:58:02.080 single person's house? 00:58:02.080 --> 00:58:06.720 Yeah. The one platform that everyone goes on. So even if you're a retailer, you kind of, yeah, 00:58:06.720 --> 00:58:11.680 they feel the Amazon has kind of forced everyone to either sell on Amazon or go bust because... 00:58:11.680 --> 00:58:16.560 Exactly. It's very sad that it's the way it is. Yeah. And then network effects. I'm thinking, 00:58:16.560 --> 00:58:21.520 you know, people might say Facebook or something, which is true, but I would say like Slack actually. 00:58:21.520 --> 00:58:22.080 Oh, okay. 00:58:22.080 --> 00:58:27.760 Or Slack or Discord or, you know, there's a bunch of little chat apps and things, but if you want to 00:58:27.760 --> 00:58:31.360 have one and you want to have a little community, you want people to be able to, well, I already 00:58:31.360 --> 00:58:36.800 have Slack open. It's just an icon next to it versus install my own app. Make sure you run it. 00:58:36.800 --> 00:58:40.160 Be sure to check it. Like people are going to forget to run it and you disappear off the, 00:58:40.160 --> 00:58:41.200 off the space, you know? 00:58:41.200 --> 00:58:45.040 That will make sense. And I do think, you know, these things don't necessarily happen accidentally. 00:58:45.040 --> 00:58:48.960 Like companies think about, okay, how do we, you know, Amazon definitely thought about this. 00:58:48.960 --> 00:58:52.880 This didn't just like happen to Amazon. Yes, they were lucky in a lot of ways, but like, 00:58:52.880 --> 00:58:54.560 you know, that's, that's a strategy. 00:58:54.560 --> 00:58:55.520 Exactly. 00:58:55.520 --> 00:58:59.760 Yeah. And then the other thing that's not relevant here is like, you know, another way is controlling 00:58:59.760 --> 00:59:04.240 a resource. That's really more, if you're like, you know, if they are physical, if there was like 00:59:04.240 --> 00:59:05.280 a physical resource. 00:59:05.280 --> 00:59:06.480 I brought the cables. Something like that. 00:59:06.480 --> 00:59:11.840 Yeah. I mean, it's, yeah. Or like in Germany, I think for a long time, the telecom, they owned 00:59:11.840 --> 00:59:13.280 the wires in the building. 00:59:13.280 --> 00:59:13.920 Right. Exactly. 00:59:13.920 --> 00:59:18.560 And they still do, I think. So they used to have the monopoly. Now they don't, but they kind of still, 00:59:18.560 --> 00:59:23.280 to some extent, they still do because they need to come. If even, no matter who you sign up for, 00:59:23.280 --> 00:59:29.120 with, for internet, telecom needs to come and activate it. So if you sign up with telecom, 00:59:29.120 --> 00:59:31.200 you usually get service a lot faster. 00:59:31.200 --> 00:59:34.720 It's a little better service. Yeah, exactly. Don't wait two weeks. Use us. 00:59:34.720 --> 00:59:38.640 That's kind of, that's how it still works, but we don't, we don't really have that here. 00:59:38.640 --> 00:59:43.760 And then the other, the next point that's very attractive. The final one is regulation. So that's 00:59:43.760 --> 00:59:49.200 kind of like, you have to have a monopoly because the government says so. And that is one where we have 00:59:49.200 --> 00:59:56.320 to be careful because if we're not like, if in all of these discussions, we're not making the distinction 00:59:56.320 --> 01:00:01.920 between the models and the actual product, you know, very different characteristics and 01:00:01.920 --> 01:00:08.320 we now do a very different things. If that gets muddy, which like, you know, a lot of also companies 01:00:08.320 --> 01:00:14.800 quite actively do in that discourse, then we might end up in a situation where we sort of accidentally, 01:00:14.800 --> 01:00:20.720 or, you know, gift a company or some companies a monopoly via the regulation. Because if you let 01:00:20.720 --> 01:00:27.360 them write the regulation, for example, and we're not just regulating products, but lumping that in with 01:00:27.360 --> 01:00:30.240 technology itself and open research. 01:00:30.240 --> 01:00:33.920 Yeah. It's a part of your talk. I can't remember if it was the person hosting it or you who brought 01:00:33.920 --> 01:00:40.480 this up, but an example of that might be all the third-party cookie banners, rather than banning, 01:00:40.480 --> 01:00:47.920 just targeted, retargeting and tracking. Like instead of banning through the GDPR, instead of banning 01:00:47.920 --> 01:00:52.160 the thing that is the problem, it's like, let's ban the implementation of the problem. 01:00:52.160 --> 01:00:57.200 That's a risk or that's like, you know, in hindsight, yes, I think in hindsight, we would all agree that 01:00:57.200 --> 01:01:01.920 like, we should have just banned targeted advertising. Instead, what we got is these 01:01:01.920 --> 01:01:06.640 cookie pop-ups. That's like really annoying. And that's actually what I feel like is one of the, 01:01:06.640 --> 01:01:12.160 as much as I think the EU, you know, I'm not an expert on like AI regulation or the EU AI Act, 01:01:12.160 --> 01:01:17.600 but what I'm seeing is at least they did make a distinction between use cases. And it's very much, 01:01:17.600 --> 01:01:22.880 there is a focus on here are the products and the things people are doing, how high risk is that, 01:01:22.880 --> 01:01:27.520 as opposed to how big is the model and how, you know, what does that, because that doesn't, 01:01:27.520 --> 01:01:32.640 doesn't say anything, but that would kind of be a very, very dangerous way to go about it. But the 01:01:32.640 --> 01:01:37.440 risk is, of course, if we're rushing regulate, like if we're a rushing regulation, then, you know, 01:01:37.440 --> 01:01:42.640 we might actually end up with something that's not quite fit for purpose. Or if we let big tech 01:01:42.640 --> 01:01:46.240 companies write the regulation or lobby. Lobby for it to get it. Yeah. 01:01:46.240 --> 01:01:50.560 These are some ideas because, you know, if they're doing that, like, I think it's pretty obvious, 01:01:50.560 --> 01:01:56.480 they're not just worried about the safety of AI and are like appealing to like Congress or whatever. 01:01:56.480 --> 01:02:01.520 Like, I think most people are aware of that, but like, yes, the, I think the intentions are even less 01:02:01.520 --> 01:02:03.760 pure than that. And I think that's a big risk. 01:02:03.760 --> 01:02:05.920 The regulation is very tricky. It's, 01:02:05.920 --> 01:02:10.400 Yeah. And you know, just for the record, I am pro-regulation. I'm very pro-regulation in general, 01:02:10.400 --> 01:02:15.600 but I also think you can, if you fuck up regulation, that can also be very damaging, 01:02:15.600 --> 01:02:16.240 obviously. 01:02:16.240 --> 01:02:21.040 Absolutely. And it can be put in a way so that it makes it hard for competitors to get into 01:02:21.040 --> 01:02:22.000 the system. 01:02:22.000 --> 01:02:22.240 Yeah. 01:02:22.240 --> 01:02:26.640 There's so much paperwork and so much monitoring that you need a team of 10 people just to operate. 01:02:26.640 --> 01:02:28.720 Well, if you, a startup, well, you can't do that because, 01:02:28.720 --> 01:02:31.760 Hey, Art, we got a thousand people and 10 of them work on this. Like, well. 01:02:31.760 --> 01:02:35.920 Even beyond that, like it's, you know, if you think back to all the stuff we talked about, 01:02:35.920 --> 01:02:41.040 like they are, this goes against a lot of the best practices of software. This, this goes, 01:02:41.040 --> 01:02:47.520 you know, this goes against a lot of what we've identified that actually makes good, secure, 01:02:47.520 --> 01:02:54.880 reliable, modular, whatever software, safe software internally. And even doing a lot of the software 01:02:54.880 --> 01:02:59.440 development internally, like there are so many benefits of that. And I think, you know, companies, 01:02:59.440 --> 01:03:05.040 companies actually working on their own product is good. And if it was suddenly true that like 01:03:05.040 --> 01:03:10.320 only certain companies could even provide AI models, I didn't even know what that would mean for open 01:03:10.320 --> 01:03:14.800 source or for academic research. Like that would make absolutely no sense. I also don't think that's 01:03:14.800 --> 01:03:20.400 like really enforceable, but it would mean that, you know, like this would limit like everyone in 01:03:20.400 --> 01:03:24.960 what they could be doing. Like I think, and there's like, you know, there's nothing to do with like, 01:03:24.960 --> 01:03:29.760 there's a lot of other things you can do if you, you know, care about AI safety, but that's really 01:03:29.760 --> 01:03:34.720 not it. And I also, you know, I just think being aware of that is good. I don't like, I can, you 01:03:34.720 --> 01:03:39.280 know, I cannot see an outcome where we really do that. It would, it would really not make sense. I 01:03:39.280 --> 01:03:44.800 could not see the reality of this, you know, shaking out, but I think it's still, it's still relevant. 01:03:44.800 --> 01:03:48.480 Yeah. I think the open source stuff and some of the smaller models really does 01:03:48.480 --> 01:03:50.480 give us a lot of hope. So that's awesome. 01:03:50.480 --> 01:03:56.560 I feel, you know, also very positive about this. I've also talked to a lot of developers at conferences 01:03:56.560 --> 01:04:00.960 that like, yeah, actually thinking and talking about this gave them some hope, which obviously is, 01:04:00.960 --> 01:04:07.040 is nice because I definitely did that some of the vibe I got, like it can be kind of easy to end up a bit 01:04:07.040 --> 01:04:13.440 disillusioned by like a lot of the narratives people here. And that, you know, also, even if you're 01:04:13.440 --> 01:04:18.400 entering the field, you're like, wait, a lot of this doesn't really make sense. Like, why is it like, 01:04:18.400 --> 01:04:23.120 like this? Yeah. It's like, no, it actually, you know, your intuition is right. Like a lot of software, 01:04:23.120 --> 01:04:29.760 software engineering best practices, of course, still matter. And, you know, no, they are like, 01:04:29.760 --> 01:04:33.520 you know, they are better ways that we're not, you know, we're not just going in that direction. 01:04:33.520 --> 01:04:37.600 And I think I definitely believe in that. A lot of the reasons why open source won in a whole bunch 01:04:37.600 --> 01:04:42.960 of areas. Yeah. Could be exactly why it wins at LLMs as well, right? Yep. And, you know, again, 01:04:42.960 --> 01:04:49.120 it's all based on open research. A lot of stuff is already published and there's no secret source. 01:04:49.120 --> 01:04:55.360 The software, you know, software industry does not run on like secrets. All the differentiators are 01:04:55.360 --> 01:05:01.920 product stuff. And yes, you know, open AI might monopolize or dominate AI powered chat assistants, 01:05:01.920 --> 01:05:06.240 or maybe Google will do like that's, you know, that's a whole race that, you know, if you're not in that 01:05:06.240 --> 01:05:10.880 business, you don't have to be a part of, but that does not mean that anyone's going to win at or 01:05:10.880 --> 01:05:15.520 monopolize AI. Those are very different things. Absolutely. All right. A good place to leave it 01:05:15.520 --> 01:05:20.080 as well. You know, thanks for being here. Yeah. Thanks. That was fun. Yeah. Yeah. People want to 01:05:20.080 --> 01:05:24.160 learn more about the stuff that you're doing, maybe check out the video of your talks or whatever. What 01:05:24.160 --> 01:05:27.920 do you recommend? Yeah, I think I definitely, I'll definitely give you some links for the show notes 01:05:27.920 --> 01:05:32.960 like this. The slides are online, so you can have a look at that. There is at least one recording of the 01:05:32.960 --> 01:05:39.040 talk online now from the really cool Python, PyCon Lithuania. It was my first time in Lithuania this 01:05:39.040 --> 01:05:43.840 year. Definitely. You know, if you have a chance to visit their conference, it was a lot of fun. 01:05:43.840 --> 01:05:50.080 I learned a lot about Lithuania as well. We also on our website, Explosion AI, we publish kind of this 01:05:50.080 --> 01:05:56.320 feed of like all kinds of stuff that's happening from maybe some talk or podcast interview, community 01:05:56.320 --> 01:06:02.240 stuff. There's still like a lot of super interesting plugins that are developed by people in community 01:06:02.240 --> 01:06:06.880 papers that are published. So we really try to give a nice overview of everything that's happening in our 01:06:06.880 --> 01:06:13.840 ecosystem. And then of course, you could try out spaCy, spaCy LLM. You know, if you want to try out 01:06:13.840 --> 01:06:20.080 some of these generative models and especially for prototyping or production, whatever you want to do 01:06:20.080 --> 01:06:26.960 for structured data. If you're any of the conferences and check out the list of events and stuff, 01:06:26.960 --> 01:06:32.960 I'm going to do a lot of travel this year. So I would love to catch up with more developers in person 01:06:32.960 --> 01:06:37.440 and also learn more about all the places I'm visiting. So that's cool. I've seen the list. It's very, 01:06:37.440 --> 01:06:43.360 very comprehensive. So I kind of a neat freak. I like to, I also be very much like to organize things in 01:06:43.360 --> 01:06:48.000 that way. So yeah. So there might be something local for people listening now that you're going to be 01:06:48.000 --> 01:06:52.240 doing all right. Well, as always, thank you for being on the show. It's great to chat with you. 01:06:52.240 --> 01:06:53.440 Yeah. Thanks. Yeah. Thanks. 01:06:53.440 --> 01:06:54.080 So next time. 01:06:54.080 --> 01:06:54.320 Bye. 01:06:54.320 --> 01:07:00.880 This has been another episode of Talk Python To Me. Thank you to our sponsors. Be sure to check 01:07:00.880 --> 01:07:05.680 out what they're offering. It really helps support the show. Take some stress out of your life. Get 01:07:05.680 --> 01:07:11.440 notified immediately about errors and performance issues in your web or mobile applications with Sentry. 01:07:11.440 --> 01:07:17.920 Just visit talkpython.fm/sentry and get started for free. And be sure to use the promo code, 01:07:17.920 --> 01:07:23.680 talkpython, all one word. This episode is sponsored by Porkbun. Launching a successful 01:07:23.680 --> 01:07:29.440 project involves many decisions, not the least of which is choosing a domain name. Get a .app, .dev, 01:07:29.440 --> 01:07:36.240 or .food domain name at Porkbun for just $1 for the first year at talkpython.fm/porkbun. 01:07:36.240 --> 01:07:41.680 Want to level up your Python? We have one of the largest catalogs of Python video courses over at Talk Python. 01:07:41.680 --> 01:07:47.600 Our content ranges from true beginners to deeply advanced topics like memory and async. And best of all, 01:07:47.600 --> 01:07:52.080 there's not a subscription in sight. Check it out for yourself at training.talkpython.fm. 01:07:52.080 --> 01:08:07.680 Be sure to subscribe to the show, open your favorite podcast app, and search for Python. We should be right at the top. You can also find the iTunes feed at /itunes, the Google Play feed at /play, and the direct RSS feed at /rss on talkpython.fm. 01:08:07.680 --> 01:08:12.480 We're live streaming most of our recordings these days. If you want to be part of the show and have your 01:08:12.480 --> 01:08:18.560 comments featured on the air, be sure to subscribe to our YouTube channel at talkpython.fm/youtube. 01:08:18.560 --> 01:08:33.600 This is your host, Michael Kennedy. Thanks so much for listening. I really appreciate it. Now get out there and write some Python code. 01:08:33.600 --> 01:08:57.720 Thank you.