WEBVTT 00:00:00.001 --> 00:00:05.700 We all know that LLMs and generative AI have been working their way into many products. 00:00:05.700 --> 00:00:08.560 Well, it's Jupyter's turn to get a really awesome integration. 00:00:08.560 --> 00:00:11.900 We have David Kui here to tell us about Jupyter AI. 00:00:11.900 --> 00:00:17.960 Jupyter AI provides a user-friendly and powerful way to apply generative AI to your notebooks. 00:00:17.960 --> 00:00:23.380 It lets you choose from many different LLM providers and models to get just the help that you're looking for. 00:00:23.380 --> 00:00:27.720 And it does way more than just add a chat pane in the UI. 00:00:27.720 --> 00:00:29.200 Listen in to find out. 00:00:29.760 --> 00:00:34.500 This is Talk Python To Me, episode 440, recorded October 30th, 2023. 00:00:34.500 --> 00:00:52.320 Welcome to Talk Python To Me, a weekly podcast on Python. 00:00:52.320 --> 00:00:54.060 This is your host, Michael Kennedy. 00:00:54.060 --> 00:00:59.160 Follow me on Mastodon, where I'm @mkennedy and follow the podcast using @talkpython. 00:00:59.520 --> 00:01:01.540 Both on Bostodon.org. 00:01:01.540 --> 00:01:06.640 Keep up with the show and listen to over seven years of past episodes at talkpython.fm. 00:01:06.640 --> 00:01:10.360 We've started streaming most of our episodes live on YouTube. 00:01:10.360 --> 00:01:17.960 Subscribe to our YouTube channel over at talkpython.fm/youtube to get notified about upcoming shows and be part of that episode. 00:01:19.260 --> 00:01:23.280 This episode is sponsored by Posit Connect from the makers of Shiny. 00:01:23.280 --> 00:01:27.820 Publish, share, and deploy all of your data projects that you're creating using Python. 00:01:27.820 --> 00:01:34.280 Streamlit, Dash, Shiny, Bokeh, FastAPI, Flask, Quattro, Reports, Dashboards, and APIs. 00:01:34.280 --> 00:01:36.680 Posit Connect supports all of them. 00:01:36.680 --> 00:01:41.040 Try Posit Connect for free by going to talkpython.fm/posit. 00:01:41.380 --> 00:01:42.340 P-O-S-I-T. 00:01:42.340 --> 00:01:46.740 And it's also brought to you by us over at Talk Python Training. 00:01:46.740 --> 00:01:51.360 Did you know that we have over 250 hours of Python courses? 00:01:51.360 --> 00:01:52.560 Yeah, that's right. 00:01:52.560 --> 00:01:55.140 Check them out at talkpython.fm/courses. 00:01:57.400 --> 00:01:59.260 David, welcome to Talk Python To Me. 00:01:59.260 --> 00:02:00.180 Awesome to have you here. 00:02:00.180 --> 00:02:01.020 Yeah, thank you, Michael. 00:02:01.020 --> 00:02:04.320 I'm really excited to see what the AIs have to say today. 00:02:04.320 --> 00:02:05.440 The AIs? 00:02:05.440 --> 00:02:06.820 Yeah, language models, sure. 00:02:06.820 --> 00:02:08.440 Yes, exactly, exactly. 00:02:08.440 --> 00:02:15.460 Now, you've built a really cool extension for Jupyter that plugs in large language models for people. 00:02:15.460 --> 00:02:17.200 And it's looking super interesting. 00:02:17.200 --> 00:02:19.200 So I'm excited to talk to you about it. 00:02:19.200 --> 00:02:21.960 Yeah, I'm excited to talk about Jupyter AI, too. 00:02:21.960 --> 00:02:24.980 I've actually presented this twice at one set. 00:02:24.980 --> 00:02:26.100 Actually, three times. 00:02:26.500 --> 00:02:30.700 I did a short demo at this tech meetup thing in Seattle. 00:02:30.700 --> 00:02:33.740 That was actually the first time Jupyter AI was shown to the public. 00:02:33.740 --> 00:02:38.420 And then I presented at PyData Seattle at Microsoft's Redmond campus. 00:02:38.420 --> 00:02:43.680 And then I got to present again at JupyterCon in Paris this May. 00:02:43.680 --> 00:02:45.700 It was a really wonderful experience. 00:02:45.700 --> 00:02:46.240 But yeah. 00:02:46.240 --> 00:02:46.380 Wow. 00:02:46.380 --> 00:02:47.920 Yeah, you're making the rounds. 00:02:47.920 --> 00:02:48.840 Yeah. 00:02:48.840 --> 00:02:50.580 I love to talk about Jupyter AI. 00:02:50.580 --> 00:02:52.800 It happens to get me some plane tickets. 00:02:52.800 --> 00:02:55.360 Just joking. 00:02:55.600 --> 00:02:59.840 Honestly, that's like half the bonus of conferences is like awesome places you get to go. 00:02:59.840 --> 00:03:02.000 The other half probably is the people you meet. 00:03:02.000 --> 00:03:02.800 You know, it's really cool. 00:03:02.800 --> 00:03:05.420 Oh, for me, it's like almost like all the people. 00:03:05.420 --> 00:03:08.180 Like the people are just so great, especially JupyterCon. 00:03:08.180 --> 00:03:08.580 Yeah. 00:03:08.580 --> 00:03:12.380 And the JupyterCon videos are now out for JupyterCon 2023. 00:03:12.380 --> 00:03:14.740 And there's a ton of good looking talks there. 00:03:14.940 --> 00:03:15.340 Yeah. 00:03:15.340 --> 00:03:16.960 Lots of really smart people. 00:03:16.960 --> 00:03:20.560 I mean, like I was chatting to a few folks there. 00:03:20.560 --> 00:03:29.440 And that's like the only place where you're going to find like people who work at these hedge funds and trading firms just lounging so idly and casually. 00:03:29.440 --> 00:03:29.940 Right. 00:03:29.940 --> 00:03:30.700 Like. 00:03:31.680 --> 00:03:32.080 Yeah. 00:03:32.080 --> 00:03:33.400 The market's opening and they're chilling. 00:03:33.400 --> 00:03:34.160 It's all fine. 00:03:34.160 --> 00:03:34.500 Yeah. 00:03:34.500 --> 00:03:34.900 Yeah. 00:03:34.900 --> 00:03:36.840 It's a, there's a lot of smart people there. 00:03:36.840 --> 00:03:37.100 Yeah. 00:03:37.100 --> 00:03:44.660 Jupyter more than a lot of programming technologies bring people from all sorts of different places together, different backgrounds. 00:03:44.840 --> 00:03:48.760 There's like a huge, yeah, there's like a lot of reasons behind that. 00:03:48.760 --> 00:03:51.080 But long story short, Jupyter's pretty awesome. 00:03:51.080 --> 00:03:53.240 And that's kind of why I work to contribute to it. 00:03:53.240 --> 00:03:53.540 Awesome. 00:03:53.540 --> 00:03:57.700 Well, let's start this whole conversation with a bit of background about yourself. 00:03:57.700 --> 00:04:02.120 And for people who didn't see your talk and don't know you yet, tell them a bit about you. 00:04:02.120 --> 00:04:05.140 I didn't really give much of an intro there either, but sure. 00:04:05.140 --> 00:04:06.360 Yeah. 00:04:06.360 --> 00:04:08.900 So I've worked for AWS as a software engineer. 00:04:09.320 --> 00:04:15.000 And I've been with AWS, specifically the AI ML organization at AWS. 00:04:15.000 --> 00:04:17.840 I've been with them for almost two years now. 00:04:17.840 --> 00:04:23.520 Right now, I, my manager is actually Brian Granger, who's the co-founder of Project Jupyter. 00:04:23.520 --> 00:04:25.080 He also works for AWS. 00:04:25.080 --> 00:04:25.600 Yeah. 00:04:25.600 --> 00:04:31.960 So he's been offering some technical and product guidance for the things that we're building. 00:04:31.960 --> 00:04:35.000 And he's a fantastic gentleman to work with. 00:04:35.000 --> 00:04:35.240 Cool. 00:04:35.340 --> 00:04:35.420 Yeah. 00:04:35.420 --> 00:04:40.520 It's a really neat, neat to have him available as a resource, you know, as a colleague. 00:04:40.520 --> 00:04:40.920 Yeah. 00:04:40.920 --> 00:04:41.760 You know, it's funny. 00:04:41.760 --> 00:04:44.720 Like I actually, yeah, I met him internally. 00:04:44.720 --> 00:04:49.920 So when I first joined, I wasn't working for him, but at tech companies, you can do this internal 00:04:49.920 --> 00:04:50.800 transfer thing. 00:04:50.800 --> 00:04:56.100 And basically my old team was kind of the team I just joined right after I joined, it sort 00:04:56.100 --> 00:05:01.680 of started to dissolve a little because they just launched a product at reInvent, which happens 00:05:01.680 --> 00:05:03.080 in like December. 00:05:03.620 --> 00:05:07.180 And then, so I joined in December and it's like, oh, hi. 00:05:07.180 --> 00:05:09.380 So then I, yeah. 00:05:09.380 --> 00:05:14.920 And then I joined, I messaged, I saw just, saw Brian Granger's name somehow and I messaged 00:05:14.920 --> 00:05:18.580 him and I didn't even know that he was the co-founder of Project Jupyter. 00:05:18.580 --> 00:05:21.540 I just wanted to work for him because I used it before. 00:05:21.540 --> 00:05:23.760 And yeah, it's a pretty funny story. 00:05:23.760 --> 00:05:24.160 Indeed. 00:05:24.160 --> 00:05:29.320 I imagine that project is, is really, you know, this Jupyter AI is a great example, but 00:05:29.320 --> 00:05:34.160 I just thinking of being say a founder of Jupyter or, or something like that. 00:05:34.160 --> 00:05:36.460 And these things take on a life of their own. 00:05:36.460 --> 00:05:41.420 And he's probably in awe of all the stuff happening and all the things going on. 00:05:41.420 --> 00:05:43.240 And there's probably a lot of stuff in Jupyter. 00:05:43.340 --> 00:05:44.700 He doesn't even know about, right? 00:05:44.700 --> 00:05:45.500 It's like, that's happening. 00:05:45.500 --> 00:05:46.520 Yeah, it's huge. 00:05:46.520 --> 00:05:47.140 And yeah. 00:05:47.140 --> 00:05:50.440 And like the leadership structure has changed to accommodate that. 00:05:50.440 --> 00:05:54.320 So Brian is no longer the benevolent dictator for life. 00:05:54.320 --> 00:05:59.840 Jupyter Project Jupyter is now governed by a committee decentralized and democratized just 00:05:59.840 --> 00:06:00.740 to allow it to scale. 00:06:00.740 --> 00:06:01.540 Yeah, of course. 00:06:01.940 --> 00:06:07.600 Let's start by talking about a bit of the role of AI and data science. 00:06:07.600 --> 00:06:09.000 I don't know how you feel about it. 00:06:09.000 --> 00:06:14.080 Obviously you must be somewhat of an advocate putting this much time and energy into bringing 00:06:14.080 --> 00:06:14.720 it to Jupyter. 00:06:14.720 --> 00:06:15.160 Wow. 00:06:15.160 --> 00:06:22.340 However, personally, when I want to know something, I don't think there's a great specific 00:06:22.340 --> 00:06:27.020 search result for it straight to ChatGPT or friends. 00:06:27.020 --> 00:06:31.920 I think there's, there's such a, such a wealth of information there from, I need to take 00:06:31.920 --> 00:06:35.860 this paragraph and clean it up and make it sound better to, I have this program. 00:06:35.860 --> 00:06:41.080 I want to convert to another language, or I have this data in this website. 00:06:41.080 --> 00:06:42.120 How do I get it? 00:06:42.120 --> 00:06:47.180 You know, like just, you can ask so many open-ended questions and really get great answers. 00:06:47.180 --> 00:06:51.880 So it seems to me, especially coming from, like I mentioned before, those diverse backgrounds, 00:06:51.880 --> 00:06:56.680 people not necessarily being like super deep in programming, maybe they're deep in finance, 00:06:56.680 --> 00:07:01.740 but they do programming that having this AI capability to ask like, Hey, I know I 00:07:01.740 --> 00:07:05.500 can, but how, what do you think for data science in particular? 00:07:05.500 --> 00:07:09.440 This is an interesting topic, right? 00:07:09.440 --> 00:07:13.440 Because I think the whole power of language models stems from their ubiquity and versatility 00:07:13.440 --> 00:07:16.960 and how they can sort of be very generally applicable. 00:07:16.960 --> 00:07:22.060 So like the thing about language models is that they're basically statistical models that have 00:07:22.060 --> 00:07:25.300 been trained on a very, very, very large corpus of data. 00:07:25.300 --> 00:07:28.160 And that's really to the computer. 00:07:28.160 --> 00:07:31.000 The computer doesn't really understand English. 00:07:31.000 --> 00:07:32.960 It doesn't really understand natural language. 00:07:32.960 --> 00:07:40.140 It has, it's basically making, it has like, when it's trained, it has like knowledge of the 00:07:40.140 --> 00:07:45.180 distribution of information and how information sort of interacts with other information. 00:07:45.180 --> 00:07:49.520 Because of that, it's a very, it has a very general applicability, right? 00:07:49.520 --> 00:07:53.060 And I don't think that's utilities limited to data science. 00:07:53.420 --> 00:07:58.480 Now, if we're talking about the field of data science specifically, I think language models 00:07:58.480 --> 00:08:05.020 have extraordinary utility and explanatory natural language tests, which I think everybody is 00:08:05.020 --> 00:08:08.740 aware of now, now that ChatGPT has been out for almost a year. 00:08:08.740 --> 00:08:14.900 But I think in the field of data science and other like deep technical fields, they're especially 00:08:14.900 --> 00:08:18.480 applicable because of how complicated some of the work is. 00:08:18.480 --> 00:08:24.040 ChatAI can also help analyze and debug code, which JupyterLab also allows you to do. 00:08:24.040 --> 00:08:28.960 I know it's statistics, but when you look at it, it seems like it understands, right? 00:08:28.960 --> 00:08:31.660 It seems like it understands my question. 00:08:31.660 --> 00:08:37.600 And I think one of the really interesting parts is the fact that these LLMs have a context. 00:08:37.600 --> 00:08:39.740 I would like you to write a program in Python. 00:08:39.740 --> 00:08:40.860 Okay, great. 00:08:40.860 --> 00:08:41.740 Tell it to me. 00:08:41.740 --> 00:08:46.360 I want a program that does X, Y, Z, and then it writes it in Python, which sounds so simple. 00:08:46.360 --> 00:08:53.440 But up until then, things like Siri and all the other voice assistants, they seemed so 00:08:53.440 --> 00:08:55.800 disjointed and so not understanding. 00:08:55.800 --> 00:08:57.780 You're like, I just asked you about the weather. 00:08:57.780 --> 00:09:01.840 And when I ask you how hot it is, you know, like, how do you not understand that that applies 00:09:01.840 --> 00:09:02.700 to the weather, right? 00:09:02.700 --> 00:09:08.500 There's just in the fact that you converse with them over a series of interactions is pretty 00:09:08.500 --> 00:09:08.860 special. 00:09:08.860 --> 00:09:09.800 The context. 00:09:09.800 --> 00:09:10.320 Yeah. 00:09:10.320 --> 00:09:15.200 It's basically implemented just by passing the history, the whole history appended to your 00:09:15.200 --> 00:09:15.580 prompt. 00:09:16.140 --> 00:09:16.880 So, yeah. 00:09:16.880 --> 00:09:23.380 It's not like any super staple magic or whatever, but it's still very interesting, like, how 00:09:23.380 --> 00:09:24.580 far you can take it. 00:09:24.580 --> 00:09:32.060 And yeah, definitely, like, the context allows you to interact with the AI a lot more conversationally 00:09:32.060 --> 00:09:33.400 and humanly. 00:09:33.400 --> 00:09:36.380 Like, you don't have to pretend like you're talking to an AI. 00:09:36.380 --> 00:09:41.200 You can actually just kind of treat it like a human and it still answers questions very well. 00:09:41.780 --> 00:09:46.540 There was even at Google, there was that engineer who said they thought it had become sentient. 00:09:46.540 --> 00:09:48.880 And there was that whole drama around that, right? 00:09:49.020 --> 00:09:50.720 This is such a crazy coincidence. 00:09:50.720 --> 00:09:54.560 But my roommate is actually friends with that gentleman. 00:09:54.560 --> 00:09:55.420 Oh, really? 00:09:55.420 --> 00:09:55.860 I know. 00:09:55.860 --> 00:09:56.180 Wow. 00:09:56.180 --> 00:09:58.040 Absolutely crazy coincidence. 00:09:58.040 --> 00:09:59.280 Like, it's just fun. 00:09:59.280 --> 00:10:01.240 I just thought it was really funny you bringing it up. 00:10:01.240 --> 00:10:05.580 It was that, it was a Cajun gentleman, a senior engineer at Google, right? 00:10:05.580 --> 00:10:05.860 Yeah. 00:10:05.860 --> 00:10:06.260 Yeah. 00:10:06.540 --> 00:10:07.400 Very funny, Mike. 00:10:07.400 --> 00:10:08.020 It's been a little while. 00:10:08.020 --> 00:10:12.380 I don't remember all the details, but yeah, I mean, it's pretty wild and pretty powerful. 00:10:12.380 --> 00:10:16.680 I think I've recently read, I'm trying to quick look it up, but I didn't find it. 00:10:16.680 --> 00:10:21.020 I think they just used LLMs to discover, like, a new protein folding. 00:10:21.020 --> 00:10:26.380 And it's that kind of stuff that makes me think, like, okay, how interesting that it's, you know, 00:10:26.380 --> 00:10:29.400 that knowledge wasn't out there in the world, necessarily. 00:10:29.400 --> 00:10:31.460 I have a lot to say on that subject. 00:10:31.680 --> 00:10:35.880 So personally, I don't believe that language models are actually intelligent. 00:10:35.880 --> 00:10:41.220 I think that people are conflating, well, they're certainly not conscious, right? 00:10:41.220 --> 00:10:41.660 Absolutely. 00:10:41.660 --> 00:10:41.940 Yeah. 00:10:41.940 --> 00:10:44.320 As to whether they're intelligent, I don't think they are. 00:10:44.320 --> 00:10:50.220 I think that intelligence has a, like, intelligence as we know it, as humans know it, has some 00:10:50.220 --> 00:10:53.420 different characteristics that language models don't really exhibit. 00:10:53.420 --> 00:10:57.660 They're best thought of as, like, really, really, really good statistical models. 00:10:57.660 --> 00:11:00.380 Like, are you familiar with the mirror test? 00:11:00.380 --> 00:11:01.660 Maybe, but I don't think so. 00:11:01.660 --> 00:11:01.860 Yeah. 00:11:01.860 --> 00:11:07.000 So it's, like, this idea in animal psychology, but, like, if a cat sees a mirror, it thinks 00:11:07.000 --> 00:11:10.280 it's another cat because it doesn't recognize its own inflection. 00:11:10.280 --> 00:11:10.740 Right. 00:11:10.740 --> 00:11:15.320 You see them all get, like, big and, like, trying to, like, act big and tough to chase it off, 00:11:15.320 --> 00:11:16.120 and it's just them, yeah? 00:11:16.120 --> 00:11:19.260 Language models are kind of like that, but for humans, right? 00:11:19.260 --> 00:11:24.800 Like, if something mimics human-like qualia closely enough, very tempting to think of it 00:11:24.800 --> 00:11:25.240 as human. 00:11:25.240 --> 00:11:25.600 Yeah. 00:11:25.600 --> 00:11:29.520 We see faces on Mars when it's really just erosion, stuff like that. 00:11:29.760 --> 00:11:30.160 Exactly. 00:11:30.160 --> 00:11:34.500 I could talk about this, like, for a full hour, but yeah, we should totally move on to 00:11:34.500 --> 00:11:36.680 another topic before I go on a tirade. 00:11:36.680 --> 00:11:42.360 Well, when you're saying it's not that intelligent, I'm wondering if maybe you've misnamed this project. 00:11:42.360 --> 00:11:45.120 Maybe it should just be Jupyter A, like the I? 00:11:45.120 --> 00:11:46.220 Do you got to drop the I? 00:11:46.220 --> 00:11:46.840 I don't know. 00:11:47.040 --> 00:11:48.680 We're just following convention, right? 00:11:48.680 --> 00:11:50.080 So I still use the term AI. 00:11:50.080 --> 00:11:51.640 Of course. 00:11:51.640 --> 00:11:52.540 You got to talk to people. 00:11:52.540 --> 00:11:52.880 Yeah. 00:11:52.880 --> 00:11:54.220 Emphasize the artificial, huh? 00:11:54.220 --> 00:11:57.360 Before we get to Jupyter AI, what came before? 00:11:57.360 --> 00:12:03.820 How did people work with things like ChatGPT and other LLMs in Jupyter before stuff like Jupyter AI came along? 00:12:03.820 --> 00:12:06.320 I think initially it was a combination. 00:12:06.320 --> 00:12:16.100 So the initial motivation for this project came through a combination of sort of a demo put together by Fernando Perez, who is another, I believe. 00:12:16.100 --> 00:12:17.440 I think he's another co-founder. 00:12:17.640 --> 00:12:27.220 another co-founder of Project Jupyter, and he put together this demo called Jupyter, which is, it's like spelled like Jupyter, except the last letter is an E. 00:12:27.220 --> 00:12:30.580 And it's a pun of like ChatGPT, right? 00:12:30.580 --> 00:12:31.360 Jupyter. 00:12:31.360 --> 00:12:46.280 There was a combination of that demo project set by Fernando and some motivation from my manager, Brian, who also was, you know, as a leader in the AWS AI organization. 00:12:46.980 --> 00:12:51.660 You know, he's always trying to think of fancy, fancy new ideas, right? 00:12:51.660 --> 00:12:54.500 And this is a pretty fun idea to work out. 00:12:54.500 --> 00:13:00.080 So I put together, I think this was sometime in early January, I put together the first demo. 00:13:00.080 --> 00:13:01.060 It was private. 00:13:01.060 --> 00:13:05.640 And I showed it off to the team and they were like, wow, this has a lot of potential. 00:13:05.640 --> 00:13:07.560 Let's see if we can grow it a bit more. 00:13:07.560 --> 00:13:15.420 And then as we worked on it for the next few months, it became clear like, oh, wow, this is actually really significant. 00:13:15.420 --> 00:13:16.860 Let's keep working on this. 00:13:16.860 --> 00:13:23.460 So it's definitely been a collaborative effort to bring Jupyter AI to where it is today. 00:13:23.460 --> 00:13:24.120 Sounds like it. 00:13:24.120 --> 00:13:26.940 Definitely a lot of contributors over on the GitHub listing. 00:13:26.940 --> 00:13:27.920 Let's get into it. 00:13:27.920 --> 00:13:28.740 What is Jupyter AI? 00:13:28.740 --> 00:13:34.080 People can guess, but it's also different in ways than maybe just plug it in a chat window. 00:13:34.080 --> 00:13:37.480 Jupyter AI is actually, right now, it's two packages. 00:13:38.020 --> 00:13:44.360 But it's best thought of as just a set of packages that bring generative AI to Project Jupyter as a whole. 00:13:44.360 --> 00:13:48.400 So not just Jupyter Lab, but also Jupyter Notebook and IPython. 00:13:48.400 --> 00:13:49.220 Even the shell. 00:13:49.220 --> 00:13:53.620 How do you, I guess you invoke it by doing like the magic there as well. 00:13:53.620 --> 00:13:58.800 It's the IPython shell, which is not the same as like a bash shell for instance, your terminal. 00:13:58.800 --> 00:13:59.320 Interesting. 00:13:59.320 --> 00:14:03.260 You could dive into a little bit more detail on what these two packages are. 00:14:03.260 --> 00:14:09.180 So we have the base Jupyter AI package, which is spelled exactly as you might imagine it. 00:14:09.180 --> 00:14:11.180 It's Jupyter hyphen AI. 00:14:11.340 --> 00:14:19.600 That is a Jupyter Lab extension that brings a UI to Jupyter Lab, which is the screenshot that you're showing on your screen. 00:14:19.600 --> 00:14:30.260 But for viewers without a screen, it basically is the package that adds that chat panel to the left-hand side and allows you to speak conversationally with an AI. 00:14:30.260 --> 00:14:38.280 And then the second package is Jupyter AI magics, which is spelled the same, except at the end, it's spelled hyphen magics. 00:14:38.280 --> 00:14:48.420 And that is actually the base library that implements some of the AI providers we use and brings things called magic commands to the IPython shell. 00:14:48.420 --> 00:14:56.760 And magic commands basically let you invoke the library code, aka like calling link, using it to like call language models, for instance. 00:14:56.760 --> 00:15:00.520 And that allows you to do it inside an IPython context. 00:15:00.520 --> 00:15:09.520 So what's crazy is that if you run IPython in your terminal shell, you can actually run Jupyter AI from your terminal, which is pretty cool. 00:15:09.520 --> 00:15:10.940 Yeah, I didn't realize that. 00:15:10.940 --> 00:15:14.220 I mean, it makes sense, of course, but I hadn't really thought about it. 00:15:14.220 --> 00:15:18.900 Yeah, I thought more of this like kind of a GUI UI type of thing that was alongside what you were doing. 00:15:18.900 --> 00:15:19.200 Yeah. 00:15:19.200 --> 00:15:20.540 We try to make it flexible. 00:15:20.540 --> 00:15:24.900 And there's reasons for the magic commands, which I can talk about that later, though. 00:15:24.960 --> 00:15:34.680 This portion of Talk Python To Me is brought to you by Posit, the makers of Shiny, formerly RStudio, and especially Shiny for Python. 00:15:34.680 --> 00:15:36.580 Let me ask you a question. 00:15:36.580 --> 00:15:38.280 Are you building awesome things? 00:15:38.280 --> 00:15:39.340 Of course you are. 00:15:39.340 --> 00:15:40.900 You're a developer or a data scientist. 00:15:40.900 --> 00:15:41.820 That's what we do. 00:15:41.820 --> 00:15:43.860 And you should check out Posit Connect. 00:15:44.500 --> 00:15:50.840 Posit Connect is a way for you to publish, share, and deploy all the data products that you're building using Python. 00:15:50.840 --> 00:15:54.020 People ask me the same question all the time. 00:15:54.020 --> 00:15:57.180 Michael, I have some cool data science project or notebook that I built. 00:15:57.180 --> 00:16:00.460 How do I share it with my users, stakeholders, teammates? 00:16:00.460 --> 00:16:05.280 Do I need to learn FastAPI or Flask or maybe Vue or React.js? 00:16:05.280 --> 00:16:06.500 Hold on now. 00:16:06.500 --> 00:16:09.420 Those are cool technologies, and I'm sure you'd benefit from them. 00:16:09.420 --> 00:16:11.300 But maybe stay focused on the data project? 00:16:11.300 --> 00:16:13.760 Let Posit Connect handle that side of things? 00:16:14.120 --> 00:16:18.500 With Posit Connect, you can rapidly and securely deploy the things you build in Python. 00:16:18.500 --> 00:16:24.980 Streamlit, Dash, Shiny, Bokeh, FastAPI, Flask, Quattro, Reports, Dashboards, and APIs. 00:16:24.980 --> 00:16:27.260 Posit Connect supports all of them. 00:16:27.260 --> 00:16:33.100 And Posit Connect comes with all the bells and whistles to satisfy IT and other enterprise requirements. 00:16:33.100 --> 00:16:37.480 Make deployment the easiest step in your workflow with Posit Connect. 00:16:37.480 --> 00:16:42.640 For a limited time, you can try Posit Connect for free for three months by going to talkpython.fm. 00:16:43.740 --> 00:16:47.260 That's talkpython.fm/P-O-S-I-T. 00:16:47.260 --> 00:16:49.140 The link is in your podcast player show notes. 00:16:49.140 --> 00:16:52.380 Thank you to the team at Posit for supporting Talk Python. 00:16:54.620 --> 00:17:08.980 Now, one thing, you said this, but I want to emphasize it a little bit in that this will run anywhere IPython kernel runs, which is JupyterLab notebook, but also Google Colab, VS Code, other places as well, right? 00:17:09.120 --> 00:17:13.160 So pretty much it comes to you wherever your Jupyter type of stuff is, right? 00:17:13.160 --> 00:17:15.380 Yeah, and the same goes for your lab extensions. 00:17:15.380 --> 00:17:23.560 So the great thing about lab extensions is that they work anywhere where the product is just sort of built on top of JupyterLab, right? 00:17:23.800 --> 00:17:31.960 So Google Colab is essentially what Google has is a, well, I can't obvious, I obviously can't attest to what they're actually doing. 00:17:31.960 --> 00:17:37.360 But most likely, it's like a set of extensions or CSS themes that are built on top of JupyterLab. 00:17:37.360 --> 00:17:39.960 But like the underlying code is still JupyterLab. 00:17:39.960 --> 00:17:41.640 It's still mostly JupyterLab. 00:17:41.860 --> 00:17:49.360 So you can actually just install extensions and they work just fine, which is another reason why JupyterLab is just pretty awesome. 00:17:49.360 --> 00:17:51.000 Yeah, it sure is. 00:17:51.000 --> 00:17:57.040 Yeah, JupyterLab itself is basically a pre-selected, pre-configured set of extensions, right? 00:17:57.040 --> 00:17:57.780 That's pretty cool. 00:17:57.780 --> 00:17:58.680 That is true, yeah. 00:17:58.680 --> 00:18:04.300 Giving preference to or showing just what I play with mostly, which is ChatGPT. 00:18:04.300 --> 00:18:07.760 There's actually a lot of language models that you can work with, right? 00:18:07.760 --> 00:18:18.880 One of the big things, and this is something I'll circle back to later, is that Jupyter AI is meant to be model agnostic, meaning that we don't discriminate against the choice of model or model provider. 00:18:18.880 --> 00:18:24.280 Because as an open source project, it's very imperative that we maintain the trust of our users, right? 00:18:24.280 --> 00:18:35.800 Users have to be sure that this isn't just some product that exists to force them and force or pigeonhole them into using a certain model provider like OpenAI or Anthropic. 00:18:36.120 --> 00:18:38.480 It's the product makes no opinions. 00:18:38.480 --> 00:18:41.660 We simply try to support everything as best as we can. 00:18:41.660 --> 00:18:49.940 And we've written a lot of code to make sure that all of these models and just play nicely together, essentially. 00:18:49.940 --> 00:18:59.980 Like every model provider, like let's say from Anthropic or AI21 or Cohere, every one of these APIs kind of has its own quirks. 00:19:00.120 --> 00:19:02.700 Every one of these Python SDKs has its own quirks. 00:19:02.700 --> 00:19:09.400 And we work very hard to basically iron out the surface and make everything have the same interface. 00:19:09.400 --> 00:19:11.220 We can talk about that later, though. 00:19:11.220 --> 00:19:11.480 Sure. 00:19:11.480 --> 00:19:13.900 And it also makes it pretty easy to try them out, right? 00:19:13.900 --> 00:19:15.960 If you switch from one to the other. 00:19:15.960 --> 00:19:21.040 You're like, I wonder how Hugging Face versus OpenAI would do to solve this problem, right? 00:19:21.180 --> 00:19:21.700 Absolutely. 00:19:21.700 --> 00:19:31.240 Like that's kind of one of the ideas is like while, you know, certain model providers, they might offer a UI if they're very well funded by their investors. 00:19:32.060 --> 00:19:34.740 For example, OpenAI to have a UI. 00:19:34.740 --> 00:19:49.280 But that UI only allows you to compare between different models from OpenAI, which, you know, like as an independent third party looking to use an AI service, like that information is obviously a little biased, right? 00:19:49.280 --> 00:19:52.260 Like you want to see like what other providers have to offer. 00:19:52.260 --> 00:19:53.920 Like what does AI21 have? 00:19:53.920 --> 00:19:55.080 What does Anthropic have? 00:19:55.080 --> 00:20:04.420 And right now it's really, there really is no like cross model provider like UI or like interface in general. 00:20:04.420 --> 00:20:09.540 But like that's kind of one of the use cases that Jupyter AI was intended to fit. 00:20:09.540 --> 00:20:09.780 Yeah. 00:20:09.780 --> 00:20:13.320 Provides a standard way to interact with all these and sort of compare them. 00:20:13.320 --> 00:20:14.680 And it's also a UI for them. 00:20:14.680 --> 00:20:16.060 And if you're using JupyterLab, yeah. 00:20:16.060 --> 00:20:20.340 I'm not familiar with all of these different models and companies. 00:20:20.340 --> 00:20:27.120 Do any of those run locally, like things like GPT for all, where it's a local model versus some kind of cloud? 00:20:27.120 --> 00:20:28.020 Where's your key? 00:20:28.020 --> 00:20:30.360 Where's your billing details and all that? 00:20:30.360 --> 00:20:35.680 We actually recently just merged a PR that adds a GPT for all support. 00:20:35.680 --> 00:20:36.140 Okay. 00:20:36.140 --> 00:20:37.680 That's included in the release. 00:20:37.680 --> 00:20:44.800 However, back when we first implemented this a few months ago, I had a few issues with the platform compatibility. 00:20:44.900 --> 00:20:53.300 So like it wouldn't, so like some of the binaries that we downloaded from GP for all didn't seem to work well on my M1 Mac, for instance. 00:20:53.300 --> 00:20:58.380 I'd say, yes, we do have local model support, but it's a bit experimental right now. 00:20:58.380 --> 00:21:03.560 We're still like ironing out the edges and testing, like seeing how we can make the experience better. 00:21:03.560 --> 00:21:08.620 Like does it sometimes only get bad output because we forgot to install the shared library? 00:21:08.620 --> 00:21:12.540 Those are the type of questions that our team is wrangling with right now. 00:21:12.540 --> 00:21:17.900 I see. And maybe I just jumped right into it, but maybe tell people what GPT for all is just real quickly. 00:21:17.900 --> 00:21:20.920 GPT for all offers a few local models. 00:21:20.920 --> 00:21:23.780 Actually, they offer several, I believe, not just a few. 00:21:23.780 --> 00:21:26.240 I think they offer maybe like 10 to 15. 00:21:26.240 --> 00:21:28.020 It's the numbers getting quite large. 00:21:28.020 --> 00:21:28.540 Yeah. 00:21:28.540 --> 00:21:30.920 To the point where I don't know what the right choice is. 00:21:30.920 --> 00:21:32.140 I'm like, which one do I download? 00:21:32.140 --> 00:21:33.760 There's they say they're all good. 00:21:33.760 --> 00:21:34.820 Of course, they're going to say they're good. 00:21:34.820 --> 00:21:35.520 I'll be frank. 00:21:35.540 --> 00:21:38.640 I don't actually have that much experience with GPT for all. 00:21:38.640 --> 00:21:44.460 It's we mainly use them as sort of a provider for like these free and open source language models. 00:21:44.460 --> 00:21:44.740 Yeah. 00:21:44.740 --> 00:21:48.180 I think they offer UI as well for multiple platforms. 00:21:48.180 --> 00:21:50.820 I've only played with them a little bit, just started checking it out. 00:21:50.820 --> 00:21:52.840 But it's basically a local. 00:21:52.840 --> 00:21:54.180 You download the model. 00:21:54.180 --> 00:21:54.940 You run it locally. 00:21:54.940 --> 00:21:58.280 You don't pay anything because it's just just running on your machine. 00:21:58.280 --> 00:22:12.240 As opposed to, say, OpenAI and others where you've at least got rate limiting and a certain amount of queries before you have to pay and maybe potentially access to better models like ChatGPT-4 versus 3.5 and so on. 00:22:12.240 --> 00:22:18.600 That's also taking the characteristics of these service providers for granted, right? 00:22:18.600 --> 00:22:26.040 So, yes, definitely, while it does hurt the wallet to pay for usage credits, right? 00:22:26.040 --> 00:22:31.120 It's also pretty remarkable how small the latency has gotten with some of these APIs. 00:22:31.120 --> 00:22:35.560 I've gotten sub-500 millisecond latency on some of these APIs. 00:22:35.560 --> 00:22:44.920 And that's really incredible because when I was using GPT-4-All, the latency was a little bit high on running locally with limited compute resources. 00:22:44.920 --> 00:22:48.340 It's really remarkable how fast these APIs are. 00:22:48.340 --> 00:22:49.160 It is pretty insane. 00:22:49.160 --> 00:22:51.660 Sometimes it drives me crazy. 00:22:51.660 --> 00:22:56.140 I'm just only, again, referring to ChatGPT because I don't have the experience with the others to the degree. 00:22:56.140 --> 00:23:04.120 But it drives me crazy how it artificially limits the response based on the speed of the response so it looks like it's chatting with you. 00:23:04.120 --> 00:23:08.100 I'm like, no, I have four pages of stuff because you just get it out. 00:23:08.100 --> 00:23:14.360 You know, it'll say something like, you can ask, I gave you a five-page program. 00:23:14.360 --> 00:23:15.500 Let's call it X. 00:23:15.500 --> 00:23:17.260 If you say, what is X? 00:23:17.260 --> 00:23:19.680 It'll just start printing it slowly, line by line. 00:23:19.680 --> 00:23:21.940 Like, you know you just are echoing it back. 00:23:21.940 --> 00:23:22.740 Just get it out. 00:23:22.740 --> 00:23:24.440 I want to ask you the next question. 00:23:24.440 --> 00:23:25.340 You know what I mean? 00:23:25.340 --> 00:23:30.800 In that case, that's actually a feature request that we've gotten because it doesn't actually slow down. 00:23:31.080 --> 00:23:33.120 Like, it's not just like a pointless animation. 00:23:33.120 --> 00:23:37.240 Yeah, the servers are streaming essentially token by token, right? 00:23:37.240 --> 00:23:39.100 As the language model generates output. 00:23:39.100 --> 00:23:44.260 So it's kind of more like a progress indicator than a superfluous animation. 00:23:44.260 --> 00:23:44.620 Yeah. 00:23:44.760 --> 00:23:45.340 Yeah, of course. 00:23:45.340 --> 00:23:50.780 But if you've got something large, large blocks of text you're working with, it can be a drag. 00:23:50.780 --> 00:23:51.020 All right. 00:23:51.020 --> 00:23:56.120 I wanted to kind of touch on some of the different features that I pulled out that I thought were cool. 00:23:56.560 --> 00:24:01.820 I mean, obviously it goes without saying that Jupyter AI is on GitHub. 00:24:01.820 --> 00:24:03.680 I mean, because it's software. 00:24:03.680 --> 00:24:09.620 And so it's open source, which I don't know if we said that, but obviously free open source on GitHub. 00:24:09.620 --> 00:24:11.440 BSD3 license. 00:24:11.980 --> 00:24:19.640 But it's also noteworthy that it's officially under the JupyterLab organization, not under the David account. 00:24:19.640 --> 00:24:20.340 You know what I mean? 00:24:20.340 --> 00:24:23.600 It's officially part of the JupyterLab subproject. 00:24:23.600 --> 00:24:27.720 And yep, as you pointed out, we're under the JupyterLab GitHub org as well. 00:24:27.720 --> 00:24:28.660 Yeah, that's awesome. 00:24:28.660 --> 00:24:32.280 Let's talk about some of the different things you can do with it. 00:24:32.280 --> 00:24:36.980 Some of them will be straightforward, like just like, how do I write a function, Jupyter AI? 00:24:36.980 --> 00:24:39.940 And others, I think, are going to be a little more interesting. 00:24:40.160 --> 00:24:43.180 So let's start with asking something about your notebook. 00:24:43.180 --> 00:24:45.460 Tell us what people can do here. 00:24:45.460 --> 00:24:51.900 Asking about your notebook basically means like you can actually teach Jupyter AI about certain files, right? 00:24:51.900 --> 00:24:55.720 So the way you do this is via a slash command in the chat UI. 00:24:55.720 --> 00:24:58.780 So you type slash learn and then file path. 00:24:58.780 --> 00:25:03.660 And it essentially teaches the Jupyter AI about that file. 00:25:03.660 --> 00:25:08.060 Now, it works best with files that are written in natural language, right? 00:25:08.160 --> 00:25:11.560 So like text files or markup or markdown rather. 00:25:11.560 --> 00:25:12.040 Yeah. 00:25:12.040 --> 00:25:16.660 So like those, like, especially like developer documentation as well, right? 00:25:16.660 --> 00:25:18.660 It works really well with those. 00:25:18.660 --> 00:25:20.480 It works best with those kinds of files. 00:25:20.480 --> 00:25:27.640 And after Jupyter AI learns about these files, you can then ask questions about the files. 00:25:27.640 --> 00:25:32.440 It's learned by prefixing your question with the slash ask command. 00:25:32.440 --> 00:25:33.360 That is so cool. 00:25:33.360 --> 00:25:34.240 It is pretty cool. 00:25:34.320 --> 00:25:41.960 I know it's so cool because what I've done a lot of times, if I want ChatGPT to help me, it's like, I'm like, all right, well, let me copy some code. 00:25:41.960 --> 00:25:42.240 Right. 00:25:42.240 --> 00:25:44.320 Then I'm going to have a conversation about it. 00:25:44.320 --> 00:25:48.080 But a lot of the context of, well, it's actually referencing this other function. 00:25:48.080 --> 00:25:49.160 And what does that do? 00:25:49.160 --> 00:25:55.080 Or just a broader understanding of what am I actually working on is missing, right? 00:25:55.080 --> 00:25:56.840 Because I've only copied it. 00:25:56.840 --> 00:26:01.040 You can't paste, you know, 20 files into ChatGPT and start talking about it. 00:26:01.100 --> 00:26:02.520 But with this, you can, right? 00:26:02.520 --> 00:26:06.200 You can say, learn about, you can say, learn about different things, right? 00:26:06.200 --> 00:26:13.420 You can say, learn about your notebook, but you can also probably tell it like, learn about my documentation or learn about my data set. 00:26:13.420 --> 00:26:14.980 And now let me talk to you about it. 00:26:14.980 --> 00:26:23.080 What's interesting is that the right now, while it works best for natural language documents, we are working on improving the experience for code. 00:26:23.260 --> 00:26:36.840 From our testing, like the code is mostly the capabilities of Jupyter AI after learning code is right now mostly limited to explaining what code does, but sort of like explains it from the doc screen. 00:26:36.840 --> 00:26:43.820 So we're working on a way to format the code in a manner that is more interpretable to a language model. 00:26:43.820 --> 00:26:44.100 Sure. 00:26:44.100 --> 00:26:47.100 We're working on ways to improve the experience for code. 00:26:47.100 --> 00:27:01.040 But yeah, definitely the long-term vision is to have Jupyter AI literally be able to learn from the whole directory of files or possibly even a URL, like a remote URL to like, like a documentation page. 00:27:01.040 --> 00:27:01.400 Yeah. 00:27:01.400 --> 00:27:02.660 We have some big ideas there. 00:27:02.660 --> 00:27:03.900 We're still working on them. 00:27:03.900 --> 00:27:06.300 I want to work with a new package XYZ. 00:27:06.300 --> 00:27:07.960 Like, I don't know what XYZ is. 00:27:07.960 --> 00:27:08.680 You know what? 00:27:08.680 --> 00:27:10.280 Here's where you can learn about it. 00:27:10.280 --> 00:27:11.500 Go and figure it out. 00:27:11.500 --> 00:27:18.240 Like, query the PyPI API, get the docs page from the metadata, and then go to that URL, scrape it. 00:27:18.240 --> 00:27:20.140 Like, lots of things we're exploring. 00:27:20.140 --> 00:27:22.000 It's still kind of early days for this, right? 00:27:22.000 --> 00:27:23.600 You've been at it about a year or so? 00:27:23.600 --> 00:27:25.000 It's been out for a while. 00:27:25.000 --> 00:27:29.480 Recently, I've had to work on a few other things as well, like Jupyter AI. 00:27:29.480 --> 00:27:33.220 Unfortunately, I cannot give my entire life to Jupyter AI. 00:27:33.220 --> 00:27:36.740 So I've been working on a few other things these past few months. 00:27:36.740 --> 00:27:41.220 But yeah, there are a lot of things that I envision for Jupyter AI. 00:27:41.220 --> 00:27:46.880 I have a much bigger vision for what I want this project to be and what it can be capable of. 00:27:46.880 --> 00:27:47.380 Exciting. 00:27:47.380 --> 00:27:52.100 So this screenshot that you got here in the section that I'll link to in the show notes 00:27:52.100 --> 00:27:56.880 is cool because you can select a piece of a portion, not even a whole cell, 00:27:56.880 --> 00:27:59.040 but a portion of code in a cell. 00:27:59.040 --> 00:28:01.600 And then you can ask, what does this code do? 00:28:01.600 --> 00:28:05.480 We have an integration with the JupyterLab editor APIs. 00:28:05.480 --> 00:28:10.360 So you can select a block of code and then include that in your prompt. 00:28:10.360 --> 00:28:13.160 And it will be appended to your prompt below, right? 00:28:13.160 --> 00:28:13.440 Right. 00:28:13.440 --> 00:28:14.540 It's appended to Bob, sorry. 00:28:14.540 --> 00:28:17.980 You can basically ask, like, so you can select a block of code. 00:28:17.980 --> 00:28:23.180 So in this screenshot right here, there's this block of code that computes the least common 00:28:23.180 --> 00:28:25.460 multiple of two integers, right? 00:28:25.460 --> 00:28:30.460 And you can select that and then click Include Selection and then ask Jupyter AI, 00:28:30.460 --> 00:28:31.440 what does this do? 00:28:31.440 --> 00:28:32.880 Which is pretty awesome. 00:28:33.020 --> 00:28:35.080 Another checkbox is Replace Selection. 00:28:35.080 --> 00:28:40.260 I'm guessing that is like, help me rewrite this code to be more efficient or if there's 00:28:40.260 --> 00:28:41.160 any bugs, fix it. 00:28:41.160 --> 00:28:44.000 So the Replace Selection checkbox is totally independent. 00:28:44.000 --> 00:28:47.120 So you can actually use both at the same time. 00:28:47.360 --> 00:28:50.080 And one of the use cases for this is refactoring. 00:28:50.080 --> 00:28:55.580 And I've actually applied this in practice a few times where you can basically select a 00:28:55.580 --> 00:28:59.820 block of code and then click both Include and Replace Selection. 00:28:59.820 --> 00:29:03.860 And then you can format your prompt to say, refactor this block of code. 00:29:03.860 --> 00:29:06.300 Do not include any additional help or text. 00:29:06.740 --> 00:29:11.700 And when you send that prompt over, it will actually refactor the code for you in your 00:29:11.700 --> 00:29:15.480 notebook, which is, yeah, like is, it's pretty great. 00:29:15.480 --> 00:29:16.480 That's pretty awesome. 00:29:16.480 --> 00:29:20.600 You know, you could do things like refactor this to use guarding clauses. 00:29:20.600 --> 00:29:24.740 So it's less nested and less, less arrow code or whatever, right? 00:29:24.740 --> 00:29:25.060 Yeah. 00:29:25.060 --> 00:29:27.220 Or like add a doc string, right? 00:29:27.220 --> 00:29:32.400 Summarize this purpose of this function and then enclose that in the doc string and add it 00:29:32.400 --> 00:29:33.000 to the function. 00:29:33.000 --> 00:29:33.320 Right. 00:29:33.380 --> 00:29:36.100 Or this code is pandas code, but I'd like to use Polars. 00:29:36.100 --> 00:29:39.960 Please rewrite it for Polars, which is not a super compatible API. 00:29:39.960 --> 00:29:42.400 It's not like Dask to pandas, where it's basically the same. 00:29:42.400 --> 00:29:42.620 Yeah. 00:29:42.620 --> 00:29:45.940 And this kind of circles back to that question that you had asked earlier. 00:29:45.940 --> 00:29:48.520 I think I went on a tangent there and didn't fully answer. 00:29:48.520 --> 00:29:53.100 But like, what is like the utility of Jupyter AI to like data practitioners, right? 00:29:53.100 --> 00:29:57.660 So we're talking data scientists, machine learning engineers, like this, the Include Selection 00:29:57.660 --> 00:29:58.160 features. 00:29:58.160 --> 00:30:03.280 We've heard great feedback about how helpful it is to like actually explain a data. 00:30:03.280 --> 00:30:09.160 So sometimes like you're working with a test set and it's not immediately clear what the 00:30:09.160 --> 00:30:13.160 features of this test set are or like what this even does, because sometimes it's like 00:30:13.160 --> 00:30:17.900 high dimensional data and they can literally select it and then click Include Selection 00:30:17.900 --> 00:30:21.500 and say, and tell Jupyter AI, explain to me what this does. 00:30:21.500 --> 00:30:24.020 Just like, what is this like data frame stuff? 00:30:24.020 --> 00:30:25.980 Like, whoa, we got data frames in data frames. 00:30:25.980 --> 00:30:27.300 Like what's going on here? 00:30:27.300 --> 00:30:28.540 Like what even is the structure? 00:30:28.760 --> 00:30:29.200 That's awesome. 00:30:29.200 --> 00:30:30.300 And I think it's super valuable. 00:30:30.300 --> 00:30:33.980 I think the, and this is like a little bit I was getting to before one of the features 00:30:33.980 --> 00:30:34.940 that I think is cool. 00:30:34.940 --> 00:30:40.080 Whereas if you just go with straight ChatGPT, you copy your code, you paste it into the chat. 00:30:40.080 --> 00:30:43.380 Hopefully it doesn't say it's too much text and then you can talk about it. 00:30:43.380 --> 00:30:46.980 But then when you get an answer, you've got to grab it, move it back over. 00:30:46.980 --> 00:30:50.400 And this just, this fluid back and forth is really nice. 00:30:50.640 --> 00:30:50.720 Yeah. 00:30:50.720 --> 00:30:56.300 And that's actually one of the design principles that we worked out when first starting this 00:30:56.300 --> 00:31:00.420 project officially is the idea that Jupyter AI should be human centered. 00:31:00.420 --> 00:31:05.600 As in, you shouldn't be expected to be a developer to know how to use this tool. 00:31:05.600 --> 00:31:10.840 Like this tool is for humans, not, not for any specific persona, just for humans in general. 00:31:10.840 --> 00:31:11.340 That's awesome. 00:31:11.340 --> 00:31:11.780 Yeah. 00:31:11.780 --> 00:31:16.640 So in this case, you select the function that does the lowest common denominator bit and 00:31:16.640 --> 00:31:21.040 you ask it what it does, it says the code will print out the least common multiple of 00:31:21.040 --> 00:31:22.580 two numbers passed to it. 00:31:22.580 --> 00:31:24.620 Super simple, very concise. 00:31:24.620 --> 00:31:25.800 Okay, great. 00:31:25.800 --> 00:31:27.660 Now we can go on to the next thing, right? 00:31:27.660 --> 00:31:28.060 Yeah. 00:31:28.060 --> 00:31:32.480 There's this LCD function that we're kind of talking about here. 00:31:32.480 --> 00:31:39.580 This example is recursive, which I think recursion is pretty insane, right? 00:31:39.580 --> 00:31:42.220 As just a concept for people to get their head around. 00:31:42.220 --> 00:31:43.480 This is the iterative version. 00:31:43.480 --> 00:31:46.160 So this is after they, yeah, this is the iterative. 00:31:46.220 --> 00:31:46.700 Oh, this is after. 00:31:46.700 --> 00:31:46.900 Yeah. 00:31:46.900 --> 00:31:48.700 So if we go back up. 00:31:48.700 --> 00:31:52.840 Oh, one of the things you ask it is things like an example is rewrite this function to 00:31:52.840 --> 00:31:54.460 be iterative, not recursive. 00:31:54.460 --> 00:31:54.680 Right. 00:31:54.680 --> 00:31:56.500 Which is actually really, really awesome. 00:31:56.500 --> 00:31:57.140 Right. 00:31:57.140 --> 00:31:58.880 You're like, this is breaking my brain. 00:31:58.880 --> 00:32:01.120 Let's, let's see if we can not do that anymore. 00:32:01.120 --> 00:32:06.740 This portion of talk Python to me is brought to you by us. 00:32:06.740 --> 00:32:11.000 Have you heard that Python is not good for concurrent programming problems? 00:32:11.200 --> 00:32:16.580 Whoever told you that is living in the past because it's prime time for Python's asynchronous features. 00:32:16.580 --> 00:32:22.420 With the widespread adoption of async methods and the async and await keywords, Python's ecosystem 00:32:22.420 --> 00:32:26.260 has a ton of new and exciting frameworks based on async and await. 00:32:26.360 --> 00:32:31.400 That's why we created a course for anyone who wants to learn all of Python's async capabilities, 00:32:31.400 --> 00:32:33.820 async techniques and examples in Python. 00:32:33.820 --> 00:32:39.700 Just visit talkpython.fm/async and watch the intro video to see if this course is for you. 00:32:39.700 --> 00:32:42.180 It's only $49 and you own it forever. 00:32:42.180 --> 00:32:43.220 No subscriptions. 00:32:43.220 --> 00:32:45.980 And there are discounts for teams as well. 00:32:48.520 --> 00:32:54.940 Another thing I wanted to talk about, and you talked a fair amount about this in your presentations that you did. 00:32:54.940 --> 00:32:59.420 I can't remember if it was the JupyterCon or the PyData one that I saw, but one of those two. 00:32:59.420 --> 00:33:06.360 You talked about generating new notebooks and how it's, it's actually quite a tricky process. 00:33:06.360 --> 00:33:11.760 You got to break it down into little steps because if you ask too much from the AI, it kind of doesn't give you a lot of great answers. 00:33:12.080 --> 00:33:13.560 Tell us about making new notebooks. 00:33:13.560 --> 00:33:18.640 Like why would I even use, like I can go to JupyterLab and say file new, it'll make that for me, right? 00:33:18.640 --> 00:33:19.400 So what's this about? 00:33:19.400 --> 00:33:25.140 The generate capability is great because it generates a file that is essentially a tutorial, 00:33:25.140 --> 00:33:28.320 that can be used as a tutorial to teach you about new subjects, right? 00:33:28.320 --> 00:33:34.900 So like you could, for example, submit a prompt, like slash generate a notebook about asyncio 00:33:34.900 --> 00:33:38.100 or a demonstration of how to use Matplotlib. 00:33:38.520 --> 00:33:45.020 And after, so this will take a bit of time, but eventually Jupyter AI is done thinking and generates a file 00:33:45.020 --> 00:33:48.480 and it names a file up and generates a notebook. 00:33:48.480 --> 00:33:53.300 And the notebook has a name, it has a title, it has like sections, table of contents, 00:33:53.300 --> 00:33:59.900 and each of the cells within it, like is tied to some like topic that is determined to be helpful 00:33:59.900 --> 00:34:02.980 and to answer the user's question. 00:34:02.980 --> 00:34:03.380 Awesome. 00:34:03.560 --> 00:34:10.540 Could I do something like, I have weather data in this format from the US weather service. 00:34:10.540 --> 00:34:15.780 Could you generate me a notebook to plot this XYZ and help me answer these questions? 00:34:15.780 --> 00:34:17.560 Or like, could I ask it something like that? 00:34:17.560 --> 00:34:18.780 Not at the moment. 00:34:18.780 --> 00:34:20.860 So like that would best be done. 00:34:20.860 --> 00:34:26.240 That would be done best if, since the data is already, like, I'm assuming that the data is already 00:34:26.240 --> 00:34:27.500 available, right? 00:34:27.560 --> 00:34:29.140 In some kind of like format. 00:34:29.140 --> 00:34:34.720 So like in a notebook, you could use a chat UI to like select that entire, select that selection 00:34:34.720 --> 00:34:39.760 and then tell it, tell Jupyter AI to generate code to plot that data set. 00:34:40.120 --> 00:34:45.860 So right now, generate only takes a natural language prompt as its only argument. 00:34:45.860 --> 00:34:46.120 I see. 00:34:46.120 --> 00:34:48.380 So it's kind of like stateless in that regard. 00:34:48.380 --> 00:34:53.180 So in this case, you can say slash generate a demonstration of how to use matplotlib. 00:34:53.180 --> 00:34:54.660 And then the response is great. 00:34:54.660 --> 00:34:56.100 I'll start working on your notebook. 00:34:56.100 --> 00:34:58.480 It'll take a few minutes, but I'll reply when it's ready. 00:34:58.480 --> 00:35:00.460 In the meantime, let's keep talking. 00:35:00.460 --> 00:35:05.180 So what happens behind the scenes that takes a few minutes here? 00:35:05.180 --> 00:35:06.300 This is a bit interesting. 00:35:06.300 --> 00:35:12.320 It does kind of dive deep into like the technical details, which I'm not sure if, which do you 00:35:12.320 --> 00:35:13.200 want to just like dive? 00:35:13.200 --> 00:35:14.560 Yeah, let's go tell us how it works. 00:35:14.560 --> 00:35:16.400 Probably a good chance to explore that. 00:35:16.400 --> 00:35:16.600 Yeah. 00:35:16.600 --> 00:35:16.900 Yeah. 00:35:16.900 --> 00:35:22.380 So slash generate, the first thing is that the prompt is first expanded into essentially 00:35:22.380 --> 00:35:23.700 a table of contents. 00:35:23.700 --> 00:35:29.600 So basically we tell the language model, generate us a table of contents conforming to this JSON 00:35:29.600 --> 00:35:30.100 schema. 00:35:30.100 --> 00:35:36.120 And when you pass a JSON schema included in your prompt, the language model will be much 00:35:36.120 --> 00:35:42.960 more, will have a much higher likelihood of returning exclusively a JSON object that matches 00:35:42.960 --> 00:35:45.860 that JSON schema that you had initially provided. 00:35:45.860 --> 00:35:49.140 So in our case, we generate a table of contents. 00:35:49.140 --> 00:35:51.080 And then we take that table of contents. 00:35:51.080 --> 00:35:56.800 And then we say for each section, we do this in parallel, like generate us some code cells 00:35:56.800 --> 00:36:00.820 that are appropriate for teaching this section of the document. 00:36:00.820 --> 00:36:03.320 So for example, for matplotlib, right? 00:36:03.320 --> 00:36:08.860 Like maybe the first section is your first, like generating your first plot, plotting 3D functions. 00:36:08.860 --> 00:36:13.120 And the next one is like plotting complex functions with phase or something like that. 00:36:13.120 --> 00:36:18.380 And then with each of these sections, we then send another prompt template to the language 00:36:18.380 --> 00:36:21.360 model for each of these sections, asking it to generate the code. 00:36:21.360 --> 00:36:26.160 And then at the end, we join it all together and then we save it to disk and then emit that 00:36:26.160 --> 00:36:27.100 message and say, we're done. 00:36:27.100 --> 00:36:33.460 Maybe the English literature equivalent would be instead of just saying, write me a story about 00:36:33.460 --> 00:36:35.880 a person who goes on an adventure and gets lost. 00:36:35.880 --> 00:36:41.320 It's like, I want, give me an outline, bullet points of interesting things that would make 00:36:41.320 --> 00:36:44.000 up a story of how somebody goes on an adventure and gets lost. 00:36:44.000 --> 00:36:47.120 And then for each one of those, you're like, now tell this part of the story. 00:36:47.120 --> 00:36:47.860 Now tell this part. 00:36:47.860 --> 00:36:51.420 And somehow that makes it more focused and accurate, right? 00:36:51.420 --> 00:36:56.940 The main limitation is that because we're a model agnostic, language models are limited 00:36:56.940 --> 00:36:59.240 in how much output they can generate, right? 00:36:59.240 --> 00:36:59.460 Yeah. 00:36:59.460 --> 00:37:04.060 The issue we were running into when we were trying to do the whole thing all at once, like generate 00:37:04.060 --> 00:37:07.220 mean the whole notebook is that some language models just couldn't do it. 00:37:07.220 --> 00:37:12.800 In an effort to, you know, sort of stay model agnostic, we deliberately implemented, we deliberately 00:37:12.800 --> 00:37:18.820 broke this process down into like smaller subtasks, each with its own like prompt template in order 00:37:18.820 --> 00:37:26.020 to accommodate these models that may lack the same token size windows that other models have. 00:37:26.020 --> 00:37:32.720 I think just even for ones that have large token spaces, I think they still, the more specific 00:37:32.720 --> 00:37:38.360 you can be, the more likely you're going to get a focused result instead of a wandering vague result. 00:37:38.360 --> 00:37:44.120 Teach me about math or teach me how to factor, you know, how to integrate this differential, 00:37:44.120 --> 00:37:47.200 solve this series of differential equations or physics. 00:37:47.200 --> 00:37:50.220 Like you're going to get a really different answer to those two questions. 00:37:50.380 --> 00:37:51.380 Yeah. 00:37:51.380 --> 00:37:56.060 That's also circles back to like a topic I didn't, I did want to call out, but I don't, I don't think 00:37:56.060 --> 00:37:56.780 we hit on it. 00:37:56.780 --> 00:38:03.420 Is that the chat UI actually does support rendering in both Markdown and LaTeX, which is a markup 00:38:03.420 --> 00:38:04.200 language for math. 00:38:04.200 --> 00:38:10.800 So you can ask it both complex engineering and mathematical questions, like asking it to explain 00:38:10.800 --> 00:38:11.880 you like, yeah. 00:38:11.880 --> 00:38:13.720 So like there might be a demo here. 00:38:13.720 --> 00:38:16.080 I'm not sure if it's on this page though. 00:38:16.080 --> 00:38:22.120 So if I had a Fourier, fast Fourier transform in LaTeX and I put it in there and say, what 00:38:22.120 --> 00:38:22.500 is this? 00:38:22.500 --> 00:38:25.340 It'll say it's a fast Fourier transform or something like that. 00:38:25.340 --> 00:38:25.660 Yes. 00:38:25.660 --> 00:38:27.620 And it also works the other way around. 00:38:27.620 --> 00:38:33.940 You can also use it to say like, Hey, explain to me what the 2D Laplace equation is, or explain 00:38:33.940 --> 00:38:35.540 to me like, what does this do? 00:38:35.540 --> 00:38:35.860 Right. 00:38:35.940 --> 00:38:42.640 And it will actually generate and format the equation inside the chat UI, which is really 00:38:42.640 --> 00:38:43.260 remarkable. 00:38:43.260 --> 00:38:44.460 I love that feature. 00:38:44.460 --> 00:38:45.540 It's actually really awesome. 00:38:45.540 --> 00:38:50.700 And it's also really appropriate for a scientific oriented thing like Jupyter, right? 00:38:50.700 --> 00:38:56.320 The remarkable thing is that because chat UBT and like other such language models, like the 00:38:56.320 --> 00:39:02.140 ones from Anthropic and AI21, because like they are founded on the premise of where like their 00:39:02.140 --> 00:39:05.000 functionality comes from having such a large corpus of data. 00:39:05.000 --> 00:39:07.680 They know a remarkable amount of information. 00:39:07.680 --> 00:39:13.460 So like we've tried like some example notebooks of quantum computing and explained those really 00:39:13.460 --> 00:39:13.900 well. 00:39:13.900 --> 00:39:19.740 We try, I tried one of like the Black Scholes options pricing model used in financial engineering. 00:39:19.740 --> 00:39:21.320 It's really remarkable. 00:39:21.320 --> 00:39:25.660 Like the utility that it offers just by being there in the side panel. 00:39:25.660 --> 00:39:30.400 Like you essentially have like a math wizard available to you and to your lab all the time. 00:39:30.700 --> 00:39:35.740 It's probably better than a lot of math professors in terms of not necessarily in the depth of 00:39:35.740 --> 00:39:36.240 one area. 00:39:36.240 --> 00:39:42.400 But, you know, if you ask somebody who does like abstract algebra about real analysis, they're 00:39:42.400 --> 00:39:43.840 like, I don't really do that part. 00:39:43.840 --> 00:39:46.880 Or if you ask somebody about real analysis about number theory, like I don't really, you can 00:39:46.880 --> 00:39:50.580 hit on all the areas, at least a generalist professor sort of thing. 00:39:50.580 --> 00:39:53.140 We talked about the slash learn command. 00:39:53.340 --> 00:39:56.160 That's pretty excellent already and where that's going. 00:39:56.160 --> 00:39:58.420 So I'm pretty excited about that. 00:39:58.420 --> 00:39:58.660 Yeah. 00:39:58.660 --> 00:40:02.220 It actually does have a lot of interesting technical tidbits to it. 00:40:02.220 --> 00:40:03.840 Like the implementation. 00:40:03.840 --> 00:40:04.520 Okay. 00:40:04.520 --> 00:40:04.900 Yeah. 00:40:04.900 --> 00:40:09.360 Actually, this is one of the really challenging things with these chat bots and things. 00:40:09.360 --> 00:40:14.520 For example, I've tried to ask ChatGPT if I gave it one, just one of the transcripts from 00:40:14.520 --> 00:40:15.100 the show. 00:40:15.100 --> 00:40:16.980 I want to have a conversation about it. 00:40:16.980 --> 00:40:17.780 It says, ah, that's too much. 00:40:17.780 --> 00:40:18.440 I can't do it. 00:40:18.440 --> 00:40:21.540 You know, it's just, it's just one, one show. 00:40:21.540 --> 00:40:25.380 And in doc, like in your documentation, there might be a lot of files in there, right? 00:40:25.380 --> 00:40:28.180 More than just one transcript levels worth. 00:40:28.180 --> 00:40:32.620 So that alone, I think it's kind of interesting just how to ingest that much data into it. 00:40:32.740 --> 00:40:32.900 Yeah. 00:40:32.900 --> 00:40:38.180 And you know, this is a very interesting subject and it actually is a bit complex. 00:40:38.180 --> 00:40:39.060 I'm sure it is. 00:40:39.060 --> 00:40:41.400 I think there are some other features you want to discuss. 00:40:41.400 --> 00:40:44.020 Let's dive into this for just a minute because I think it is interesting. 00:40:44.020 --> 00:40:46.240 How do you make, because this makes it yours, right? 00:40:46.240 --> 00:40:50.580 It's one thing to ask vaguely, like, tell me about the Laplace equation and how does it 00:40:50.580 --> 00:40:51.840 apply to heat transfer? 00:40:51.840 --> 00:40:52.580 Like, okay, great. 00:40:52.580 --> 00:40:56.380 I have a specific problem with a specific library and I want to solve it. 00:40:56.380 --> 00:40:58.820 And you don't seem to understand about enough of it. 00:40:58.860 --> 00:41:03.060 So it really limits the usefulness if it doesn't, if it's not a little closer to what you're 00:41:03.060 --> 00:41:03.520 actually doing. 00:41:03.520 --> 00:41:04.820 And I think this brings it closer. 00:41:04.820 --> 00:41:06.140 So yeah, tell us about it. 00:41:06.140 --> 00:41:11.600 Language models aren't just governed by like their intelligence, however you measure that, 00:41:11.600 --> 00:41:11.860 right? 00:41:11.860 --> 00:41:14.580 They're also governed by how much context they can take. 00:41:14.580 --> 00:41:19.680 So one of the reasons ChatGPT was so remarkable is that it had a great way of managing context 00:41:19.680 --> 00:41:20.980 through conversation history. 00:41:20.980 --> 00:41:30.600 And that seemingly small leap and seemingly small feature is what made ChatGPT so remarkably 00:41:30.600 --> 00:41:34.860 disruptive to this industry is because of that additional context. 00:41:34.860 --> 00:41:37.040 And we think about extending that idea. 00:41:37.040 --> 00:41:41.840 How do we give an AI more context, make it even more human-like and personal? 00:41:41.840 --> 00:41:44.160 Well, the idea is similar. 00:41:44.160 --> 00:41:47.160 We add more context and that's what learning does, right? 00:41:47.160 --> 00:41:52.500 And so the way learning works is that we're actually using another set of models called 00:41:52.500 --> 00:41:53.500 embedding models. 00:41:53.500 --> 00:42:00.500 And embedding models are very, very underrated in the AI-like modeling space, right? 00:42:00.500 --> 00:42:02.700 These are really remarkable things. 00:42:02.700 --> 00:42:07.540 And they have this one, I'll only cover like the most important characteristic of embedding 00:42:07.540 --> 00:42:15.340 models, which is embedding models take syntax and map it to a high-dimensional vector space 00:42:15.340 --> 00:42:16.840 called a semantic space. 00:42:16.840 --> 00:42:23.060 And inside of the semantic space, nearby vectors indicate semantic similarity. 00:42:23.060 --> 00:42:26.300 I know that's like a lot of words, so I'm going to break that idea down, right? 00:42:26.300 --> 00:42:30.380 So like canine and dog, let's take these two words as an example, right? 00:42:30.380 --> 00:42:32.340 These two words are completely different. 00:42:32.340 --> 00:42:36.680 They don't even share a single character and similarity together, right? 00:42:36.680 --> 00:42:38.880 They don't have a single letter in common with one another. 00:42:39.420 --> 00:42:44.940 And yet we know as humans that these two dogs, these two words mean the same thing. 00:42:44.940 --> 00:42:46.400 They refer to a dog. 00:42:46.400 --> 00:42:51.620 So like they have different syntax, but the same semantics, the same semantic meaning. 00:42:51.620 --> 00:42:52.860 So their vectors would be... 00:42:52.860 --> 00:42:53.840 Would be mapped close. 00:42:53.840 --> 00:42:56.220 Would be very close by whatever metric you're using, yeah. 00:42:56.220 --> 00:43:01.500 If you extend this idea and like you imagine, okay, what if you split a document? 00:43:01.660 --> 00:43:05.020 What if you split a file into like one to two sentence chunks? 00:43:05.020 --> 00:43:08.960 And then for each of these like sentences, let's just say sentences, for example. 00:43:08.960 --> 00:43:14.140 Let's say we split a document to sentences and then we take each of those sentences and then map, 00:43:14.140 --> 00:43:19.060 use an embedding model to compute their embedding and then store them inside of a vector store. 00:43:19.060 --> 00:43:26.020 Like basically like a local database that has, that just stores all of these vectors in like a file or something, right? 00:43:26.020 --> 00:43:31.740 Now imagine what happens if we then take a prompt, like a question that we might have, 00:43:31.740 --> 00:43:33.440 encode that as an embedding. 00:43:33.440 --> 00:43:37.820 And then we say to the vector store, okay, for this prompt embedding, 00:43:37.820 --> 00:43:41.760 find me all of the other embeddings are close to this. 00:43:41.760 --> 00:43:46.220 Well, what you've just done in this process is called a semantic search. 00:43:46.220 --> 00:43:53.580 So it's kind of like syntax search, except instead of searching based off of key words or tokens or other syntactic traits, 00:43:53.580 --> 00:43:59.020 you are searching based off the actual natural language meaning of the word. 00:43:59.020 --> 00:44:04.740 This is much more applicable when it comes to like natural language prompts and natural language, 00:44:04.740 --> 00:44:09.960 like corpuses of data, because this is like the actual information that's being stored. 00:44:09.960 --> 00:44:11.440 Not, we don't care about the characters. 00:44:11.440 --> 00:44:14.180 We care about the information that they represent, right? 00:44:14.180 --> 00:44:15.100 The essence of it, yeah. 00:44:15.100 --> 00:44:18.600 And these vectors are computed by the larger language model? 00:44:18.600 --> 00:44:22.240 The vectors, the embeddings are computed by an embedding model. 00:44:22.240 --> 00:44:27.520 And they're actually a separate category of model that we have our own special APIs for. 00:44:27.520 --> 00:44:31.340 So in our settings panel, you can change the language model. 00:44:31.340 --> 00:44:33.240 And I think we already discussed that, right? 00:44:33.240 --> 00:44:33.540 Yeah. 00:44:33.620 --> 00:44:38.240 But what's interesting is that underneath that, you'll also see a section that says embedding model. 00:44:38.240 --> 00:44:45.940 And you can change the embedding model to like, we also offer that same principle of model agnosticism there. 00:44:45.940 --> 00:44:46.180 Yeah. 00:44:46.180 --> 00:44:47.200 This is very interesting. 00:44:47.200 --> 00:44:47.920 Very interesting. 00:44:48.100 --> 00:44:49.580 Let's talk a little bit about the format. 00:44:49.580 --> 00:44:54.880 You said, obviously, that you can do LaTeX, which you say in, you say math, right? 00:44:54.880 --> 00:44:56.220 You tell us, give me math. 00:44:56.220 --> 00:44:56.440 Yeah. 00:44:56.440 --> 00:44:56.920 Give me math. 00:44:56.920 --> 00:44:58.280 Which is, yeah, it's pretty interesting. 00:44:58.280 --> 00:45:02.780 But you can do images, markdown, code, HTML, JSON, text. 00:45:02.780 --> 00:45:05.820 There's a lot of, a lot of different formats you can get the answer back in. 00:45:05.900 --> 00:45:13.360 When you use the AI magics, we can pass them to like a renderer first before we show the output to the user. 00:45:13.360 --> 00:45:13.600 Yeah. 00:45:13.600 --> 00:45:20.980 And with the AI magic, the %AI in the cell, you can also specify, that's where you put the format potentially, 00:45:20.980 --> 00:45:25.700 but you can also specify the model and the service, I guess, for the provider. 00:45:25.700 --> 00:45:32.560 The IPython magics are basically stateless in the sense that you always have to specify the model explicitly. 00:45:32.960 --> 00:45:36.340 They don't operate off the premise that you are using JupyterLab. 00:45:36.340 --> 00:45:42.060 They don't run off the premise that you have JupyterLab installed or are using the lab extension that we offer. 00:45:42.060 --> 00:45:46.620 Because of that, like the model is stated explicitly every time. 00:45:46.620 --> 00:45:47.460 That's by design. 00:45:47.460 --> 00:45:53.820 When you sit down, what is your JupyterAI provider set to? 00:45:53.820 --> 00:45:54.820 What's your favorite? 00:45:54.820 --> 00:46:00.880 As a developer, I like literally pick a random one to give the most test coverage at all times. 00:46:01.280 --> 00:46:03.180 And that's actually a great way of finding bugs. 00:46:03.180 --> 00:46:05.360 So, yeah, I don't have a favorite one. 00:46:05.360 --> 00:46:07.800 My favorite one is the one that works. 00:46:07.800 --> 00:46:09.680 And hopefully that should be all of them. 00:46:09.680 --> 00:46:09.980 So. 00:46:09.980 --> 00:46:14.340 I guess you can also tell it to forget what I was talking about. 00:46:14.340 --> 00:46:15.280 We're going to start over. 00:46:15.280 --> 00:46:20.140 That's pretty interesting that you can do that along the way because you maybe had a bunch of conversations. 00:46:20.140 --> 00:46:24.440 And we talked about the benefit of it, like knowing the history of that conversation. 00:46:24.440 --> 00:46:25.960 But you're like, all right, new idea. 00:46:25.960 --> 00:46:27.040 Switching topics. 00:46:27.040 --> 00:46:27.700 Chat. 00:46:27.700 --> 00:46:32.360 I think the last one I wanted to talk about specifically was interpolating prompts. 00:46:32.840 --> 00:46:43.420 Kind of almost like f-strings where you can put in the prompt text, you can put a variable and then other parts of your notebook or program can set that value. 00:46:43.420 --> 00:46:43.880 Yeah. 00:46:43.880 --> 00:46:44.480 Tell us about this. 00:46:44.660 --> 00:46:48.360 You can define a variable in your IPython kernel, right? 00:46:48.360 --> 00:46:52.340 So, like, and that's just dumb, but just like how you define any other variable. 00:46:52.340 --> 00:46:57.780 But what's interesting is that IPython is actually aware of the variables that you are defining. 00:46:57.780 --> 00:47:02.720 So, we can programmatically access that when we implement the magic, right? 00:47:02.720 --> 00:47:09.360 Basically, if you define any variable at the top level scope, like let's say poet equals Walt Whitman, right? 00:47:09.360 --> 00:47:10.920 So, we have a name variable called poet. 00:47:10.920 --> 00:47:18.760 And then you can send a prompt over, like, write a poem in the style of curly braces poet and curly braces. 00:47:18.760 --> 00:47:27.320 And when that is run, the value of that variable is interpolated and substitutes around the curly braces. 00:47:27.320 --> 00:47:31.140 So, the final prompt becomes write a poem in the style of Walt Whitman. 00:47:31.140 --> 00:47:35.920 And when that prompt is sent, well, you can imagine it generates a poem of Walt Whitman. 00:47:35.920 --> 00:47:42.700 The variable interpolation that we offer in IPython is very useful for, like, very quick, like, debugging. 00:47:42.700 --> 00:47:47.140 So, you can actually reference a cell in the notebook directly. 00:47:47.140 --> 00:47:54.900 I think a lot of people don't know this, but, like, in a Jupyter notebook, there's, like, the in and out indicators to the left of each cell. 00:47:54.900 --> 00:47:57.900 So, it'll say, like, in dot one, out dot one, right? 00:47:57.900 --> 00:47:59.760 So, those are actual variables. 00:47:59.760 --> 00:48:01.760 And you can use them here, too. 00:48:01.760 --> 00:48:03.600 So, you can reference, like, debug. 00:48:03.600 --> 00:48:05.440 Tell me why this code is failing. 00:48:05.440 --> 00:48:07.520 Curly braces in dot one. 00:48:07.520 --> 00:48:11.000 That's, you just, on the screen there, just scroll there. 00:48:11.000 --> 00:48:19.100 Explain what this does, the in bracket 11, or what went wrong in out bracket 11, or, yeah, something like this. 00:48:19.100 --> 00:48:19.440 Right. 00:48:19.440 --> 00:48:22.080 It's fine long as you don't go and rerun that cell. 00:48:22.080 --> 00:48:22.440 Yeah. 00:48:22.580 --> 00:48:25.420 But, like you said, this is not for long term. 00:48:25.420 --> 00:48:35.080 You can make it independent of the order of execution just by, like, assigning whatever variable that is to, whatever, like, content that is to a named variable. 00:48:35.080 --> 00:48:38.160 That way, no matter what order you run them, like. 00:48:38.160 --> 00:48:46.620 People might be thinking, when you describe this interpolation thing, just bracket, boom, bracket, curly brace, curly brace, you're like, we already have that in Python. 00:48:46.620 --> 00:48:48.900 You just put an F in front of the string, and so on. 00:48:48.900 --> 00:48:55.000 But this is in the message that goes to the AI cell magic, not straight Python, right? 00:48:55.000 --> 00:48:55.960 That's the relevance. 00:48:55.960 --> 00:48:57.560 That's why this is interesting, right? 00:48:57.560 --> 00:48:57.980 Yes. 00:48:57.980 --> 00:49:00.820 I guess it really comes down to the different models that you select. 00:49:00.820 --> 00:49:05.120 So, you opt into this a little bit, or maybe you need to understand it that way. 00:49:05.120 --> 00:49:06.780 But talk to us a bit about privacy. 00:49:07.140 --> 00:49:10.580 If I select something and say, what does this do? 00:49:10.580 --> 00:49:11.460 What happens? 00:49:11.460 --> 00:49:21.060 Something important to emphasize here is that whenever you use a language model that's hosted by a third party, so, like, it's running on their servers, right? 00:49:21.060 --> 00:49:24.200 Regardless of whether this model is free or not. 00:49:24.200 --> 00:49:32.920 Like, the fact that you're sending data to a third party over the internet, like, that's where the privacy and security concerns happen, right? 00:49:33.060 --> 00:49:37.140 And that happens, like, whenever, like, you're sending data across the wire over the internet. 00:49:37.140 --> 00:49:48.860 But we have some special safeguards in place here, specifically to assuage fears of, like, concerns over privacy and security that a lot of open source users have. 00:49:49.220 --> 00:49:54.700 And one of the important ideas here is that Jupyter AI is both transparent and traceable. 00:49:54.700 --> 00:50:02.000 So, when we send a prompt to a language model, that's always captured in the server logs by default. 00:50:02.000 --> 00:50:03.480 So, that's always being logged. 00:50:03.480 --> 00:50:04.600 That's always being captured. 00:50:04.600 --> 00:50:05.560 Yeah. 00:50:05.560 --> 00:50:07.500 So, like, it's always going to be traceable. 00:50:07.720 --> 00:50:09.160 There's no secret back channel. 00:50:09.160 --> 00:50:11.800 Like, you tell them, people, this is happening, okay? 00:50:11.800 --> 00:50:12.180 Yeah. 00:50:12.180 --> 00:50:20.300 So, if, like, an operator needs to audit, like, oh, dang, like, let me check just to make sure nothing scary was sent over to OpenAI. 00:50:20.300 --> 00:50:30.440 Well, the operator can review the server logs and make sure that all usage is compliant with whatever privacy policy their company has, right? 00:50:30.440 --> 00:50:42.680 And Jupyter AI is also exclusively user-driven, meaning that we will never, by default, send data to a language model, even if you selected one, right? 00:50:43.100 --> 00:50:48.960 Like, we will never send data to that language model until explicit action is done by the user. 00:50:48.960 --> 00:50:53.520 So, in this case, like, clicking the send button, clicking shift, enter, and running to sell. 00:50:53.520 --> 00:50:53.740 Yeah. 00:50:53.740 --> 00:50:57.320 Nothing is sent to language model or embedding model until that happens. 00:50:57.320 --> 00:50:58.540 That's really all you can do. 00:50:58.540 --> 00:50:59.760 That sounds great. 00:50:59.760 --> 00:51:04.140 Because you don't control what happens once it hits OpenAI or Anthropic or whatever, right? 00:51:04.140 --> 00:51:06.760 That's why the transparency is so important, right? 00:51:06.760 --> 00:51:09.160 And, oh, I forgot to touch on traceability. 00:51:09.340 --> 00:51:21.620 So, like, with these AI-generated cells, right, so, like, the output cells, in the metadata, we indicate that with, like, the model that was used to generate an output cell, if it comes from the Jupyter AI magic. 00:51:21.620 --> 00:51:28.480 So, that way, it's also, like, traceable not just in the logs, but, like, in the actual files metadata and stuff as well. 00:51:28.480 --> 00:51:35.700 That's cool, because it'd be really easy to say, have the cell magic, and then, you know, how notebooks store their last output. 00:51:35.700 --> 00:51:40.520 Like, if you upload them to GitHub, they'll keep their last output, unless you say clear all cell outputs. 00:51:40.520 --> 00:51:46.480 Depending on which model you have selected, you might not get the same output, not even close to the same output. 00:51:46.480 --> 00:51:49.020 So, you might want to know, like, well, how did you get that picture? 00:51:49.020 --> 00:51:52.120 Oh, I had Anthropic selected and not the other, right? 00:51:52.120 --> 00:51:53.440 Like, that is really nice. 00:51:53.440 --> 00:51:58.240 I've actually used the server logs myself to debug, like, prompt templates, for instance, right? 00:51:58.240 --> 00:52:02.680 So, like, because what we show in the logs is the full prompt. 00:52:02.680 --> 00:52:07.420 Like, after applying our template, after applying, like, the edits, like, that's what's actually shown. 00:52:07.420 --> 00:52:12.120 So, that's really also really helpful for developers who need to debug what's happening. 00:52:12.120 --> 00:52:12.800 Yeah, of course. 00:52:12.800 --> 00:52:20.980 Or if you're a scientist and you're looking for reproducibility, I doubt there's much guaranteed reproducibility even across the versions of the same model. 00:52:20.980 --> 00:52:25.140 But at least you were in the same ballpark, you know, at least I know what model it came from. 00:52:25.140 --> 00:52:29.720 You can set the temperature to zero, but it won't generate very fun output. 00:52:29.720 --> 00:52:33.460 That is a workaround if you truly need a reproducibility. 00:52:33.460 --> 00:52:34.560 I suppose, yeah. 00:52:34.560 --> 00:52:41.760 The temperature being the ability to tell it how creative do you want to be or how focused do you want the answer to be, right? 00:52:41.760 --> 00:52:49.180 It's a hyperparameter that basically governs the randomness, like, how far away from the mean it's willing to deviate. 00:52:49.180 --> 00:52:52.740 Kind of, yeah, vaguely describable as creativity, yeah. 00:52:52.740 --> 00:52:53.460 Yeah, I suppose. 00:52:53.460 --> 00:52:58.120 I guess if you're looking for privacy, the GPT for all might be a good option. 00:52:58.120 --> 00:52:59.080 Oh, yeah, absolutely. 00:52:59.080 --> 00:52:59.840 For that, right? 00:52:59.840 --> 00:53:01.340 Because that's not going anywhere. 00:53:01.520 --> 00:53:01.740 Yeah. 00:53:01.740 --> 00:53:04.680 However, some of them do have license restrictions. 00:53:04.680 --> 00:53:14.120 And that's also why we have also taken it this low when it comes to adding more GPT for all support is because different models are licensed differently. 00:53:14.120 --> 00:53:17.140 And that's another consideration we have to take in mind. 00:53:17.140 --> 00:53:17.360 Yeah. 00:53:17.360 --> 00:53:18.080 Yeah, of course. 00:53:18.080 --> 00:53:22.880 You're playing in a crazy space with many different companies evolving licenses. 00:53:22.880 --> 00:53:23.300 Yeah. 00:53:23.700 --> 00:53:25.820 Let's close it out with one more thing here. 00:53:25.820 --> 00:53:26.760 Maybe two more things. 00:53:26.760 --> 00:53:30.600 One is there's a lot of interest in these agents. 00:53:30.600 --> 00:53:38.280 And it sounds, for example, your create a notebook that does this sort of thing, like teach me about Matplotlib, is a little bit agent driven. 00:53:38.280 --> 00:53:43.840 Thinking of like LangChain and stuff like that, or even GPT engineer. 00:53:43.840 --> 00:53:45.040 What's the story? 00:53:45.040 --> 00:53:46.600 Do you have any integrations with that? 00:53:46.600 --> 00:53:47.700 Any of those types of things? 00:53:47.700 --> 00:53:52.480 We're working on one where basically you won't need to use slash commands anymore. 00:53:52.720 --> 00:53:58.040 So this is like, again, like we're just kind of playing around with this, seeing how well it behaves. 00:53:58.040 --> 00:54:02.580 But we are trying to use agents to, for example, remove the need to use slash commands. 00:54:02.580 --> 00:54:07.220 So when you say, like generate a notebook, like it will just generate one. 00:54:07.220 --> 00:54:09.000 You don't have to note a slash command for that. 00:54:09.000 --> 00:54:10.200 Like it will just know. 00:54:10.200 --> 00:54:11.380 So like, yeah. 00:54:11.380 --> 00:54:15.900 However, we don't have any, we don't use agents at the present moment now. 00:54:15.900 --> 00:54:23.020 And the reason for that is that they have very, they're not very model agnostic, at least the ones from our research. 00:54:23.020 --> 00:54:27.860 Like they only work well for a specific model and a specific prompt template. 00:54:27.860 --> 00:54:29.480 But beyond that, it's hard. 00:54:29.480 --> 00:54:29.760 All right. 00:54:29.760 --> 00:54:30.400 Last question. 00:54:30.400 --> 00:54:31.720 Running out of time here. 00:54:31.720 --> 00:54:32.060 Yeah. 00:54:32.060 --> 00:54:33.920 What's your favorite thing you've done with Jupyter AI? 00:54:33.920 --> 00:54:38.620 Not like a feature, but what have you made it do to help you that you really like? 00:54:38.620 --> 00:54:40.720 Teach Jupyter AI about Jupyter AI. 00:54:40.720 --> 00:54:42.040 That's an easy one. 00:54:42.320 --> 00:54:44.060 So we have the documentation. 00:54:44.060 --> 00:54:44.760 What did it learn? 00:54:44.760 --> 00:54:46.820 Well, learn about itself. 00:54:46.820 --> 00:55:00.700 So like I was at a conference and I, so this was at PyData and I actually got enough questions to the point where I did, I just downloaded, I had the documentation already available in my home directory because of some other previous. 00:55:00.700 --> 00:55:01.980 Or restructured text them. 00:55:01.980 --> 00:55:02.260 Yeah. 00:55:02.260 --> 00:55:03.560 And the markdown source. 00:55:03.560 --> 00:55:03.920 Okay. 00:55:03.920 --> 00:55:04.700 The markdown source. 00:55:04.700 --> 00:55:05.040 Right. 00:55:05.040 --> 00:55:07.080 And then I just had slash learn. 00:55:07.080 --> 00:55:08.460 I just learned that doc. 00:55:08.580 --> 00:55:14.420 And then I just had that laptop on the side and told people, like, if you have any questions, try answering that. 00:55:14.420 --> 00:55:15.400 Yeah. 00:55:15.400 --> 00:55:17.400 In case you don't want to wait in line. 00:55:17.400 --> 00:55:21.440 So yeah, that was, yeah, it's pretty remarkable. 00:55:21.440 --> 00:55:21.840 Yeah. 00:55:21.840 --> 00:55:24.840 The learn feature by far is definitely my favorite. 00:55:24.840 --> 00:55:28.780 And it's the one I want to spend the most time developing and pushing it better. 00:55:28.780 --> 00:55:36.120 I think there's massive possibility there because to deeply understand what you're working on in a multifaceted way. 00:55:36.120 --> 00:55:37.180 What are the documentations? 00:55:37.180 --> 00:55:37.900 What is the code? 00:55:37.900 --> 00:55:38.680 What is the data? 00:55:38.680 --> 00:55:41.720 What is the network topology and servers I can work with? 00:55:41.720 --> 00:55:44.640 Like all of that kind of stuff or pick your specialty. 00:55:44.640 --> 00:55:45.140 Yep. 00:55:45.140 --> 00:55:47.120 And that's how we make AI more human. 00:55:47.120 --> 00:55:48.160 Right before it takes over. 00:55:48.160 --> 00:55:49.280 So no problem there. 00:55:49.280 --> 00:55:49.820 Oh, yeah. 00:55:49.820 --> 00:55:50.100 Just kidding. 00:55:50.100 --> 00:55:51.520 Just kidding. 00:55:51.520 --> 00:55:51.820 Yeah. 00:55:51.880 --> 00:55:52.940 Let's wrap it up. 00:55:52.940 --> 00:55:53.980 I think we're out of time, David. 00:55:53.980 --> 00:55:55.460 So final question. 00:55:55.460 --> 00:56:00.700 Some notable PyPI package, maybe something awesome you discovered to help you write Jupyter AI. 00:56:00.700 --> 00:56:02.480 Anything you want to give a shout out to? 00:56:02.480 --> 00:56:04.520 We definitely use LangChain a lot. 00:56:04.520 --> 00:56:07.780 And it has some pretty fantastic integrations. 00:56:07.780 --> 00:56:10.740 And we're actually built on top of LangChain, really. 00:56:10.740 --> 00:56:12.060 But also Dask. 00:56:12.060 --> 00:56:13.260 Dask is really nice. 00:56:13.260 --> 00:56:13.580 Yeah. 00:56:13.580 --> 00:56:18.000 Dask has some great visualization capabilities for when you're doing parallel compute. 00:56:18.560 --> 00:56:22.280 It has a great dashboard that's also available in Jupyter Lab. 00:56:22.280 --> 00:56:22.640 Yeah. 00:56:22.640 --> 00:56:26.000 That shows it running and distributing the work in Jupyter Lab. 00:56:26.000 --> 00:56:26.580 It's amazing. 00:56:26.580 --> 00:56:29.320 And one of the contributors also reached out once. 00:56:29.320 --> 00:56:33.320 He had heard that I was integrating Dask into Jupyter AI. 00:56:33.320 --> 00:56:38.800 He actually reached out to help us and offer direct one-on-one guidance with using Dask. 00:56:38.800 --> 00:56:42.200 And yeah, it's just been a fantastic experience using Dask. 00:56:42.200 --> 00:56:43.020 Yeah. 00:56:43.020 --> 00:56:44.100 I have no complaints. 00:56:44.100 --> 00:56:50.740 It's just pretty awesome that somebody has finally made parallel and distributed compute better in Python. 00:56:50.740 --> 00:56:52.020 For sure. 00:56:52.020 --> 00:56:52.360 Yeah. 00:56:52.360 --> 00:56:53.160 Yeah, Dask is cool. 00:56:53.160 --> 00:56:53.980 Dask is very cool. 00:56:53.980 --> 00:56:54.860 All right. 00:56:54.860 --> 00:56:55.860 Well, final call to action. 00:56:55.860 --> 00:56:57.280 People want to get started with Jupyter AI. 00:56:57.280 --> 00:56:57.880 What do you tell them? 00:56:57.880 --> 00:56:59.860 I can install Jupyter AI or Konda. 00:56:59.860 --> 00:57:01.280 Konda install works too. 00:57:01.280 --> 00:57:02.000 It's on both. 00:57:02.000 --> 00:57:02.380 Awesome. 00:57:02.380 --> 00:57:03.620 Well, really good work. 00:57:03.620 --> 00:57:04.960 This is super interesting. 00:57:04.960 --> 00:57:07.100 And I think a lot of people are going to find value in it. 00:57:07.100 --> 00:57:11.320 So I can see some nice comments in the audience that people are excited as well. 00:57:11.320 --> 00:57:12.560 So thanks for being here. 00:57:12.560 --> 00:57:12.800 Yeah. 00:57:13.040 --> 00:57:13.600 Thank you so much. 00:57:13.600 --> 00:57:13.880 Yeah. 00:57:13.880 --> 00:57:14.560 See you later. 00:57:14.560 --> 00:57:14.920 Bye-bye. 00:57:14.920 --> 00:57:15.120 Bye. 00:57:15.120 --> 00:57:18.420 This has been another episode of Talk Python To Me. 00:57:18.420 --> 00:57:20.220 Thank you to our sponsors. 00:57:20.220 --> 00:57:21.840 Be sure to check out what they're offering. 00:57:21.840 --> 00:57:23.240 It really helps support the show. 00:57:23.240 --> 00:57:29.040 This episode is sponsored by Posit Connect from the makers of Shiny. 00:57:29.040 --> 00:57:33.540 Publish, share, and deploy all of your data projects that you're creating using Python. 00:57:33.540 --> 00:57:40.020 Streamlit, Dash, Shiny, Bokeh, FastAPI, Flask, Quattro, Reports, Dashboards, and APIs. 00:57:40.020 --> 00:57:42.420 Posit Connect supports all of them. 00:57:42.680 --> 00:57:46.780 Try Posit Connect for free by going to talkpython.fm/posit. 00:57:46.780 --> 00:57:48.100 P-O-S-I-T. 00:57:48.100 --> 00:57:50.980 Want to level up your Python? 00:57:50.980 --> 00:57:55.020 We have one of the largest catalogs of Python video courses over at Talk Python. 00:57:55.020 --> 00:58:00.200 Our content ranges from true beginners to deeply advanced topics like memory and async. 00:58:00.200 --> 00:58:02.860 And best of all, there's not a subscription in sight. 00:58:02.860 --> 00:58:05.780 Check it out for yourself at training.talkpython.fm. 00:58:06.160 --> 00:58:07.840 Be sure to subscribe to the show. 00:58:07.840 --> 00:58:10.620 Open your favorite podcast app and search for Python. 00:58:10.620 --> 00:58:11.940 We should be right at the top. 00:58:12.460 --> 00:58:17.720 You can also find the iTunes feed at /itunes, the Google Play feed at /play, and the 00:58:17.720 --> 00:58:21.280 direct RSS feed at /rss on talkpython.fm. 00:58:21.720 --> 00:58:24.260 We're live streaming most of our recordings these days. 00:58:24.260 --> 00:58:28.380 If you want to be part of the show and have your comments featured on the air, be sure to 00:58:28.380 --> 00:58:32.040 subscribe to our YouTube channel at talkpython.fm/youtube. 00:58:32.780 --> 00:58:34.160 This is your host, Michael Kennedy. 00:58:34.160 --> 00:58:35.440 Thanks so much for listening. 00:58:35.440 --> 00:58:36.600 I really appreciate it. 00:58:36.600 --> 00:58:38.520 Now get out there and write some Python code. 00:58:38.520 --> 00:58:39.520 Bye. 00:58:39.520 --> 00:58:40.520 Bye. 00:58:40.520 --> 00:58:40.520 Bye. 00:58:40.520 --> 00:58:40.520 Bye. 00:58:40.520 --> 00:58:41.520 Bye. 00:58:41.520 --> 00:58:41.520 Bye. 00:58:41.520 --> 00:58:41.520 Bye. 00:58:41.520 --> 00:58:42.520 Bye. 00:58:42.520 --> 00:58:43.520 Bye. 00:58:43.520 --> 00:58:44.520 Bye. 00:58:44.520 --> 00:58:45.520 Bye. 00:58:45.520 --> 00:58:46.520 Bye. 00:58:46.520 --> 00:58:46.520 Bye. 00:58:46.520 --> 00:58:46.520 Bye. 00:58:46.520 --> 00:58:46.520 Bye. 00:58:46.520 --> 00:58:46.520 Bye. 00:58:46.520 --> 00:58:47.520 Bye. 00:58:47.520 --> 00:58:48.520 Bye. 00:58:48.520 --> 00:58:48.520 Bye. 00:58:48.520 --> 00:58:48.520 Bye. 00:58:48.520 --> 00:58:48.520 Bye. 00:58:48.520 --> 00:58:48.520 Bye. 00:58:48.520 --> 00:58:48.520 Bye. 00:58:48.520 --> 00:58:49.520 Bye. 00:58:49.520 --> 00:58:50.520 Bye. 00:58:50.520 --> 00:58:50.520 Bye. 00:58:50.520 --> 00:58:50.520 Bye. 00:58:50.520 --> 00:58:50.520 Bye. 00:58:50.520 --> 00:58:51.520 Bye. 00:58:51.520 --> 00:58:52.520 Bye. 00:58:52.520 --> 00:58:52.520 Bye. 00:58:52.520 --> 00:58:52.520 Bye. 00:58:52.520 --> 00:58:53.520 Bye. 00:58:53.520 --> 00:58:54.520 Bye. 00:58:54.520 --> 00:58:54.520 Bye. 00:58:54.520 --> 00:58:55.520 Bye. 00:58:55.520 --> 00:58:56.020 you 00:58:56.020 --> 00:58:56.520 you 00:58:56.520 --> 00:58:57.020 you 00:58:57.020 --> 00:58:59.020 Thank you. 00:58:59.020 --> 00:59:29.000 Thank you.