#467: Data Science Panel at PyCon 2024 Transcript

00:00 I have a special episode for you this time around. We're coming to you live from PyCon 2024. I had

00:06 the chance to sit down with some amazing people from the data science side of things. Jody Burchell,

00:11 Maria Jose, Molina Contreras, and Jessica Green. We cover a whole set of recent topics from a data

00:18 science perspective. Though we did have to cut the conversation a bit short as they were coming

00:23 from and going to talks they were all giving, but it's still a pretty deep conversation. I know you'll

00:28 enjoy it. This is Talk Python to Me, episode 467 recorded on location in Pittsburgh on May 18th, 2024.

00:36 Are you ready for your host, Darius? You're listening to Michael Kennedy on Talk Python to Me.

00:42 Live from Portland, Oregon, and this segment was made with Python.

00:46 Welcome to Talk Python to Me, a weekly podcast on Python. This is your host, Michael Kennedy.

00:54 Follow me on Mastodon, where I'm @mkennedy, and follow the podcast using @talkpython,

01:00 both on fosstodon.org. Keep up with the show and listen to over seven years of past episodes at

01:06 talkpython.fm. We've started streaming most of our episodes live on YouTube. Subscribe to our

01:11 YouTube channel over at talkpython.fm/youtube to get notified about upcoming shows and be part of

01:17 that episode. This episode is brought to you by Sentry. Don't let those errors go unnoticed. Use

01:22 Sentry like we do here at Talk Python. Sign up at talkpython.fm/sentry. And it's brought to you by

01:29 Code Comments, an original podcast from Red Hat. This podcast covers stories from technologists

01:34 who've been through tough tech transitions and share how their teams survived the journey.

01:40 Episodes are available everywhere you listen to your podcast and at talkpython.fm/code-comments.

01:47 Hello from PyCon. Hello, Jessica, Jody, Maria. Welcome to Talk Python to Me. It's awesome to

01:53 have you all here. And I'm looking forward to talking about data science, some fun LLM questions

01:58 maybe, some controversial questions, some data science tools, all sorts of good things. Of course,

02:03 before we get to that, Jody, you've been on the show a time or two, and people may know you, but

02:09 maybe not. So how about a quick introduction, what you all are into? Maria, you want to start?

02:13 Oh, okay. Well, my name is Maria. I am originally from Barcelona, but I am based in Berlin. I work

02:20 as a data scientist in a small startup that we are trying to solve some sustainability problems.

02:29 And yeah, that is me. Excellent.

02:31 Yeah. So my name's Jody and I am a data science developer advocate. I've been working in data

02:36 science for about eight years. And yeah, I'm currently working at JetBrains, as you can see

02:39 from the shirt. And the background. And so I'd say my interest at the moment is natural language

02:48 processing, because I worked in that a big chunk of my career, but core statistics will always be

02:52 my love. So tabular data, I'm there for you always. Beautiful.

02:56 Yeah. My name's Jessica. So I'm an ML engineer at Coja, which is the search engine for a better

03:02 planet. I am actually a career changer. So I used to roast coffee for a living, and I really just

03:08 got into this field in the last six years. So I don't have like any formal training. I'm a community

03:14 slash self-taught engineer. And I went through more of a like a backend focused path. And now

03:20 I've started to work in the ML realm. So really exciting.

03:24 Yeah. Very, very interesting. Another thing I absolutely love is coffee.

03:27 I think we're running on it at PyCon.

03:32 Pretty much. We are. Yeah. We're getting farther into the show and more coffee is needed.

03:38 But I do want to ask you, what do you think about being in the data science space? That's a really

03:44 different world that interacting with people all day and working with your hands more or whatever.

03:49 Yeah. How has it been with this switch? There is a lot of synergies actually, when you're

03:54 stood behind the Espresso machine and you're getting all the orders in and then you need to

03:57 like problem solve to like how you get everyone their correct order to the way that they like it.

04:04 So there was a lot of transferable skills, I will say. But I think what I found really powerful,

04:09 especially maybe learning at this specific period of time, is how accessible a lot of the tools are

04:16 today. So like how, I wouldn't say easy because I put a lot of hard work into it, but like how

04:22 possible it is, even with a background like mine to get into the field.

04:26 Awesome. I switched, I didn't have a formal education either. I took two computer college

04:31 courses just because they match, you know, I needed for something else. And yeah, I thought,

04:37 I think you can completely succeed here teaching yourself. There's so many resources. Honestly,

04:42 the problem is what resources do you choose to learn these days, right? You can spend all your

04:46 time while I'm doing another tutorial, I'm doing another class, like some point you got to start

04:50 doing something, right? Yeah. And I think actually it felt like that probably when we all started.

04:55 So data science was just getting hot when I started and oh my God, back when I started,

05:01 this is how long ago it was. There were actually like those articles like R versus Python,

05:05 like there's no conversations anyone's having anymore, but they have similar conversations.

05:08 And I think it makes it super difficult for beginners because the field felt inaccessible,

05:13 I think, eight years ago. The field feels very hostile to beginners right now, I think,

05:18 because of the AI hype. I don't actually think the field has changed that much in fundamentals.

05:24 It's just NLP has become a bigger thing and computer vision recently, but we can get into that.

05:30 Yeah, I completely agree with half of you. To be honest, for me, data science is super broad

05:36 world full of a lot of things that are kind of popping up, doing different evolution during time.

05:44 And it's so interesting to see the evolution in the last eight years. I started eight years ago in

05:52 data science. And I remember when how I was doing things eight years ago, and how I'm doing things

05:59 now. And I love it. I love to see this progression. And I am pretty sure that in eight more years,

06:07 we're gonna be in something completely different. And so I totally agree with that.

06:13 I do. And I also think data science is interesting because coming into it, you can be a data

06:19 scientist, but because some other reason, right, I could be a data science because I'm interested in

06:24 biology or sustainability or something. Whereas you're if you're a web developer, or you build

06:29 API's, or you optimize, you know, whatever, you, you're more focused on I care about the thing at

06:35 the code itself, rather than I'm trying to I care about that. And this is a tool to address that.

06:40 Yeah, yeah.

06:41 Yeah, actually, I was gonna say, I met a bioinformatician yesterday, like, that's also a data scientist, like someone who works in genetic data.

06:48 Yeah, absolutely. I had a comment from I did a show recently from about how Python's used in

06:54 neurology labs, right? And somebody wrote me, this is my favorite episode. It speaks to me.

06:58 I'm also a neurologist. Like, it's really cool. All right, we're looking out,

07:02 kind of the backside of the little bit. We're looking at the expo hall here at PyCon. So

07:08 I don't know about you all feel but for me, this is like my geek holiday, I get to come here. And

07:13 it's really special to me because I get to see my friends who I've collaborated with projects on,

07:19 and I admire and I've worked with, but I might never see them outside of this week, you know,

07:24 maybe they live in Australia or Europe or some oddly just down the street. And yet still,

07:31 I don't see them except here. So maybe what are your thoughts on PyCon here?

07:36 It's my first time attending. So I'm super stoked, I have to say like, it's slightly

07:41 overwhelming, because there's so many things going on. And like you mentioned, the opportunity to

07:46 meet so many folks that I either already knew in some capacity, but had never met or didn't meet

07:50 before, but have heard of their work. So yeah, it's been a real honor to be here, right? And get

07:55 to I mean, we are all based in Berlin. So we do actually know each other. But it's also a pleasure

08:00 just to come away on a geek holiday with friends. Yeah, and we were actually all just at PyCon DE

08:06 just before this, like a month ago. Yeah, well, yeah, it's a different scale. Let's put it

08:12 that way. But I think it's a similar feel like one thing that I value so much about the Python

08:16 community is that it's community. And I'm very lucky to have gotten involved in a program called

08:22 Hatchery, which you two have also been involved in. It's a Hatchery we're running is Humble Data.

08:28 And what I like is this program got accepted at a Python conference, which is designed for people

08:34 who have never coded and who are career changers, because I'm also a career changer from academia.

08:39 And this is what makes I think Python special, the community and I think the PyCon are an

08:44 absolute representation of that. Yeah, absolutely. For me, it's the same feeling I love to go to

08:51 different conferences of PyCon. And because we have a lot of things in common. But also,

08:59 we have differences and the different conferences bring a different point of value. And I think it's

09:07 awesome. I came here and meet friends that this is my third time in here. And I'm super, super

09:13 excited and happy. And I'm super eager to next year. And also the Python in Espanola. Yeah,

09:20 of course. And also we have even here we have a track that is PyCon Charla's to be even more

09:26 welcoming to different people from different communities. And it's just amazing. It's super

09:32 nice, to be honest. Awesome. Yeah, I definitely want to encourage people out there listening

09:36 who feel like oh, I'm not high enough of a level of Python. I know to come. I'm not ready for PyCon.

09:43 I believe last year, I haven't heard any numbers this year. I believe last year 50% of the

09:47 attendees were first time attendees. And I think that's generally true. A lot of times people are

09:53 it's their first time coming and yeah, it's I think you can get a lot out of it even if you're

09:57 not super advanced, maybe even more so than if you are super advanced. I definitely have had

10:02 the opportunity like the honor, I would actually say to like listen into conversations around

10:07 topics that I find interesting, but aren't part of my day to day work. And it's just like general

10:12 vibe that whether it's at lunch or during the breaks or after a talk, you get to partake in

10:18 these conversations, which ultimately will advance you. So if you also want to get sponsoring,

10:23 right, like a lot of people need their work to sponsor them. I think there's a lot of reasoning

10:28 behind asking for PyCon as a conference because there's so much value. Jessica, that's a great

10:33 point. And I think also, I was talking to someone earlier about how much more affordable this is

10:38 than a lot of tech conferences. A lot of them are like, how many thousand dollars is just the ticket?

10:43 And this is not that cheap, but it's relatively cheap compared. So I was going to say you could

10:50 do a plug for EuroPython while you're here. We have also the option to have grants. There is a

10:56 different programs, PyLadies Grants or the conference organizers grants. Also, this is

11:02 something that could help people to try to apply or come here. Yeah, they mentioned that at the

11:10 opening keynote or the introductions before the keynote. It's some significant number of grants

11:16 that were given. I can't remember the number, but it's like half a million dollars or something

11:19 in grants. Was that what it was? I think it was around that scale. Yeah. Yeah. Yeah. It's a really

11:24 big deal. And I suppose all three of you being from Berlin, we should say generally the same

11:29 stuff applies to EuroPython as well, I imagine. Right? Yeah. So if you're in Europe, the biggest

11:34 deal is to get all the way to the US, maybe go to EuroPython as well, which would be fun. Yeah.

11:39 Or something more local. This portion of Talk Python to Me is brought to you by

11:44 Open Telemetry support at Sentry. In the previous two episodes, you heard how we use Sentry's error

11:50 monitoring at Talk Python, and how distributed tracing connects errors, performance and slowdowns

11:56 and more across services and tiers. But you may be thinking, our company uses Open Telemetry. So

12:03 it doesn't make sense for us to switch to Sentry. After all, Open Telemetry is a standard, and you've

12:08 already adopted it, right? Did you know, with just a couple of lines of code, you can connect

12:14 Open Telemetry's monitoring and reporting to Sentry's backend. Open Telemetry does not come

12:20 with a backend to store your data, analytics on top of that data, a UI or error monitoring. And

12:26 that's exactly what you get when you integrate Sentry with your Open Telemetry setup. Don't fly

12:31 blind, fix and monitor code faster with Sentry. Integrate your Open Telemetry systems with Sentry

12:38 and see what you've been missing. Create your Sentry account at talkpython.fm/sentry-telemetry.

12:44 And when you sign up, use the code TALKPYTHON, all caps, no spaces. It's good for two free months of

12:50 Sentry's business plan, which will give you 20 times as many monthly events as well as other

12:55 features. My thanks to Sentry for supporting Talk Python and me. - Jody, you have been on the

13:01 receiving end of many, many questions and you've been, let's see here, doing demos, sworn with

13:07 people for a day and a half. I'm surprised you still have your voice. - I've got to give a talk

13:12 in two hours too, so I hope I have a voice. - Speak quietly. Save a little bit for that. One

13:20 of the questions you said was that people are still just have core data science questions.

13:24 They're not necessarily trying to figure out how LLMs are going to change the world, but how do

13:29 you do that with pandas or whatever? What are your thoughts on this? What are your takeaways? - So I

13:34 alluded to the fact I have an academic background. I've probably talked about this on the last

13:37 podcast, but basically my background is in behavioral sciences, so a lot of core statistics

13:44 and working with what's called tabular data, data in tables. And pretty much I would say, look, this

13:50 is a guesstimate. This is not scientific, but my kind of gut feeling, PyCon after PyCon, conference

13:56 after conference that I do, I think like 80% of people are probably still doing this stuff because

14:01 business questions are not necessarily solved with the cutting edge. Business questions are

14:05 solved with the simplest possible models that will address your needs. I think we talked about this in

14:11 the last podcast. So like for an example, my last job, we had to deal with low latency systems, like

14:17 very low latency. So we used a decision tree to solve the problem. Decision tree is a very old

14:23 algorithm. It's not sexy anymore, but everyone's secretly still using it. And so yeah, some people

14:28 are doing cutting edge LLM stuff. But my feeling is this is a technology that maybe has more interest

14:35 than real profitable applications, because these are expensive models to run and deploy and to set

14:43 up reliable pipelines for. Yeah. My feeling is, gut feeling is a lot of people are still just

14:48 doing boring linear regression, which I will defend until the day I die. My favorite algorithm.

14:53 Amazing. Yeah.

14:54 Yeah. And I mean, I think we've seen that in our work as well is we don't per se need the biggest,

15:00 fanciest thing. We need something that works and provides users with useful information. I think

15:06 there's also still a lot of problems with large language models, like Simon alluded to in the

15:11 keynote today around security. So if you want to put this into a product, it's still kind of early

15:17 days, but I don't think those base kind of NLP techniques are going to go away anytime soon.

15:23 And I think like we spoke about learners earlier and people coming into the field,

15:28 there's still a huge amount of value just to go and learn this core aspects that will serve

15:33 you really well. Absolutely. Way more than LLMs and AIs and all that stuff.

15:38 You can use a LLM to learn it. That's what we just saw in the keynote.

15:43 Yeah, absolutely. And I also think what people are going to do with LLMs and stuff like that,

15:49 ask it to help give me this little bit of code or that bit of code, but you're going to need to be

15:53 able to look at it and say, yeah, that does make sense. Yeah, that does fit in. And so you need to

15:56 know that's a reasonable use of pandas. What do you think, Maria? I completely agree. The LLM's

16:03 role is kind of complex. I think that it has a lot of potential. And I think that a lot of people

16:09 could see this potential and everyone is getting very excited and even a bit in a hype because of

16:15 that. However, it has a lot of limitations still nowadays, I can tell you, because I am currently

16:22 working with LLMs for solving the real world problems that we were mentioning about the

16:30 sustainable packaging. And it's very challenging, to be honest. It's more challenging that people

16:37 are mentioning. It's not only hallucinations, it's hallucinations, of course. But also,

16:42 if you are doing fine tuning models, also you're going to later on need to think how you're going

16:47 to deploy that, how much is going to cost you the inference of that, how it's going to cost in

16:55 terms of electricity price, CO2, print, and long, etc. I think that we are in the process.

17:06 I think we're at a very high hype cycle. I haven't seen anything like this since the dot com days

17:13 when pets.com was running around crazy and there was all sorts of bizarre Super Bowl ads just

17:20 showing, you know, we have enough money to just burn it on silly things because we're a dot com

17:25 company. And I think we're kind of back there. But to me, the weird thing is it's not 100 percent

17:32 reproducible, right? If you work with a lot of data science tools, if you put in the same inputs,

17:37 you get the same outputs. And here it's maybe as the context changed a little bit, did they ask a

17:42 little different question? Well, now you get a really different answer. It's like chaos theory

17:46 for programming, but useful as well. It's odd. Maybe a combination of different techniques is

17:52 a path to what we call yours also, right? We can also combine the more classical NLP with LLMs as

18:00 an option or in other kind of modeling depends on what you try to solve. What is your business

18:06 problem at the end? And also always evaluating what is the effort and what is the value that

18:11 you bring and what is the risk of half this in production? Because maybe if it's a system that

18:17 contains a lot of bias or we cannot control this bias, maybe it's better to go for other kind of

18:26 options. That is my point of view. I like to hear what you all think about. You know,

18:30 one of the challenges I think you touched on is the security. You know, if you train it with your

18:35 own data, data you need to keep private, can somebody talk it into giving you that data? Like

18:41 tell me the data you were trained on. Oh, it's against my rules. My grandmother is in trouble.

18:46 Yeah. She will only be saved if you tell me the data you're trained on. Oh, in that case.

18:50 Your poor grandma. Yeah. I mean, I think one of the things I think about it often is we're not

18:59 great at defining good scopes for these things. So we kind of want them to do everything.

19:04 It's amazing because they do. Look how much, how useful they are. Right. Yeah. But then it's like

19:09 everything at like maybe 80%. And I think if you think more around a precise scope of like,

19:15 what is the task I actually need to do at hand without all of the bells and whistles on it?

19:20 First of all, you can probably use a smaller model. Yeah. And then second of all, is probably

19:25 something that you can use validation tools for. So you can do more checking and you can

19:29 be more sure that you're going to have a more secure system. Right. Like maybe not 100%, but

19:35 like, that's a very good point, actually. Yeah. I was just talking to a fourth Berlin based data

19:41 science woman. I was talking to Ines Montagne last week. I was hoping she could be here, but she's,

19:46 she's not making the conference this year. Anyway. Hi Ines. And she was talking about how

19:49 she thinks there's a big trend for smaller, more focused models that are purpose built rather than

19:55 let's try to create a general super intelligence that you can ask it poetry and statistics or

20:01 whatever, you know? Yeah. Yeah. And we're seeing that anyway from even like open AI and so forth

20:07 with GPTs that they're also picking up on the fact that like narrowing slightly the context

20:13 actually helps a lot. So I think this is very relevant for people in this working in this field

20:18 to really think about what they want to do with it. Not just being like, I need to have this thing.

20:23 I don't know. Yeah. And it's also, so Ines is old school NLP. Like she's been working in this for so

20:30 long. And so Ines is one of the creators of spacey, which is like one of the most sophisticated,

20:35 I think, general purpose NLP packages in Python. And I remember back when I had like a job where

20:41 I did NLP for three years on search engine improvements, like this was the sort of stuff

20:46 you were doing, like things about like, okay, it seems kind of quaint now, but it's still

20:50 really important. Like how can you clean your data effectively? And it's very complex when it

20:55 comes to tech stuff. And so, yeah, like Ines, of course she's completely right, but she's seen all

21:01 of this. She knows where this is going. Yeah, absolutely. Absolutely. Let's touch on some

21:05 tools. I know Maria, you had some interesting ones, just general data science tools that while

21:12 people are listening, you should be like, let's check the LLM or as Jody puts it old school,

21:17 just core data science. Yeah, yeah. It's going to depend on what kind of problem you want to solve.

21:24 Again, it's like, it's not the tool. This is my perspective. It's not only one tool or 10 tools.

21:31 It depends on your problem. And depends on your problem, we have tools that are going to help us

21:37 more or easier than others. For instance, some tools that I'm using currently, just for giving

21:45 you an example, this LangChain or Discord and yeah, and they are two open source libraries.

21:55 LangChain is more focusing in that chat system in case that you want to develop a chat system or

22:03 of course has a lot of more applications because LangChain is super useful also for handling all the

22:11 large language models. Yeah. There's some cool boosts here that are boosted with cool products

22:16 based on LangChain as well. Oh, really? I'm going to take a look.

22:20 That then you export as a Python application. It's very neat. Anyway. Very good. Yeah. But you

22:27 also said Discord. Yeah. G I S K R D. Exactly. Okay. It's the one that has a turtle, the logo,

22:34 very cute. These people is developing a library for evaluating the models. Try to take a look

22:43 in the bias of the system, has tests, test your models and generate metrics to help you understand

22:53 if the model that you are using or training or fine tuning is something that you can trust or

22:59 not or you need to re-evaluate or restart the system or whatever you need to do. I think these

23:06 kind of libraries are super necessary, especially right now that the still it's very young, the

23:13 field. And I think that they are very, very important. This portion of Talk Python to Me

23:19 is brought to you by Code Comments, an original podcast from Red Hat. You know, when you're

23:23 working on a project and you leave behind a small comment in the code, maybe you're hoping to help

23:29 others learn what isn't clear at first. Sometimes that code comment tells a story of a challenging

23:35 journey to the current state of the project. Code Comments, the podcast features technologists who

23:40 have been through tough tech transitions, and they share how their teams survived that journey.

23:46 The host, Jamie Parker is a Red Hatter and an experienced engineer. In each episode, Jamie

23:51 recounts the stories of technologists from across the industry who've been on a journey implementing

23:57 new technologies. I recently listened to an episode about DevOps from the folks at Worldwide

24:02 Technology. The hardest challenge turned out to be getting buy in on the new tech stack rather than

24:08 using that tech stack directly. It's a message that we can all relate to. And I'm sure you can

24:13 take some hard one lessons back to your own team. Give Code Comments a listen. Search for Code

24:19 Comments in your podcast player or just use our link, talkpython.fm/code-comments. The link is in

24:26 your podcast players show notes. Thank you to Code Comments and Red Hat for supporting Talk Python to

24:31 me. -Jerry? -Yeah, so maybe I'm going to do a little plug for my talk. So when I was doing psychology,

24:38 I was fascinated by psychometrics. And what you learn when you learn psychometrics is

24:43 measurement captures one specific thing, and you need to be very clear about what it captures. And

24:49 so at the moment, we're seeing a lot of leaderboards to help people evaluate LLM performance, but also

24:56 things like hallucination rates or things like bias and toxicity. What we need to understand is

25:00 these things have extremely specific definitions. So in my talk, I'm going to be delving into a

25:05 package which I do, a package, sorry, a measurement that I love called Truthful QA. But Truthful QA is

25:10 designed to measure a specific type of hallucinations in English-speaking communities,

25:15 because it assesses incorrect facts, things like misconceptions, misinformation, conspiracies.

25:22 They're not going to be present in other languages. And so it's not as easy as looking at,

25:26 okay, this model has a low hallucination rate. What does that mean? Or this model has good

25:31 performance. Does it have that performance in your domain? How did they assess that? So it's very

25:36 boring, but actually it's not because measurement is super sexy. You need to think about this stuff.

25:41 It's really interesting, but it's challenging and it requires a lot of hard graph from you.

25:46 Awesome. And while people will be watching this in the future, after your talk is out, that talk

25:52 will be on YouTube, right? Yes, it'll be recorded. Yeah. So people can check out your talk. What's

25:56 the title? Lies, Damn Lies and Large Language Models. Oh, I love it. It's the best title I've

26:01 ever come up with. That is a good title. I love it. Jessica, tools, libraries, packages? Maybe

26:08 I'll plug my tutorial that was two days ago and we'll also be recording somewhere at some point.

26:14 We were working on looking at monitoring and observability of Python applications,

26:20 which could well be your AI, LLM kind of thing. And we're using a package called Code Carbon.

26:27 So it measures the carbon emissions of your code, of your workload. So this is one way that we can

26:35 start to kind of get an idea of the impact that we're having with these things. So I think it's

26:41 a really great library. It's open source. They're looking for contributors. And it's not the full

26:46 picture, of course, because if you're using like a cloud provider, you also need to ask and follow

26:51 up with them to get further information. How much of there is renewable versus non-renewable

26:56 energy? Yeah, exactly. Is it a coal plant? Please say it's not a coal plant. Yeah. We live in

27:01 Germany. Germany is not too bad, but yeah, there is a lot of coal in there. So I think this is a

27:07 great way to start to think about it as technologists, because often it's easy to see

27:11 these problems as something out of our control or beyond the scope of the work that we do every day.

27:18 But I think there's still a lot that we actually can do. Make a huge difference. And just as simple

27:23 as could we cache this output and then reuse it or let it run for five minutes on the cluster and,

27:29 oh, we're not that big of a hurry. Let's let it run over and over and over and then let it run

27:33 in continuous integration. And exactly. Yeah, exactly. And I mean, the good thing that also is

27:38 those things cost money, too. So, yeah, you don't just need to save the planet. You can also save

27:42 yourself some money. It's not 100 percent the same, but usually, yeah, you're you have this

27:48 benefit that other people care more about money or time, money and time. Right. But it's easier

27:54 to sell. Yeah, absolutely. You know, I've had a couple of episodes on this previously, but just

27:58 give people a sense of how much energy is in training some of these large models. And since

28:04 it's one of the shows that I talked to, there was some research done to say training one of

28:09 these large models just one time is as much as, say, a person driving a car for a year type of

28:14 energy. And you're like, oh, that's no joke. And so that that might encourage you to run

28:20 smaller models or things like that, which I think for a long time we were thinking like, oh, it's

28:25 the training that's everything. And then it's kind of like fine once the training's done. But

28:30 actually, the inference is also just as compute heavy when you see the slow words coming out.

28:35 That's yeah. And it's because it's autoregressive. It moves. Yeah, I think it's you have to look at

28:41 it holistically. I think it's very useful to have these metrics that we compare to other things,

28:46 because then we get a sense of like how daunting that is. I think like comparing it to like air

28:52 travel or like to cars and so forth is good. But we tend to focus a little bit on like, oh,

28:58 it's just this part of the system and not the system as a whole. Well, I think the training

29:03 was done a lot previously and the usage was done less. And now the usage has just gone out of

29:09 control. Like if you don't have a eye in your menu ordering app, it's a useless thing, right?

29:14 It's like everybody needs it. Yeah, they don't really need it, I think. But they think they

29:18 need it or the VCs think they need it or something. I think also like a lot of people might think,

29:23 oh, we need to train our own models. But with things like rag, like retrieval, augmentation,

29:27 generation, that now a lot of vector database services are promoting and educating people

29:32 around how to do. That's not true. So you can take like a base model and start to give it your data

29:38 without the need to like tune something yourself, like train something yourself. Yeah, we are very,

29:44 very nearly out of time here, ladies. We all have different things we got to run off and do. But let

29:49 me just close out with some quick thoughts. And really, this deserves maybe two hours, but we've

29:54 got two minutes for data scientists out there listening who are concerned that things like

30:00 Copilot and Devon and all these weird, I'll write code for you things are going to make learning

30:06 data science not relevant. What do you think? I think it's still going to be super relevant, but

30:11 I think that going to help a lot. And I think that could be seen as a potential useful tool

30:21 that could help to a lot of people. It's even for beginners for learning. I think for people who

30:29 are starting to code could be super useful to try to take a look with Copilot or with LLMs and say,

30:37 hey, I don't understand the code. Can you explain to me what is happening in this function and

30:41 something like that. From here to be able to introduce an idea and have a production ready

30:49 code, we are very far away, to be honest, right now. We need more work and the field needs to

30:56 improve a bit of that. But I truly believe that's going to help us a lot at some point in time.

31:04 I think maybe I'll take a different perspective and say that I think for data scientists,

31:09 the core concern for us is not really code. It's more data, I guess.

31:14 Oh, yeah, absolutely.

31:15 Yeah. So I think I'm seeing some potential, even with our own tools at JetBrains,

31:20 to potentially help introduce people to the idea of how to work with data. But there's not really

31:26 necessarily huge shortcuts here because you're still going to learn how to clean a data set

31:30 and evaluate for quality. And so the science part of data science, I don't think it's ever

31:36 going to go away. You still need to be able to think about business problems. You still need

31:39 to be able to think about data.

31:40 We'll be there forever.

31:41 It'll be there forever. Thank God. It's so good.

31:44 That's fun.

31:46 Maybe as not a data scientist, I can give a slightly different perspective. I feel like

31:51 because it comes up just for general programming all the time as well, right? And I think one of

31:56 the things that is at the moment most hurting our industry is the lack of getting people into

32:02 junior level jobs and not AI or any technology itself. It's a very human problem. As are pretty

32:09 much all of the problems with AI itself. So I think, to be honest, what we need to do is really

32:17 hire more juniors, make more entry level programs, get people into these positions and get them

32:23 trained up on using the tools. We don't need to gatekeep. There's going to be plenty of work for

32:28 the rest of us for the next foreseeable future, considering all the big social problems that we

32:33 have to solve. So I just think we should do that.

32:37 All right. Well, let's leave it there. Maria, Jody, Jessica, thank you so much for being on

32:42 the show.

32:42 Thank you.

32:42 Thank you very much. It was amazing.

32:44 Bye.

32:45 Bye.

32:45 Bye.

32:46 This has been another episode of Talk Python to Me. Thank you to our sponsors. Be sure to

32:52 check out what they're offering. It really helps support the show. Take some stress out of your

32:56 life. Get notified immediately about errors and performance issues in your web or mobile

33:01 applications with Sentry. Just visit talkpython.fm/sentry and get started for free. And be

33:08 sure to use the promo code TALKPYTHON, all one word.

33:11 Code comments and original podcast from Red Hat. This podcast covers stories from technologists

33:17 who've been through tough tech transitions and share how their teams survived the journey.

33:23 Episodes are available everywhere you listen to your podcast and at talkpython.fm/code-comments.

33:29 Want to level up your Python? We have one of the largest catalogs of Python video courses over at

33:34 Talk Python. Our content ranges from true beginners to deeply advanced topics like memory and async.

33:40 And best of all, there's not a subscription in sight. Check it out for yourself at

33:43 training.talkpython.fm.

33:45 Be sure to subscribe to the show. Open your favorite podcast app and search for Python.

33:50 We should be right at the top. You can also find the iTunes feed at /itunes,

33:54 the Google Play feed at /play, and the Direct RSS feed at /rss on talkpython.fm.

34:00 We're live streaming most of our recordings these days. If you want to be part of the show and have

34:05 your comments featured on the air, be sure to subscribe to our YouTube channel at talkpython.fm/youtube.

34:12 This is your host, Michael Kennedy. Thanks so much for listening. I really appreciate it.

34:16 Now get out there and write some Python code.

34:18 [Music]