WEBVTT 00:00:00.001 --> 00:00:03.060 Are you building or running an internal machine learning team? 00:00:03.060 --> 00:00:04.860 How about looking for a new ML position? 00:00:04.860 --> 00:00:09.840 On this episode, I talk with Chip Heughan from Snorkel AI about building ML teams, 00:00:09.840 --> 00:00:12.900 finding ML positions, and teaching machine learning. 00:00:12.900 --> 00:00:17.980 This is Talk Python To Me, episode 298, recorded November 18th, 2020. 00:00:17.980 --> 00:00:35.800 Welcome to Talk Python To Me, a weekly podcast on Python, the language, the libraries, the 00:00:35.800 --> 00:00:37.200 ecosystem, and the personalities. 00:00:37.200 --> 00:00:39.020 This is your host, Michael Kennedy. 00:00:39.020 --> 00:00:43.160 Follow me on Twitter, where I'm @mkennedy, and keep up with the show and listen to past 00:00:43.160 --> 00:00:48.060 episodes at talkpython.fm, and follow the show on Twitter via at Talk Python. 00:00:48.060 --> 00:00:51.340 This episode is sponsored by Datadog and Linode. 00:00:51.340 --> 00:00:53.880 Please check out what they're sponsoring during their episodes. 00:00:53.880 --> 00:00:55.420 It really helps support the show. 00:00:55.420 --> 00:00:57.640 Chip, welcome to Talk Python To Me. 00:00:57.640 --> 00:00:58.480 Hi, Michael. 00:00:58.480 --> 00:00:59.400 Nice to meet you. 00:00:59.400 --> 00:01:00.700 Thanks so much for having me. 00:01:00.700 --> 00:01:03.340 Nice to meet you, and thanks for coming on the show. 00:01:03.340 --> 00:01:08.560 It's going to be a lot of fun to talk about ML and putting ML into production and building 00:01:08.560 --> 00:01:09.320 ML teams. 00:01:09.320 --> 00:01:12.260 We're going to talk a lot of, probably cover a lot of buzzwords, right? 00:01:12.260 --> 00:01:16.220 Like, AIS and ML are so top of mind in all of the... 00:01:16.220 --> 00:01:19.320 I need to impress people by throwing out all the buzzwords, yeah. 00:01:19.320 --> 00:01:20.680 That's right, right. 00:01:20.680 --> 00:01:25.920 IBM had this really funny commercial, which is ironic that it was IBM, but it had this really 00:01:25.920 --> 00:01:29.200 funny commercial, like, 10 years ago called Buzzword Bingo. 00:01:29.200 --> 00:01:32.060 I don't know if you ever saw that, but it was really, really hilarious. 00:01:32.060 --> 00:01:35.580 We'll see if I can link to it in the show notes, if I can dig it back up on YouTube. 00:01:35.580 --> 00:01:40.440 But yeah, so we could definitely win that one today, just because it's such a growing 00:01:40.440 --> 00:01:41.620 and interesting topic. 00:01:41.620 --> 00:01:42.100 Yeah. 00:01:42.100 --> 00:01:44.940 But before we get to all that, of course, let's start with your story. 00:01:44.940 --> 00:01:48.520 How do you get into working with programming in Python and machine learning? 00:01:48.520 --> 00:01:54.940 It's really funny because when I was younger, I thought being a programmer was the most boring 00:01:54.940 --> 00:01:55.800 job in the world. 00:01:55.800 --> 00:02:00.620 I was like, why would anyone want to spend the rest of their life sitting in a basement looking 00:02:00.620 --> 00:02:01.500 to come to screen? 00:02:02.180 --> 00:02:04.540 This is for the anti-social people. 00:02:04.540 --> 00:02:06.760 They don't want to go outside. 00:02:06.760 --> 00:02:08.520 They just, like you said, sit in a basement. 00:02:08.520 --> 00:02:08.780 Yeah. 00:02:08.780 --> 00:02:09.900 Don't they have friends? 00:02:09.900 --> 00:02:10.860 Yeah, I know. 00:02:10.860 --> 00:02:11.660 Yes, exactly. 00:02:11.660 --> 00:02:12.480 Come on. 00:02:12.480 --> 00:02:12.720 Yeah. 00:02:12.960 --> 00:02:16.080 The views changed over the last 20 years or so, right? 00:02:16.080 --> 00:02:16.760 Like it's... 00:02:16.760 --> 00:02:17.020 Yeah. 00:02:17.380 --> 00:02:20.120 Society has sort of viewed that differently, but yeah, it's how it was. 00:02:20.120 --> 00:02:24.800 I think as far as growing up, you just realize you becomes a person that you make fun of, 00:02:24.800 --> 00:02:25.080 you know? 00:02:25.080 --> 00:02:27.100 So that's the story of my life. 00:02:27.100 --> 00:02:30.480 So when I was younger, I actually come from a writing background. 00:02:30.900 --> 00:02:32.580 So I was traveling the world. 00:02:32.580 --> 00:02:36.000 I know it sounds like nice and stuff, but it's like not that nice. 00:02:36.000 --> 00:02:40.900 So I was traveling the world and writing a lot about culture, people, a lot of food. 00:02:40.900 --> 00:02:45.900 So I can't just stand for thinking I would major in script writing because I thought it 00:02:45.900 --> 00:02:46.780 would be fun. 00:02:46.780 --> 00:02:51.600 But then I took some CS courses and they talked to CS friends and they were like, what? 00:02:51.600 --> 00:02:53.480 Is this what you make for an internship? 00:02:53.480 --> 00:02:56.460 It's like what my family makes like the entire year or something. 00:02:56.460 --> 00:02:58.840 So I was like, what is so cool about it? 00:02:58.840 --> 00:03:00.180 So I took CS courses. 00:03:00.180 --> 00:03:00.520 Wait a minute. 00:03:00.660 --> 00:03:02.000 Maybe I should pay attention to this, right? 00:03:02.000 --> 00:03:03.120 This is starting to sound good. 00:03:03.120 --> 00:03:05.140 Yes, yes. 00:03:05.140 --> 00:03:10.600 So yeah, so I took some CS courses and I think Stanford did a really great job of like getting 00:03:10.600 --> 00:03:15.980 people interested in computer science because also introductory courses are extremely well 00:03:15.980 --> 00:03:19.720 designed and exercises are not just like, I don't know, boring things. 00:03:19.720 --> 00:03:22.180 It's like trying to design a button to increase something. 00:03:22.180 --> 00:03:22.520 I don't know. 00:03:22.520 --> 00:03:27.360 I don't know what boring lectures are, but there's an exercise like building games. 00:03:27.360 --> 00:03:28.360 So you could play games. 00:03:28.360 --> 00:03:28.980 Yeah. 00:03:28.980 --> 00:03:30.700 So a lot of fun things. 00:03:30.700 --> 00:03:32.520 So I took them and I took the courses. 00:03:32.520 --> 00:03:33.720 I really enjoyed them. 00:03:33.720 --> 00:03:35.040 And then I took more courses. 00:03:35.040 --> 00:03:41.480 I think initially it was more of like financial needs because I need to TA to get some pocket 00:03:41.480 --> 00:03:41.820 money. 00:03:41.820 --> 00:03:46.140 But then I TA'd and then I met some wonderful people who are so TAing. 00:03:46.140 --> 00:03:48.240 I think at Stanford was of course, we couldn't session leading. 00:03:48.240 --> 00:03:49.660 It was really fun. 00:03:49.920 --> 00:03:52.700 And I think I just getting sucked into it. 00:03:52.700 --> 00:03:57.260 And fast forward four years, I major in computer science. 00:03:57.260 --> 00:03:57.740 Yeah. 00:03:57.740 --> 00:03:58.420 That's amazing. 00:03:58.420 --> 00:04:00.020 Do you still do any writing? 00:04:00.020 --> 00:04:00.860 Oh, yes. 00:04:00.860 --> 00:04:02.480 I still write a lot. 00:04:02.760 --> 00:04:06.820 So I think it's kind of tricky because of writing. 00:04:06.820 --> 00:04:10.900 And technical writing is very different from the kind of writing I did before. 00:04:10.900 --> 00:04:13.760 So I can't really switch among them easily. 00:04:13.760 --> 00:04:19.280 So I think for me, it would be like a month just focused on technical writing. 00:04:19.280 --> 00:04:21.660 So like blog posts and documentation. 00:04:22.300 --> 00:04:28.240 So I write a lot about, for example, like paper summarizations or just some new techniques 00:04:28.240 --> 00:04:28.900 that I learned. 00:04:28.900 --> 00:04:30.520 I think recently I did something about. 00:04:30.520 --> 00:04:35.940 So I look up like 200 machining tools I could find and I try to analyze. 00:04:35.940 --> 00:04:36.460 Oh, no, no. 00:04:36.460 --> 00:04:37.140 Wow. 00:04:37.140 --> 00:04:38.380 It took me so much time. 00:04:38.380 --> 00:04:42.100 So the kind of writing is very different because in people in political writing, they want to 00:04:42.100 --> 00:04:42.940 get to the point, right? 00:04:42.940 --> 00:04:46.920 But also sometimes I still like to write stories like non-fictions. 00:04:46.920 --> 00:04:50.080 And when you write stories, people want to take another journey. 00:04:50.080 --> 00:04:53.200 They just don't want to show the destination right away. 00:04:53.200 --> 00:04:57.520 So I tried and I do a month of that and then I switch my set to another. 00:04:57.520 --> 00:04:59.120 So yes, I still write a lot. 00:04:59.120 --> 00:04:59.720 That's cool. 00:04:59.720 --> 00:05:00.860 It sounds like a lot of fun. 00:05:00.860 --> 00:05:06.680 So one of the things that is interesting about your story is like you decided to do some programming 00:05:06.680 --> 00:05:07.980 classes and get into it. 00:05:07.980 --> 00:05:12.700 And at first you were not so sure that that was the full on path that you wanted to go 00:05:12.700 --> 00:05:13.020 down. 00:05:13.020 --> 00:05:16.540 And as you got deeper into it, you saw it as more interesting. 00:05:16.540 --> 00:05:20.900 You made friends and connections and you sort of saw the human side of it and got sucked 00:05:20.900 --> 00:05:21.960 more and more into it, right? 00:05:21.960 --> 00:05:22.680 Yeah. 00:05:22.680 --> 00:05:24.660 Like, yeah, programmers do have friends. 00:05:24.660 --> 00:05:24.920 Yes. 00:05:24.920 --> 00:05:25.640 I learned that. 00:05:25.640 --> 00:05:27.260 So yeah, that's good. 00:05:27.260 --> 00:05:27.500 Right. 00:05:27.500 --> 00:05:28.540 Encouraging. 00:05:28.540 --> 00:05:29.060 Yeah. 00:05:29.340 --> 00:05:34.180 Well, the thing that's interesting to me is a lot of times you hear people early in 00:05:34.180 --> 00:05:36.780 their career thinking about like, well, what should they study? 00:05:36.780 --> 00:05:38.380 What should they go into? 00:05:38.380 --> 00:05:40.340 Or if they're changing careers, what should they go into? 00:05:40.340 --> 00:05:43.320 And a lot of times the advice is follow your passion. 00:05:43.320 --> 00:05:44.440 Like, what are you passionate about? 00:05:44.440 --> 00:05:45.800 Like, well, I'm really passionate about soccer. 00:05:45.800 --> 00:05:46.100 Okay. 00:05:46.100 --> 00:05:47.020 Well, go into soccer. 00:05:47.120 --> 00:05:52.140 Like what I've, I think I've seen over the years is a lot of people who are actually 00:05:52.140 --> 00:05:53.900 super passionate about what they're doing. 00:05:53.900 --> 00:05:57.760 They didn't go to it because they just knew from the beginning that that was it. 00:05:57.760 --> 00:06:02.180 It's like somehow as you get pulled in, as you master a topic and you learn more about 00:06:02.180 --> 00:06:07.140 it, like it, it's like this mastery and understanding leads to passion, not the other way around. 00:06:07.140 --> 00:06:07.560 Yeah. 00:06:07.560 --> 00:06:09.200 No, I totally agree with you. 00:06:09.200 --> 00:06:11.940 So this is something I think about sometimes. 00:06:12.200 --> 00:06:16.300 So I realized over the years that there are things I thought I would enjoy doing. 00:06:16.300 --> 00:06:17.540 So I thought it was my passion. 00:06:17.540 --> 00:06:21.260 So at some point I think like I would totally want to do AI research. 00:06:21.260 --> 00:06:25.680 And then I took three months off in travel and I realized it's like I read every day and 00:06:25.680 --> 00:06:28.980 write every day, but I didn't read a single paper in the three months. 00:06:28.980 --> 00:06:31.360 So like, it's just not something I enjoy doing. 00:06:31.360 --> 00:06:36.100 So I realized that I want to become the person who do AI research, but I don't want to do 00:06:36.100 --> 00:06:37.260 actually be doing it. 00:06:37.260 --> 00:06:37.780 Yeah. 00:06:37.780 --> 00:06:41.960 So I think those things, you just need to like spend time and think about it and like 00:06:41.960 --> 00:06:45.020 to just observe what you do in the three times you'll know what's so passionate. 00:06:45.020 --> 00:06:47.840 And so don't think of passion as something you find. 00:06:47.840 --> 00:06:49.640 I think passion is something you cultivate. 00:06:49.640 --> 00:06:51.640 So you might not know. 00:06:51.640 --> 00:06:53.100 So, so you know, have you ever had this moment? 00:06:53.100 --> 00:06:55.920 Like you study, you learn something and it's like so weird. 00:06:55.920 --> 00:06:56.720 You don't understand anything. 00:06:56.720 --> 00:06:57.680 It's like miserable. 00:06:57.680 --> 00:06:59.320 And it has a gotcha moment. 00:06:59.320 --> 00:07:00.900 Everything just makes so much sense. 00:07:00.900 --> 00:07:03.160 And it's just like keep doing deeper and deeper into it. 00:07:03.160 --> 00:07:04.500 And it suddenly becomes a passion. 00:07:04.500 --> 00:07:04.980 Yeah. 00:07:04.980 --> 00:07:08.940 So I do think that in the beginning, I think it's really useful to just try a new thing, 00:07:08.940 --> 00:07:11.720 but like, don't just do too short, like give it some time to actually 00:07:11.720 --> 00:07:14.580 actually see how much you learn, how much you grow in it. 00:07:14.580 --> 00:07:16.760 And I think it's no, there's no pain. 00:07:16.760 --> 00:07:18.060 You know, I just, there's no shame. 00:07:18.060 --> 00:07:21.500 It's like leaving something you don't think is for you, but you definitely need to give 00:07:21.500 --> 00:07:22.160 things to Tom. 00:07:22.160 --> 00:07:23.020 That's good advice. 00:07:23.020 --> 00:07:23.680 Yeah. 00:07:23.680 --> 00:07:27.000 A lot of people want to find that thing that they love and just go for it. 00:07:27.000 --> 00:07:31.580 And I think actually the answer might be just experience a lot of things and then decide, 00:07:31.580 --> 00:07:31.820 right? 00:07:31.820 --> 00:07:32.340 Which is awesome. 00:07:32.340 --> 00:07:36.140 So you took some CS courses and did a bunch of writing. 00:07:36.140 --> 00:07:41.040 And now all of a sudden you're on the other side of the podium, right? 00:07:41.040 --> 00:07:42.380 You're doing a little teaching as well. 00:07:42.380 --> 00:07:42.840 Yes. 00:07:42.840 --> 00:07:45.940 I don't think learning and teaching have to be mutually exclusive. 00:07:45.940 --> 00:07:51.260 So I started teaching when I was a student and I started out not because I was an expert. 00:07:51.260 --> 00:07:51.920 As a TA? 00:07:52.320 --> 00:07:54.060 No, as an instructor for the course. 00:07:54.060 --> 00:07:58.940 So I started out not because, I started teaching it not because I was an expert, but rather 00:07:58.940 --> 00:08:00.560 because I wanted to become an expert. 00:08:00.560 --> 00:08:06.040 And so in the beginning it was like, so my first course I taught as an instructor was TensorFlow 00:08:06.040 --> 00:08:07.320 for deep learning research. 00:08:07.320 --> 00:08:12.960 So at that point, TensorFlow was fairly new and I was using it in my own internship and I couldn't 00:08:12.960 --> 00:08:15.220 really find good training material. 00:08:15.220 --> 00:08:19.440 So I went to my professors and was like, hey, can you take a course on it? 00:08:19.440 --> 00:08:20.180 Take a course on it? 00:08:20.240 --> 00:08:22.020 And they were like, oh, we don't have time. 00:08:22.020 --> 00:08:24.780 Like, you know, for professors, you put their name on the line. 00:08:24.780 --> 00:08:28.020 They could have to like, and I was going to have to make a lot of investment to make it 00:08:28.020 --> 00:08:28.280 good. 00:08:28.280 --> 00:08:29.800 And they were like, why don't you do it? 00:08:29.800 --> 00:08:30.540 And I was like, what? 00:08:30.540 --> 00:08:31.120 And I was like, yeah. 00:08:31.120 --> 00:08:34.660 They have Stanford has this thing that you allow students to initiate course. 00:08:34.660 --> 00:08:35.660 So I did it. 00:08:35.660 --> 00:08:40.080 And in the process, so it's not really it's like having a group of people who also want 00:08:40.080 --> 00:08:41.960 to learn TensorFlow and learn to quit them. 00:08:41.960 --> 00:08:46.540 So I just tried to like anticipate a lot of questions by just Googling a lot. 00:08:46.680 --> 00:08:50.420 It was like, I know I spent like half of my waking hours, like a flow or something. 00:08:50.420 --> 00:08:52.440 So that's how I started. 00:08:52.440 --> 00:08:54.540 Anything now I still continue doing it. 00:08:54.540 --> 00:08:55.140 That's cool. 00:08:55.140 --> 00:08:58.180 I think you could just learn so much when you try to teach something. 00:08:58.180 --> 00:09:01.520 It's a really valuable way to just get deeper and deeper into it. 00:09:01.520 --> 00:09:02.800 And this is at Stanford, right? 00:09:02.800 --> 00:09:03.700 Yeah, it was Stanford. 00:09:03.700 --> 00:09:06.860 I think it's not so like learning, but that you realize what you don't know. 00:09:06.860 --> 00:09:09.980 Because, you know, sometimes you think that you know something and then you start explaining 00:09:09.980 --> 00:09:13.460 to people it was like, you have no idea how it goes. 00:09:13.460 --> 00:09:14.260 Yeah, for sure. 00:09:14.260 --> 00:09:17.980 The way that I think about it is like if you were, say, a consultant at a company and you 00:09:17.980 --> 00:09:21.680 had a programming problem to solve, like let's say you need to do something with multi-threading, 00:09:21.680 --> 00:09:22.180 right? 00:09:22.180 --> 00:09:24.560 If you find one way that works, you're done. 00:09:24.560 --> 00:09:25.920 You move on to the next thing. 00:09:25.920 --> 00:09:26.400 Yeah. 00:09:26.400 --> 00:09:30.220 But if there's three ways you could have done it as a teacher, you have to know what the 00:09:30.220 --> 00:09:31.000 three ways are. 00:09:31.000 --> 00:09:34.120 When should you use one versus the other? 00:09:34.240 --> 00:09:34.840 What are the three? 00:09:34.840 --> 00:09:38.380 These are questions that just a lot of times you don't have time or energy to dig into. 00:09:38.380 --> 00:09:40.820 But once you start teaching, you're like, well, I better know it because they're going to 00:09:40.820 --> 00:09:41.240 ask me. 00:09:41.240 --> 00:09:42.880 There's more ways than one. 00:09:42.880 --> 00:09:43.800 How do I do it? 00:09:43.800 --> 00:09:44.300 And why? 00:09:44.300 --> 00:09:44.660 Right. 00:09:44.660 --> 00:09:49.080 Like it really just makes you give it like this other perspective on trying to learn about 00:09:49.080 --> 00:09:49.360 stuff. 00:09:49.360 --> 00:09:49.800 Yeah. 00:09:49.800 --> 00:09:52.440 No, I think there must be like some like teaching rule. 00:09:52.440 --> 00:09:55.380 If there's a question you don't want to answer, students don't ask it. 00:09:55.380 --> 00:09:56.960 Oh, they can like use. 00:09:56.960 --> 00:09:57.420 Yes. 00:09:57.420 --> 00:10:00.380 I remember from teaching math classes as well. 00:10:00.380 --> 00:10:03.940 Like, you know that they're just going to hone right in on that one thing that you were 00:10:03.940 --> 00:10:04.240 afraid. 00:10:04.240 --> 00:10:05.600 Please don't ask me this thing. 00:10:05.600 --> 00:10:06.060 I don't totally. 00:10:06.060 --> 00:10:07.640 They can smell fear. 00:10:07.640 --> 00:10:09.180 Exactly. 00:10:09.180 --> 00:10:10.240 Yeah, exactly. 00:10:10.240 --> 00:10:10.880 All right. 00:10:10.880 --> 00:10:16.020 So you are doing teaching at Stanford right now, but you're also working at a new startup, 00:10:16.020 --> 00:10:16.240 right? 00:10:16.240 --> 00:10:17.000 So what are you doing today? 00:10:17.000 --> 00:10:18.880 That is a great question. 00:10:18.880 --> 00:10:22.040 I have been asking myself a question ever since I joined a startup. 00:10:22.040 --> 00:10:26.260 So I think startup life is great. 00:10:26.260 --> 00:10:27.620 I think it's so dynamic. 00:10:28.120 --> 00:10:29.940 We have been growing so much. 00:10:29.940 --> 00:10:34.340 The company has increased in size and multiple times since I joined in December. 00:10:34.340 --> 00:10:36.260 So it's been like less than a year. 00:10:36.260 --> 00:10:37.620 So it's a blessing. 00:10:37.620 --> 00:10:43.240 I think when I joined the Snorkel AI, I told the founding team that I'm looking for environment 00:10:43.240 --> 00:10:48.020 where I can learn like different aspects of business because eventually I want to start 00:10:48.020 --> 00:10:48.660 my own company. 00:10:48.820 --> 00:10:50.320 And they've been extremely supportive. 00:10:50.320 --> 00:10:55.880 And so in the beginning, before we launched, we were like very much heads down building the 00:10:55.880 --> 00:10:56.260 platform. 00:10:56.260 --> 00:11:01.420 So my job was like entirely on the engineering side, like building out the modeling service 00:11:01.420 --> 00:11:02.820 and like other features. 00:11:03.120 --> 00:11:07.860 And then there's a company launch and we suddenly like had a lot of interest from people. 00:11:07.860 --> 00:11:09.320 Like I was humble brag. 00:11:09.320 --> 00:11:10.300 We had two people. 00:11:10.300 --> 00:11:11.420 We don't have time to talk to them. 00:11:11.420 --> 00:11:12.820 No. 00:11:12.820 --> 00:11:14.220 So we still had a lot of interest. 00:11:14.220 --> 00:11:19.660 So I think we needed more people to like responding, like just talk to potential customers. 00:11:19.660 --> 00:11:23.340 So I have been spending more and more time on that side. 00:11:23.340 --> 00:11:24.200 So yeah. 00:11:24.200 --> 00:11:30.300 So recently I just decided to switch like most of the time on the go-to-market side. 00:11:30.300 --> 00:11:34.000 You said your ultimate goal in the long run is to do something on your own potentially. 00:11:34.000 --> 00:11:38.660 And those two things you talked about, those are the two really hard aspects of starting 00:11:38.660 --> 00:11:42.320 like one, the technical side, because you have to sort of bootstrap it and get it going. 00:11:42.320 --> 00:11:47.020 But the other is like marketing and get the word out, positioning, like all that stuff just 00:11:47.020 --> 00:11:51.820 so often on the technical side just gets ignored until you build it and no one comes. 00:11:51.820 --> 00:11:53.440 You're like, all right, well now what are we going to do? 00:11:53.440 --> 00:11:53.660 Right? 00:11:53.660 --> 00:11:54.940 Yeah. 00:11:54.940 --> 00:11:59.120 I think it's definitely, they are both like really different, difficult aspects. 00:11:59.240 --> 00:12:01.180 I think like another aspect is like recruiting. 00:12:01.180 --> 00:12:06.880 I think like, I think companies can like, like good people can like making a bet higher 00:12:06.880 --> 00:12:09.360 can basically like bankrupt the company early on. 00:12:09.360 --> 00:12:14.600 And so, so I think like one reason why I joined the company, this company I'm with is that, 00:12:14.600 --> 00:12:18.520 so I look at the teams like, wow, how did you manage to like convince like great people? 00:12:18.520 --> 00:12:21.780 And I'm saying, it's like when I said how they managed to convince great people, I mean, 00:12:21.780 --> 00:12:23.780 how does it manage to convince me to join them? 00:12:23.780 --> 00:12:24.220 I'm just kidding. 00:12:24.740 --> 00:12:27.540 I feel like everyone on the team is like, it's pretty great. 00:12:27.540 --> 00:12:29.680 But I feel like it's with a strong team. 00:12:29.680 --> 00:12:29.940 Yeah. 00:12:30.180 --> 00:12:34.160 So I think what I learned about startup life is that like, you don't just stick to an 00:12:34.160 --> 00:12:37.920 idea, like from beginning to end, you try out different things and try to pivot. 00:12:37.920 --> 00:12:40.820 You try to like, in the beginning you have a hypothesis, right? 00:12:40.820 --> 00:12:44.660 You think that like, this might be something people want, but you don't know for sure until 00:12:44.660 --> 00:12:47.000 you're actually like working with like customers. 00:12:47.360 --> 00:12:51.180 So over time you learn different things, you change your ideas or like maybe there's some 00:12:51.180 --> 00:12:55.020 like giant company launching the exact same thing and like, how do you compete with that? 00:12:55.020 --> 00:12:55.360 Yeah. 00:12:55.360 --> 00:12:59.600 So sometimes it's not about the ideas, not even about the product, but it's about the people 00:12:59.600 --> 00:13:04.400 because like with a strong team, even if you throw out some existing product and it be a 00:13:04.400 --> 00:13:06.940 new product, it can still has a chance of like competing. 00:13:06.940 --> 00:13:11.580 But if you have a bad team and it's a current idea proof should be like wrong, that you can't, 00:13:11.580 --> 00:13:13.080 you can't really recover from it. 00:13:13.080 --> 00:13:13.540 Yeah. 00:13:13.540 --> 00:13:17.260 One of the things I think is really interesting about working with small, in a small company 00:13:17.260 --> 00:13:21.000 like a startup is you get exposure to so many different things, right? 00:13:21.000 --> 00:13:25.920 You're not just the person that does billing in this way or build that part of like this 00:13:25.920 --> 00:13:26.620 pipeline. 00:13:26.620 --> 00:13:29.400 Like you have to really get your hands into many parts. 00:13:29.400 --> 00:13:32.860 It's stressful, but also I think you grow a lot if you get that opportunity. 00:13:32.860 --> 00:13:33.420 Yeah. 00:13:33.420 --> 00:13:37.220 I think, I think it's the discussion people have been talking about like the difference between 00:13:37.220 --> 00:13:39.860 looking at a big company and a small company, right? 00:13:39.860 --> 00:13:44.400 So like at big companies, you have to, you can't, you're allowed to like focus on one 00:13:44.400 --> 00:13:48.720 small thing and go really deep into it and you spend like many, many of the waking hours 00:13:48.720 --> 00:13:49.240 on it. 00:13:49.240 --> 00:13:53.580 But as startups, like yeah, many things going on and you have to like maybe like cycle among 00:13:53.580 --> 00:13:54.240 them like quickly. 00:13:54.240 --> 00:13:59.440 So let's say like at big companies, like big companies can afford to hire specialists who 00:13:59.440 --> 00:14:01.860 can, who can do one small things really well. 00:14:01.860 --> 00:14:06.100 But startups, like a lot of, they might want somebody who can do a lot of things. 00:14:06.100 --> 00:14:08.320 Okay-ish, like not like expert. 00:14:09.000 --> 00:14:09.240 Yeah. 00:14:09.240 --> 00:14:09.640 Yeah. 00:14:09.640 --> 00:14:14.900 A lot of prototypes, do it quickly, try it out, then work, do something else, find a 00:14:14.900 --> 00:14:16.620 gap, fill that hole, all that kind of stuff, right? 00:14:16.620 --> 00:14:17.020 Yeah. 00:14:17.020 --> 00:14:17.920 I think it's good. 00:14:17.920 --> 00:14:20.200 Like really depends on like the faces of life. 00:14:20.200 --> 00:14:24.200 I know there are people who, who really just want to like keep their heads down and focus 00:14:24.200 --> 00:14:25.100 on one thing. 00:14:25.100 --> 00:14:27.480 So, I mean, there's no, there's no shame. 00:14:27.480 --> 00:14:32.220 I think I have so much respect for people who can do it as I have so much respect for people 00:14:32.220 --> 00:14:36.240 who can like adapt quickly and learn things quickly and just like build things. 00:14:36.240 --> 00:14:36.820 Yeah. 00:14:36.920 --> 00:14:38.320 You got to find the one that works for you. 00:14:38.320 --> 00:14:42.880 So one of the things that you've been writing a lot about, you're working on a book actually, 00:14:42.880 --> 00:14:48.840 is basically about building teams in the ML space and hiring people. 00:14:48.840 --> 00:14:53.220 And I think that this is a big challenge right now because so much of the folks in the data 00:14:53.220 --> 00:14:57.200 science space is so hot and so many people are coming from different areas, right? 00:14:57.200 --> 00:15:03.000 Like there might be somebody doing ML, but three years ago that person was, I don't know, 00:15:03.000 --> 00:15:05.520 working in finance or maybe another person. 00:15:05.520 --> 00:15:09.660 She was like a biologist, but she got into programming and now she's doing machine learning 00:15:09.660 --> 00:15:10.800 because she's sort of right. 00:15:10.800 --> 00:15:16.460 So it's, I think it's actually a little bit challenging to hire people in this space because 00:15:16.460 --> 00:15:20.720 it's not just, well, show me your machine learning PhD and we'll talk about it. 00:15:20.720 --> 00:15:20.920 Right. 00:15:20.960 --> 00:15:24.140 Like there's probably not that many people in that realm, right? 00:15:24.140 --> 00:15:27.760 Like there's not as much traditional education in the workforce yet. 00:15:27.760 --> 00:15:28.200 Yeah. 00:15:28.200 --> 00:15:35.340 So I think I agree with you that hiring is hard, but I think that hiring is hard right 00:15:35.340 --> 00:15:37.960 now for machine learning for many reasons. 00:15:37.960 --> 00:15:42.560 I think the first reason is like, it's probably because companies don't even know what they 00:15:42.560 --> 00:15:43.620 are hiring for yet. 00:15:43.620 --> 00:15:46.400 I think because machine learning is like, it's really new. 00:15:46.400 --> 00:15:50.780 And if you're like imagining a company, like you have never deployed a machine learning model 00:15:50.780 --> 00:15:51.240 before. 00:15:51.240 --> 00:15:53.460 And now you're trying to start a new team. 00:15:53.460 --> 00:15:56.760 So you're probably like, what do you need to build a machine learning model? 00:15:56.760 --> 00:15:58.120 And you have no idea. 00:15:58.120 --> 00:16:01.080 So you probably come up with some very generic ways. 00:16:01.080 --> 00:16:05.700 And like you, like you say somebody who's like doing like state of the art research, somebody 00:16:05.700 --> 00:16:07.140 who can code really well. 00:16:07.140 --> 00:16:07.920 Yeah. 00:16:07.920 --> 00:16:09.880 Somebody who can explain what they are doing. 00:16:09.880 --> 00:16:12.180 So it's just like, these people just don't exist. 00:16:12.180 --> 00:16:12.860 Yeah. 00:16:12.860 --> 00:16:13.420 That's a good point. 00:16:13.420 --> 00:16:18.380 But I think the second thing is that machine learning itself is not new, but machine learning 00:16:18.380 --> 00:16:23.240 in productions, like especially from the explosion of like deep learning CNA 2012. 00:16:23.240 --> 00:16:28.560 I think the first major application of deep learning in industry is probably Google Translate 00:16:28.560 --> 00:16:29.980 in 2016. 00:16:29.980 --> 00:16:32.940 And since then, a lot of companies have been looking into it. 00:16:32.940 --> 00:16:38.140 So it means that, so I think like industry is like lagging behind like research, like a 00:16:38.140 --> 00:16:38.580 few years. 00:16:38.580 --> 00:16:38.800 Right. 00:16:38.860 --> 00:16:43.220 So like research like grows and you have a lot of people knowing like how to do machine 00:16:43.220 --> 00:16:44.500 in academic environment. 00:16:44.500 --> 00:16:48.820 And then like industry say, oh wait, you can actually use that like to improve our business. 00:16:48.820 --> 00:16:49.620 So let's do it. 00:16:49.620 --> 00:16:53.920 So like at that time, like, so we're in the phase when companies are looking into it. 00:16:54.240 --> 00:16:57.600 But most people who know machine learning comes from an academic environment. 00:16:57.600 --> 00:17:02.200 So they are familiar with like how to like do machine learning research, but they actually 00:17:02.200 --> 00:17:04.940 might not be like familiar with like doing machine in productions. 00:17:04.940 --> 00:17:08.800 And there are not many people who can teach them because usually you need a hands-on experience. 00:17:08.800 --> 00:17:12.840 So we have a very early phase of machine adoption in the industry. 00:17:12.840 --> 00:17:15.120 And I think like, so that's why we are liking people. 00:17:15.120 --> 00:17:17.460 But I think in a few years, we will have a lot more. 00:17:17.460 --> 00:17:23.300 And hopefully like understanding of machine in productions plus availability of people 00:17:23.300 --> 00:17:28.460 with actually hands-on experience will make hiring less, a lot less difficult for companies. 00:17:28.460 --> 00:17:28.900 Yeah. 00:17:28.900 --> 00:17:29.880 That's really interesting. 00:17:29.880 --> 00:17:35.320 I can imagine if I was hiring, say like a couple more web developers or somebody to do 00:17:35.320 --> 00:17:38.640 database ETL, like bring in the data and clean it up. 00:17:38.640 --> 00:17:39.080 Yeah. 00:17:39.080 --> 00:17:43.600 You already have people in your organization who do that and you can say, well, what do you 00:17:43.600 --> 00:17:43.960 need? 00:17:43.960 --> 00:17:45.560 And please talk to this person. 00:17:45.560 --> 00:17:49.820 But if you're creating an ML team, like I know there's really large companies out there 00:17:49.820 --> 00:17:54.200 that don't have a single person who's doing like productionized machine learning. 00:17:54.200 --> 00:17:57.140 So it's like you pointed out, like it's starting from zero. 00:17:57.140 --> 00:18:02.040 And a lot of times, you know, is that person who you're talking to really competent to make 00:18:02.040 --> 00:18:03.960 that decision or make the right trade-offs, right? 00:18:03.960 --> 00:18:05.840 Or even know what you're hiring the person for. 00:18:05.840 --> 00:18:06.280 Yeah. 00:18:06.280 --> 00:18:06.980 So yeah. 00:18:06.980 --> 00:18:13.680 So you hire the person for and also like having people to like evaluate the skills can be very, 00:18:13.680 --> 00:18:14.340 very hard. 00:18:14.340 --> 00:18:19.640 And so, so I think I say a lot of, so, so, but actually I do see some shift in the future. 00:18:19.640 --> 00:18:23.620 So I think like a lot of aspects of machine learning are being commoditized. 00:18:23.620 --> 00:18:27.260 So for example, like you're seeing a lot of pre-trained models, right? 00:18:27.260 --> 00:18:30.240 And people is like trained the model for you already and they open source it. 00:18:30.240 --> 00:18:33.660 And you have a lot of like pre-built model, like hugging face. 00:18:33.660 --> 00:18:38.740 So, so you can just like code API and then you can incorporate like some machine model in those systems. 00:18:38.960 --> 00:18:45.440 So actually like, so like a lot of tools to allow you like future engineering or like runes-based systems, 00:18:45.440 --> 00:18:47.840 like monitoring tools and deployment tools. 00:18:47.840 --> 00:18:54.480 So, so I do believe that there's a bottleneck for machine learning in production now will be in the engineering part. 00:18:54.820 --> 00:18:56.620 So I'm not saying that we stop doing research. 00:18:56.620 --> 00:19:00.520 I'm not saying that like a lot less companies will do research. 00:19:00.520 --> 00:19:07.320 I think that doing like the machine part can be like a few very large established company who know what they are doing. 00:19:07.480 --> 00:19:13.580 And then a lot of other companies who use machine learning can just like use like existing tools and platforms. 00:19:13.580 --> 00:19:18.040 So the challenges can be like engineering challenges and not machine learning challenges. 00:19:18.040 --> 00:19:18.480 Yeah. 00:19:18.480 --> 00:19:20.840 There's a lot of stuff that's getting pre-built out there. 00:19:20.840 --> 00:19:24.960 Like I think Apple ships with some pre-trained models for running on iOS. 00:19:24.960 --> 00:19:28.160 You've got like Azure cognitive services, stuff like that, right? 00:19:28.160 --> 00:19:30.220 Where you just, you kind of just bring that in. 00:19:30.220 --> 00:19:34.900 This portion of Talk Python To Me is brought to you by Datadog. 00:19:34.900 --> 00:19:39.540 Are you having trouble visualizing latency and CPU or memory bottlenecks in your app? 00:19:39.540 --> 00:19:42.420 Not sure where the issue is coming from or how to solve it? 00:19:42.420 --> 00:19:47.300 Datadog seamlessly correlates logs and traces at the level of individual requests, 00:19:47.300 --> 00:19:50.200 allowing you to quickly troubleshoot your Python application. 00:19:50.200 --> 00:19:55.680 Plus, their continuous profiler allows you to find the most resource consuming parts of your production code 00:19:55.680 --> 00:19:58.560 all the time at any scale with minimal overhead. 00:19:58.560 --> 00:20:01.580 Be the hero that got that app back on track at your company. 00:20:01.580 --> 00:20:06.300 Get started today with a free trial at talkpython.fm/Datadog 00:20:06.300 --> 00:20:08.980 or just click the link in your podcast player's show notes. 00:20:08.980 --> 00:20:11.540 Get the insight you've been missing with Datadog. 00:20:11.540 --> 00:20:16.500 So I guess one question, just thinking, sort of reversing it a little bit, 00:20:16.500 --> 00:20:17.680 I guess you could see it from both ways. 00:20:17.680 --> 00:20:20.920 If I was somebody who was looking for a machine learning job 00:20:20.920 --> 00:20:22.420 or I was hiring somebody, 00:20:22.420 --> 00:20:25.900 how much do you think that engineering side should matter? 00:20:25.980 --> 00:20:27.020 It sounds like it's pretty important. 00:20:27.020 --> 00:20:31.260 So should, say, being competent with Git and source control be important? 00:20:31.260 --> 00:20:31.880 Yeah. 00:20:31.880 --> 00:20:32.780 Continuous integration. 00:20:32.780 --> 00:20:36.680 Should you know something like FastAPI or Flask or something like that 00:20:36.680 --> 00:20:38.460 to build a service around your model? 00:20:38.460 --> 00:20:38.860 Yeah. 00:20:38.860 --> 00:20:41.400 What are the skills you think are really important there? 00:20:41.400 --> 00:20:45.160 I think it really depends on what type of jobs do you want. 00:20:45.320 --> 00:20:49.720 So I think Zezia, like, I think I want to say the term traditional machine learning 00:20:49.720 --> 00:20:53.200 engineering job, but I don't think machine learning engineering is that old 00:20:53.200 --> 00:20:55.040 to deserve the term traditional. 00:20:55.040 --> 00:20:58.420 But I think that people have been using the last few machine engineering 00:20:58.420 --> 00:21:01.560 as in, like, future engineering, creating models, 00:21:01.560 --> 00:21:03.480 training models, like babysitting models. 00:21:03.480 --> 00:21:05.840 And I think that part requires quite a lot of machine learning 00:21:05.840 --> 00:21:06.880 and less engineering. 00:21:07.060 --> 00:21:10.440 But, like, if you work to, like, set up a distributed pipeline, 00:21:10.440 --> 00:21:14.680 like how should you do, like, how can you process data in a parallel? 00:21:14.680 --> 00:21:16.900 How can you print model? 00:21:16.900 --> 00:21:19.780 How can you deploy model so that it can serve, like, 00:21:19.780 --> 00:21:22.280 a lot of requests at the same time but lower agencies? 00:21:22.280 --> 00:21:26.880 Then you probably need more systems and databases and machine learning. 00:21:27.200 --> 00:21:31.580 And if you are in the part when you want to, like, monitor the system, 00:21:31.580 --> 00:21:33.480 like, the maintenance and, like, monitoring, 00:21:33.480 --> 00:21:36.940 so you can, like, how do you, like, push updates without, like, 00:21:36.940 --> 00:21:38.060 interrupting the service? 00:21:38.060 --> 00:21:40.960 Or how do you, if something happens, like, 00:21:40.960 --> 00:21:43.280 how can you be alerted when some bad things happen 00:21:43.280 --> 00:21:46.440 and then you can address it quickly or how can you run back the system? 00:21:46.440 --> 00:21:49.180 Then I think a lot of it has, like, it's very similar to DevOps. 00:21:49.180 --> 00:21:51.760 So you need a lot more things as well. 00:21:51.760 --> 00:21:52.220 Right. 00:21:52.220 --> 00:21:55.940 So it really depends on what roles you want, the company you want, 00:21:56.040 --> 00:21:58.080 because I think one thing I noticed is that, like, 00:21:58.080 --> 00:22:01.760 companies have very different structure for their machine learning teams. 00:22:01.760 --> 00:22:05.380 So companies like, for example, Netflix, 00:22:05.380 --> 00:22:08.640 so they have this, like, separate, like, algorithm team. 00:22:08.640 --> 00:22:11.520 They focus on the aspect, algorithmic aspect of machine learning, right? 00:22:11.520 --> 00:22:12.940 But so, so... 00:22:12.940 --> 00:22:16.820 Yeah, the recommender engine is so important over Netflix, right? 00:22:16.820 --> 00:22:18.160 I mean, that's so central. 00:22:18.160 --> 00:22:18.480 Yeah. 00:22:18.480 --> 00:22:21.580 They even had that million-dollar prize to see who could recommend... 00:22:21.580 --> 00:22:22.720 Yeah, they do. 00:22:22.720 --> 00:22:24.500 ...movies you should watch next best, right? 00:22:24.500 --> 00:22:29.440 Yeah, but it's just really funny because, like, I think, like, these competitions are great, 00:22:29.440 --> 00:22:33.120 but, like, the result is it very hard to actually, like, be deployed. 00:22:33.120 --> 00:22:37.680 So, so I think, like, I'm not saying Netflix is not using the winning result. 00:22:37.680 --> 00:22:42.160 I'm just saying that, like, for a lot of these competitions, like, the winning solutions are... 00:22:42.160 --> 00:22:44.760 Even though the winning solutions perform well on the little board, 00:22:44.760 --> 00:22:49.620 they tend to, like, be very hard to be deployed because the solution is way too complex 00:22:49.620 --> 00:22:52.080 to be reliably deployable. 00:22:52.080 --> 00:22:52.480 Yes. 00:22:52.480 --> 00:22:53.240 Oh, interesting. 00:22:53.240 --> 00:22:53.820 Yeah. 00:22:53.820 --> 00:22:57.880 Or maybe it's over-trained exactly on that one thing, and it's perfect at that, but it's 00:22:57.880 --> 00:22:59.240 not generalized enough or something. 00:22:59.240 --> 00:23:01.060 Yes, that's one thing. 00:23:01.060 --> 00:23:05.080 I think that's one thing we have been talking about, like, how just start a competition, 00:23:05.080 --> 00:23:10.440 like, leaderboard-driven oriented work is actually not very much close to real life. 00:23:10.560 --> 00:23:14.360 Because when you have a leaderboard, right, you tend to have one single objective you 00:23:14.360 --> 00:23:15.240 work toward. 00:23:15.240 --> 00:23:18.920 For, in this case, you know, like, how good a model with the best performance. 00:23:18.920 --> 00:23:22.060 But whereas in productions, you don't have one single objective. 00:23:22.060 --> 00:23:25.980 Like, you might have different stakeholders in the company, and they help as one different 00:23:25.980 --> 00:23:26.440 things. 00:23:26.440 --> 00:23:29.920 Like, one person might want, like, hey, we want the best performance. 00:23:29.920 --> 00:23:32.060 But then it's like, hey, we want the lowest legacies. 00:23:32.060 --> 00:23:36.180 And that is like, hey, how do we can, like, do it in a way that we can show the most ads 00:23:36.180 --> 00:23:37.140 without being obnoxious? 00:23:37.840 --> 00:23:39.180 So there's a lot of things. 00:23:39.180 --> 00:23:43.700 And sometimes you just optimize for one thing, like, you can't really go for other. 00:23:43.700 --> 00:23:49.200 I think it's some interesting example of how, like, a machining model that can do very well 00:23:49.200 --> 00:23:52.000 on leaderboards that's, like, not going to be useful in real life. 00:23:52.000 --> 00:23:52.960 So think about that. 00:23:52.960 --> 00:23:58.740 Do you remember that, I think about 10 years ago, it was like, so these giant retail companies 00:23:58.740 --> 00:24:03.260 who have been, like, trying to, like, predict whether someone is pregnant, so that they can, 00:24:03.260 --> 00:24:05.400 like, advertise directly to that, right? 00:24:05.400 --> 00:24:06.580 Yes, yes. 00:24:06.820 --> 00:24:11.580 So, and, like, someone found out, and then they sent, like, all the baby products to, 00:24:11.580 --> 00:24:14.000 like, this teenage girl, to her family. 00:24:14.000 --> 00:24:17.040 And, like, they didn't know about it yet, and now they suddenly know about it. 00:24:17.040 --> 00:24:18.760 So that's an example of, like... 00:24:18.760 --> 00:24:19.840 Yeah, they got really angry. 00:24:19.840 --> 00:24:21.180 Like, why are you sending my daughter this? 00:24:21.180 --> 00:24:23.140 And it turns out actually she was pregnant, right? 00:24:23.140 --> 00:24:24.060 Oh, my gosh. 00:24:24.060 --> 00:24:24.700 That's... 00:24:24.700 --> 00:24:24.960 Yeah. 00:24:24.960 --> 00:24:26.020 That's not so good for her. 00:24:26.020 --> 00:24:27.060 No, no, it's not. 00:24:27.060 --> 00:24:30.280 So that's an example of, like, it can be so good, it's creepy. 00:24:30.280 --> 00:24:32.380 And you don't want that. 00:24:32.380 --> 00:24:32.920 Or... 00:24:32.920 --> 00:24:33.140 Yeah. 00:24:33.140 --> 00:24:37.580 I think about, like, how we have a machining model that's, like, that can help the users 00:24:37.580 --> 00:24:39.840 to, like, solve their problem really well. 00:24:39.840 --> 00:24:41.580 So there are two things that can happen here. 00:24:41.580 --> 00:24:45.260 Like, one is that, like, it solves the problem so well, that the user is just done. 00:24:45.260 --> 00:24:48.200 They never have to come back to you ever again, and you just lose business. 00:24:48.200 --> 00:24:52.300 Or they, like, they solve the problem so well, that the user just loves the system, they keep 00:24:52.300 --> 00:24:53.120 coming back for more. 00:24:53.360 --> 00:24:57.640 So, like, it's really hard to find the linear relationship between the model performance 00:24:57.640 --> 00:24:58.880 and business performance. 00:24:58.880 --> 00:24:59.540 Yeah. 00:24:59.540 --> 00:25:04.480 I did really think about the relationship of these, like, competition winning algorithms 00:25:04.480 --> 00:25:05.400 and models and stuff. 00:25:05.400 --> 00:25:09.040 But, yeah, that makes a lot of sense that just because it solves that one problem, it might 00:25:09.040 --> 00:25:13.240 not be practical to run in production or to maintain or whatever and evolve it. 00:25:13.240 --> 00:25:13.640 Yeah. 00:25:13.640 --> 00:25:15.020 I think you need to, like... 00:25:15.020 --> 00:25:19.400 So I think that's why it's important for people who, like, who are in charge to, like, 00:25:19.400 --> 00:25:24.900 give a good sense of what they want and how to, like, balance between different objectives 00:25:24.900 --> 00:25:27.640 of different stakeholders in a project. 00:25:27.640 --> 00:25:28.140 Yeah. 00:25:28.140 --> 00:25:29.660 So give me... 00:25:29.660 --> 00:25:31.420 Put the hiring hat on for a minute. 00:25:31.420 --> 00:25:36.340 And if you're at a company that does not yet have an internal in-house machine learning 00:25:36.340 --> 00:25:41.380 team, but you think maybe you want to, maybe we can analyze all this data we have and we 00:25:41.380 --> 00:25:43.520 can find some trends and do more interesting stuff. 00:25:43.520 --> 00:25:43.920 Yeah. 00:25:43.920 --> 00:25:45.840 And you want to create an in-house ML team. 00:25:45.840 --> 00:25:47.480 Like, what advice do you have for those people? 00:25:47.480 --> 00:25:47.960 Okay. 00:25:47.960 --> 00:25:49.720 So it really depends on who you are. 00:25:49.720 --> 00:25:52.740 Like, you know, like, Walmart or, like, McDonald's, right? 00:25:52.740 --> 00:25:57.420 You just want to acquire, like, a very promising ML startup and just have an in-house team. 00:25:57.820 --> 00:26:01.380 And I think a lot of the big companies are, like, going for that approach. 00:26:01.380 --> 00:26:01.840 Yeah. 00:26:01.840 --> 00:26:06.280 I think another approach is that I think a lot of companies are doing is, like, to transitioning 00:26:06.280 --> 00:26:06.940 to ML. 00:26:06.940 --> 00:26:10.060 You might want to, like, use some existing talent in the company. 00:26:10.060 --> 00:26:13.360 So machining is new, but data science is not. 00:26:13.360 --> 00:26:17.060 So I think data science people, teams have been, like, a company has been having data science 00:26:17.060 --> 00:26:18.300 team for the long run. 00:26:18.300 --> 00:26:22.700 And data science teams also, like, work with data and they do a lot of it. 00:26:22.700 --> 00:26:27.000 They probably already have access to data and they also try to get, like, patterns from 00:26:27.000 --> 00:26:27.320 data. 00:26:27.460 --> 00:26:32.380 So I think a lot of teams, like, in the beginning, they transition, like, use data science team 00:26:32.380 --> 00:26:36.800 as, like, hey, why don't you learn machine learning and, like, try these things out? 00:26:36.800 --> 00:26:41.320 Maybe a couple of you could learn PyTorch and work on this project and get started. 00:26:41.320 --> 00:26:45.640 You might joke about it, but I think it's pretty much how people do it. 00:26:45.640 --> 00:26:49.400 And I think I see a lot of people in data science transition into machine learning. 00:26:49.400 --> 00:26:54.000 And I think especially now with abundance of machine learning courses, there's just so many 00:26:54.000 --> 00:26:55.560 courses online for free. 00:26:55.560 --> 00:26:56.040 Yeah. 00:26:56.040 --> 00:26:58.460 I think it's great that people are taking advantage of it. 00:26:58.460 --> 00:27:02.800 I was looking up, like, so courses, like, do you know Android machine learning course? 00:27:02.800 --> 00:27:03.660 Yeah. 00:27:03.660 --> 00:27:04.980 I haven't taken it, but I've heard of it. 00:27:04.980 --> 00:27:05.220 Yeah. 00:27:05.480 --> 00:27:09.160 I think it has, like, more than 2 million people who have taken the course already. 00:27:09.160 --> 00:27:10.220 2 million students, right? 00:27:10.220 --> 00:27:10.440 Yeah. 00:27:10.440 --> 00:27:10.880 Yeah. 00:27:10.880 --> 00:27:11.400 Crazy. 00:27:11.400 --> 00:27:11.800 Yeah. 00:27:11.800 --> 00:27:12.160 Yeah. 00:27:12.160 --> 00:27:13.100 And it's pretty new, right? 00:27:13.100 --> 00:27:14.000 I think it's pretty new. 00:27:14.000 --> 00:27:14.240 Yeah. 00:27:14.240 --> 00:27:15.040 That's really crazy. 00:27:15.380 --> 00:27:15.560 Yeah. 00:27:15.560 --> 00:27:18.080 It's new compared to other disciplines. 00:27:18.080 --> 00:27:21.360 But I think, like, in machine learning, it's, like, one of the older courses. 00:27:21.360 --> 00:27:23.860 So I think a lot of teams do that. 00:27:23.860 --> 00:27:29.000 And I think, but I think, like, for companies who do that, I think one, too, like, hopes that 00:27:29.000 --> 00:27:32.720 they just, like, to look into the difference between data science and machine learning. 00:27:32.720 --> 00:27:38.680 So data science is, like, to look at data, like, the output, like, insight, like, to help 00:27:38.680 --> 00:27:40.000 make decisions about business. 00:27:40.000 --> 00:27:44.940 For us, you can predict the, like, how much the customer demand in the future or, like, 00:27:44.940 --> 00:27:45.140 yeah. 00:27:45.140 --> 00:27:50.140 But machine learning is, like, the goal is to have, like, to build product, to be, like, 00:27:50.140 --> 00:27:50.600 engineering. 00:27:50.600 --> 00:27:55.580 So for data science, like, you want people with stronger statistics skills because you look 00:27:55.580 --> 00:27:56.700 at the data and get insight. 00:27:56.700 --> 00:27:59.400 But for machine learning, it's more engineering. 00:27:59.400 --> 00:28:03.460 So you want somebody with, like, stronger engineering skill and less that, yeah. 00:28:03.460 --> 00:28:04.060 Right. 00:28:04.060 --> 00:28:09.660 So as a data scientist, maybe your output might be, here's a Jupyter notebook with a Plotly 00:28:09.660 --> 00:28:12.500 analysis of what we're thinking. 00:28:12.500 --> 00:28:17.060 Whereas as a machine learning person, your output is, here's the API that gives you the answer. 00:28:17.060 --> 00:28:19.060 Yes, you can think of it that way. 00:28:19.060 --> 00:28:19.700 Something like that? 00:28:19.700 --> 00:28:20.280 Yes, yes. 00:28:20.280 --> 00:28:21.280 Something like that, yeah. 00:28:21.280 --> 00:28:21.560 Okay. 00:28:21.560 --> 00:28:24.000 So I think this is a very different focus. 00:28:24.000 --> 00:28:27.300 I'm not saying just, like, I'm not trying to make a general statement here. 00:28:27.300 --> 00:28:31.080 I'm just saying just, like, just from talking to a lot of people, I tend to notice that, 00:28:31.080 --> 00:28:35.300 like, data scientists are seeing much better statistics, whereas, like, machine engineers 00:28:35.300 --> 00:28:36.280 are much better engineers. 00:28:36.280 --> 00:28:36.820 Yeah. 00:28:36.820 --> 00:28:39.200 Yeah, I've seen that as well in some of the trends. 00:28:39.200 --> 00:28:39.440 So. 00:28:39.440 --> 00:28:39.940 Yeah. 00:28:39.940 --> 00:28:41.200 It seems totally reasonable. 00:28:41.200 --> 00:28:43.200 Let's reverse this a little bit. 00:28:43.200 --> 00:28:45.100 So we were talking about if you want to build a team. 00:28:45.100 --> 00:28:48.860 And you did point out, by the way, bringing someone in from the inside. 00:28:48.860 --> 00:28:56.260 Like, I feel like data science, more than software developer, that role needs to be sort of intimately 00:28:56.260 --> 00:29:01.280 familiar with the way that the business works and the way the data is collected and all the 00:29:01.280 --> 00:29:02.900 little idiosyncrasies around it. 00:29:02.900 --> 00:29:06.140 And so having somebody who already knows all that stuff, and now you're just like, okay. 00:29:06.140 --> 00:29:06.540 Yeah. 00:29:06.540 --> 00:29:11.040 Adapt that to machine learning might be easier than getting somebody who's good, but has no 00:29:11.040 --> 00:29:12.120 experience in the business. 00:29:12.200 --> 00:29:16.000 I think, like, I make a living out of, like, saying that I know machine learning, right? 00:29:16.000 --> 00:29:20.260 So, of course, I want to, like, make machine learning as hype as possible. 00:29:20.260 --> 00:29:25.040 But I have to admit that, like, machine learning for a lot of, like, simple models, you don't 00:29:25.040 --> 00:29:26.120 need to learn. 00:29:26.120 --> 00:29:31.080 You don't need to spend years and years and years of, like, learning to, like, be able to 00:29:31.080 --> 00:29:32.060 use simple models. 00:29:32.060 --> 00:29:38.000 So I think that's, like, one thing I noticed is, like, it's actually a lot easier for good 00:29:38.000 --> 00:29:39.880 engineers to, like, pick up machine learning. 00:29:40.180 --> 00:29:43.160 I've done for machine learning experts to pick up, like, good engineering. 00:29:43.160 --> 00:29:43.840 Gotcha. 00:29:43.840 --> 00:29:44.540 Yeah. 00:29:44.540 --> 00:29:49.480 So, like, if I was to start a team, I would probably try to get, like, really good engineers 00:29:49.480 --> 00:29:52.900 and have them learn machine learning and then, like, apply machine learning. 00:29:52.900 --> 00:29:56.620 Then, like, to hire machine learning experts and then, like, having them, like, spend, like, 00:29:56.620 --> 00:29:58.220 several decades to become good engineers. 00:29:58.220 --> 00:29:59.620 Yeah. 00:29:59.620 --> 00:30:01.100 That's a really good perspective. 00:30:01.100 --> 00:30:01.600 Yeah. 00:30:01.600 --> 00:30:02.080 All right. 00:30:02.080 --> 00:30:07.100 So switching the role here to being interviewed for a machine learning job. 00:30:07.420 --> 00:30:11.260 So you're working on this book for machine learning interviews. 00:30:11.260 --> 00:30:15.140 It gives you, like, a sense of sort of if you're going to go apply for one of these jobs, 00:30:15.140 --> 00:30:19.480 what are some of the skills and things you might expect to be asked about and so on, right? 00:30:19.480 --> 00:30:20.760 Want to tell us quickly about that? 00:30:20.760 --> 00:30:21.260 Yeah. 00:30:21.260 --> 00:30:25.020 So this is a book I've been working on for, like, oh, my God, a year and a half now. 00:30:25.020 --> 00:30:29.240 Do you know, like, how it has so many great plans for 2020 and none of them happened? 00:30:29.240 --> 00:30:30.800 I think this is what's the case. 00:30:30.800 --> 00:30:31.820 That's the case with my book. 00:30:31.940 --> 00:30:35.180 I think it has so much great plan for it, like, and then, like, boom. 00:30:35.180 --> 00:30:36.500 So, yeah. 00:30:36.500 --> 00:30:37.260 So it's a slow. 00:30:37.260 --> 00:30:38.140 It's coming along. 00:30:38.140 --> 00:30:43.720 So I think my book is not just a book for, like, here are the questions they're going to ask you or, like, 00:30:43.720 --> 00:30:45.200 how to answer them. 00:30:45.200 --> 00:30:51.620 I think part of what I want to do with the book is to have some standardizations or understanding into the process. 00:30:51.620 --> 00:30:53.620 I think it's new in the industry. 00:30:53.740 --> 00:30:56.940 So it's new for both interviewees and interviewers. 00:30:56.940 --> 00:31:01.980 So, for example, people still ask me, like, people would be confused, like, what is a machine engineer? 00:31:01.980 --> 00:31:03.500 What is a data scientist? 00:31:03.500 --> 00:31:06.680 Like, what's the difference between big company and small company? 00:31:06.680 --> 00:31:08.040 What is the hiring process? 00:31:08.040 --> 00:31:09.360 What skills do you need? 00:31:09.360 --> 00:31:16.300 So I think there's just so many skills that one might need, but usually, like, you don't need all of them for a single role. 00:31:16.300 --> 00:31:18.960 So I think my book is pretty, definitely start from it. 00:31:18.960 --> 00:31:20.980 It's a different, I think I lost you. 00:31:20.980 --> 00:31:22.960 Sorry, I don't know what happened to my network. 00:31:23.080 --> 00:31:25.400 It just said that it lost and, like, all my stuff disconnected. 00:31:25.400 --> 00:31:26.500 But we're back. 00:31:26.500 --> 00:31:27.560 Yeah. 00:31:27.560 --> 00:31:33.500 So we were talking about the book and you said it wasn't just for people, like, to know what the questions and answers were, 00:31:33.500 --> 00:31:38.600 but that it, like, it's such a new industry that it's both new for interviewers and interviewees. 00:31:38.600 --> 00:31:40.000 And I think we were going from there. 00:31:40.000 --> 00:31:40.520 Okay. 00:31:40.520 --> 00:31:41.080 Yeah. 00:31:41.080 --> 00:31:48.480 So part of the book, it should give some understanding, standardization, and it should be a process of differences between different type roles. 00:31:48.480 --> 00:31:49.660 Like, what is a data scientist? 00:31:49.660 --> 00:31:51.060 What is a machine engineer? 00:31:51.060 --> 00:31:52.800 Or what is a research engineer? 00:31:52.920 --> 00:31:58.240 Also, like, it's a difference between, like, for example, like, machine engineering and data science and MLOps. 00:31:58.240 --> 00:32:04.600 So it's going to, like, get a good picture of the process, what skills are needed for each process. 00:32:04.600 --> 00:32:08.100 And I see interview and interview pipeline, building pipeline. 00:32:08.100 --> 00:32:09.060 Yeah. 00:32:09.060 --> 00:32:10.220 And a lot more. 00:32:10.800 --> 00:32:16.580 Yeah, that sounds, look, we need all the help I think we can get for fixing the interview process. 00:32:16.580 --> 00:32:17.400 Oh, my God. 00:32:17.400 --> 00:32:19.200 In software development and data science. 00:32:19.200 --> 00:32:20.800 It seems so broken to me. 00:32:20.800 --> 00:32:26.900 I've had some friends who have gone through it recently, and it just seemed really, really rough. 00:32:27.260 --> 00:32:28.680 And I actually did an episode. 00:32:28.680 --> 00:32:30.960 Wait, let me do a quick search. 00:32:30.960 --> 00:32:33.760 What are some of the highlights of the, like, pinpoints? 00:32:34.080 --> 00:32:45.700 I think a lot of it is you get asked to work on, like, low-level algorithms, like, explain or create or recreate low-level algorithms, sometimes even just on a whiteboard. 00:32:45.700 --> 00:32:46.160 Yeah. 00:32:46.480 --> 00:32:48.900 Where, like, you know, go create quick sort. 00:32:48.900 --> 00:32:52.100 And then never, ever in your job will you ever go and create quick sort. 00:32:52.100 --> 00:32:53.940 Or something like that, right? 00:32:53.940 --> 00:32:54.280 Yeah. 00:32:54.280 --> 00:32:57.920 Like, you would just go to the list and say dot sort, and it would be done. 00:32:57.920 --> 00:33:03.080 I interviewed Susan Tan a while ago, back up, way, way, way, in episode 123. 00:33:03.080 --> 00:33:08.600 And she said, she did a talk called Lessons from 100 Straight Developer Job Interviews. 00:33:08.600 --> 00:33:10.840 I think she was in San Francisco as well. 00:33:10.840 --> 00:33:12.100 I'm pretty sure if I remember correctly. 00:33:12.100 --> 00:33:14.140 A hundred is so much. 00:33:14.380 --> 00:33:18.560 She literally did a hundred and then, like, took notes about what worked. 00:33:18.560 --> 00:33:19.340 Oh, my God. 00:33:19.340 --> 00:33:22.100 You know, you'll get, like, these big, like, homework projects. 00:33:22.100 --> 00:33:22.140 Yeah. 00:33:22.140 --> 00:33:22.660 Right? 00:33:22.660 --> 00:33:24.060 Like, work on this for, like, a week. 00:33:24.060 --> 00:33:25.880 And then, you know, there's a hundred applicants. 00:33:25.880 --> 00:33:29.380 So, like, the energy put into that is often not. 00:33:29.380 --> 00:33:32.980 Anyway, I think helping both sides of that story would be really good. 00:33:32.980 --> 00:33:33.400 Yeah. 00:33:33.400 --> 00:33:38.960 I would definitely love to, like, read her interview because it sounds exactly like what I've been working on. 00:33:38.960 --> 00:33:39.880 I'm curious. 00:33:39.880 --> 00:33:43.100 Does she, like, propose, like, what are some things that work? 00:33:43.420 --> 00:33:48.060 She did, and it's been, gosh, it's been, like, two or three years since I spoke to her about it. 00:33:48.060 --> 00:33:51.300 But I know she had some advice for, like, these things were really bad. 00:33:51.300 --> 00:33:53.520 And these things I experienced were really good. 00:33:53.520 --> 00:33:54.020 Yeah. 00:33:54.020 --> 00:33:58.920 And so, she basically laid out, like, what are some bad interviews I had and what are some good ones and why? 00:33:58.920 --> 00:34:01.820 And I think probably in there you could pull out some good advice. 00:34:01.820 --> 00:34:11.380 Yeah, this is, like, really interesting because I think, like, before, as I was still, like, interviewing for jobs, I was, like, I have so much to complain about the interviewing process, right? 00:34:11.380 --> 00:34:17.840 But now, as a part of, like, a startup and we're trying to build the reading pipeline, we realize that it's really hard. 00:34:17.840 --> 00:34:23.340 Like, even though we complain about the existing pipeline, it's really hard to come up with something that is better. 00:34:23.860 --> 00:34:25.880 So, I think it's just, like, too many. 00:34:25.880 --> 00:34:31.580 So, the first of all is, like, interviews are just, like, proxy to evaluate somebody's skills, right? 00:34:31.580 --> 00:34:31.880 Yeah. 00:34:32.040 --> 00:34:39.260 So, you know, like, how even, like, so, like, this example, I know it's maybe, like, not very exact, but, like, first of all, I think about dating, right? 00:34:39.260 --> 00:34:45.300 You try to find somebody and it's just approximate whether that person is a good fit for you and you might go dating for, like, years. 00:34:45.300 --> 00:34:48.620 And you still end up with, like, some bad partner, if possible, right? 00:34:48.620 --> 00:34:49.700 So, like... 00:34:49.700 --> 00:34:50.940 Exactly. 00:34:50.940 --> 00:34:53.300 The divorce rate's, like, 50% or something, right? 00:34:53.300 --> 00:34:55.960 Like, we're not totally getting this nailed. 00:34:55.960 --> 00:34:56.460 Yeah. 00:34:56.460 --> 00:35:01.600 So, I think, like, for job interviews, like, you try to, like, admittedly, like, the stake is lower. 00:35:01.600 --> 00:35:03.240 It's, like, for a job, not for a partner. 00:35:03.240 --> 00:35:05.200 But you still have much less time, right? 00:35:05.200 --> 00:35:06.480 Like, you only have, like, a resume. 00:35:06.480 --> 00:35:09.980 Everyone say you shouldn't keep the resume longer than one page. 00:35:09.980 --> 00:35:10.860 You have one page of that. 00:35:10.860 --> 00:35:13.600 And then you maybe go on LinkedIn, social media, local things. 00:35:13.600 --> 00:35:14.920 And then you have, like, a few hours. 00:35:14.920 --> 00:35:18.420 Like, it's really hard to, like, get a good picture from it. 00:35:18.420 --> 00:35:21.740 And a lot of it's, like, biases, like, because interviewers are humans. 00:35:21.740 --> 00:35:27.100 And even though we try not to, like, we learn, we are taught that you shouldn't let biases, 00:35:27.100 --> 00:35:29.500 you shouldn't decide, like, judge people beyond that. 00:35:29.500 --> 00:35:32.420 But something we, like, we grew it. 00:35:32.420 --> 00:35:33.440 Like, something is, like... 00:35:33.440 --> 00:35:33.640 Yeah. 00:35:33.640 --> 00:35:37.120 We just do it without even, like, being conscious of doing so. 00:35:37.120 --> 00:35:41.420 And also, like, it's very different for different people because something that might work for 00:35:41.420 --> 00:35:44.380 a group of people might not work for as a group of people. 00:35:45.020 --> 00:35:49.560 So I think, like, for example, like, we have been trying to debate on take-home challenges. 00:35:49.560 --> 00:35:53.760 So a lot of candidates told us, like, oh, my God, interviews, like, so stressful. 00:35:53.760 --> 00:35:55.200 Like, one-on-one is really hard. 00:35:55.200 --> 00:35:56.780 Why don't you just give a take-home challenge? 00:35:56.780 --> 00:35:58.980 Like, just make it, like, I don't know, make it hard. 00:35:58.980 --> 00:36:00.560 We're going to spend, like, a day on it. 00:36:00.560 --> 00:36:02.860 When it be done, and you can see how good a way we are. 00:36:02.860 --> 00:36:05.120 But then, like, we thought about it, and we talked to people. 00:36:05.120 --> 00:36:10.040 And then we realized, like, for people who have a lot of responsibilities outside of work, 00:36:10.040 --> 00:36:13.100 like, especially, like, for example, women or, like, people with small kids, 00:36:13.340 --> 00:36:15.560 they can't spend a day, like, do or take-home challenges. 00:36:15.560 --> 00:36:18.020 So I think, like, it's what might work for me. 00:36:18.020 --> 00:36:18.440 Yeah. 00:36:18.440 --> 00:36:22.820 And if they apply to 100 jobs, then all of a sudden that's half a year or something like that, right? 00:36:22.820 --> 00:36:23.680 Yeah. 00:36:23.680 --> 00:36:24.400 Yeah. 00:36:24.400 --> 00:36:25.460 So it's very hard. 00:36:25.460 --> 00:36:29.680 So some companies, some people told me that, oh, they like this concept, like that company, 00:36:29.680 --> 00:36:33.800 when they bring you on to, like, as an intern-ish for, like, a month, they pay you. 00:36:34.120 --> 00:36:36.480 And then if you do well, then you can get a job. 00:36:36.480 --> 00:36:40.340 And somebody say, oh, that's great, because now everyone gets a chance to, like, show how good they are at the job. 00:36:40.340 --> 00:36:46.120 But then not everyone can afford to, like, just go on a job without any commitment for, like, for a month, right? 00:36:46.120 --> 00:36:48.560 And it's going to be totally excluded on immigrants. 00:36:48.560 --> 00:36:52.320 Like, for example, if somebody needs visa sponsorships, they can't just go and work for it. 00:36:52.500 --> 00:36:52.660 Right. 00:36:52.660 --> 00:36:59.420 It's a very precarious situation if your presence in the country is based on, you know, you have to have a job. 00:36:59.420 --> 00:37:03.120 And if it lapses for more than a month or two, then you've got to leave. 00:37:03.120 --> 00:37:03.920 That's really stressful. 00:37:03.920 --> 00:37:04.180 Yeah. 00:37:04.180 --> 00:37:04.620 Yeah. 00:37:04.620 --> 00:37:07.440 So I think it's really hard to find something that can work for everyone. 00:37:07.440 --> 00:37:08.120 Yeah. 00:37:08.120 --> 00:37:09.480 So a couple of thoughts. 00:37:09.480 --> 00:37:12.480 One, it's been a very long time since I hired anybody. 00:37:12.480 --> 00:37:18.360 But I used to help with hiring people to do training, people who would become trainers to teach, you know, 00:37:18.360 --> 00:37:21.100 basically for professional development for software developers. 00:37:21.500 --> 00:37:24.520 And we would obviously go through the resumes and see if they made any sense. 00:37:24.520 --> 00:37:27.800 And then we would just do a quick, like, 30-minute call. 00:37:27.800 --> 00:37:33.340 And I would say, okay, so imagine this person says, I'm a Python expert and I specialize in Flask. 00:37:33.340 --> 00:37:34.860 All right, we're on a Zoom call. 00:37:34.860 --> 00:37:35.980 Share your screen. 00:37:35.980 --> 00:37:39.900 Build me a Flask app that has one function that returns JSON. 00:37:39.900 --> 00:37:42.360 If I give it two numbers, it adds them. 00:37:42.360 --> 00:37:47.240 I mean, like, anybody who's ever worked with Flask should be able to knock that out in five minutes. 00:37:47.240 --> 00:37:52.000 And you can tell from, like, one minute in, is that person on that path to, like, get there? 00:37:52.000 --> 00:37:55.820 Because they know you start with import Flask and then you create app equals Flask. 00:37:55.820 --> 00:37:58.560 Or are they just flailing about, right? 00:37:58.560 --> 00:37:59.840 They just have no idea. 00:37:59.840 --> 00:38:07.300 And there's no way that they can both be an expert in Flask and not be able to create, like, a Hello World app in it, right? 00:38:07.540 --> 00:38:15.640 And so, I mean, that was the first sort of filter we used before we actually would ask them the equivalent of, like, here's a take-home project or something. 00:38:15.640 --> 00:38:22.160 It's just like, show me live that you're semi-confident with the tools you claim to be, like, top-notch in, right? 00:38:22.160 --> 00:38:24.120 And that actually worked pretty well, I think. 00:38:24.120 --> 00:38:30.580 I was blown away at how many people would claim to be, like, I've done five years of this and I'm a super expert and I'm ready to teach it to other people. 00:38:30.660 --> 00:38:32.940 And then they can't even begin to touch it. 00:38:32.940 --> 00:38:35.360 So, I think that's an interesting interview approach. 00:38:35.360 --> 00:38:42.980 But I also find it's, like, for, if we do interviews, it's very, like, tailored, like, specific, like, overfit to a specific tool. 00:38:42.980 --> 00:38:48.620 Then we might find people who are really good at one tool, but then don't really just, like, scale, like, right at a range of scale. 00:38:48.620 --> 00:38:57.460 And I think that for startups, if, like, if a fast-changing company where you need to, like, have a lot of, like, new problems and have to keep, yeah, we will have to, like, keep learning new things. 00:38:57.560 --> 00:39:02.060 If you just, like, get someone who's really good at, like, one thing and they can't generalize the other. 00:39:02.060 --> 00:39:02.560 Sure. 00:39:02.560 --> 00:39:09.060 What I was trying to more find, what we were trying to discern was they said they were expert at this thing. 00:39:09.060 --> 00:39:09.580 Yeah. 00:39:09.580 --> 00:39:12.120 Are they actually, like, how much can you trust? 00:39:12.120 --> 00:39:16.500 So, if they can show they're expert at this thing they said, then probably the other stuff that they said they're pretty good at. 00:39:16.500 --> 00:39:17.920 They're probably also in that realm. 00:39:17.920 --> 00:39:18.300 Yeah. 00:39:18.300 --> 00:39:25.640 But if they're, like, really far from, like, how they describe themselves in one axis, then they're probably not really going to be in a good fit. 00:39:25.640 --> 00:39:26.220 So, yeah. 00:39:26.220 --> 00:39:26.740 I don't know. 00:39:26.740 --> 00:39:27.400 It worked okay. 00:39:27.480 --> 00:39:28.540 We didn't do that much hiring. 00:39:28.540 --> 00:39:32.540 This portion of Talk Python To Me is sponsored by Linode. 00:39:32.540 --> 00:39:36.880 Simplify your infrastructure and cut your cloud bills in half with Linode's Linux virtual machines. 00:39:36.880 --> 00:39:40.940 Develop, deploy, and scale your modern applications faster and easier. 00:39:40.940 --> 00:39:48.360 Whether you're developing a personal project or managing large workloads, you deserve simple, affordable, and accessible cloud computing solutions. 00:39:48.360 --> 00:39:52.600 As listeners of Talk Python To Me, you'll get a $100 free credit. 00:39:52.600 --> 00:39:56.740 You can find all the details at talkpython.fm/Linode. 00:39:57.400 --> 00:40:02.460 Linode has data centers around the world with the same simple and consistent pricing regardless of location. 00:40:02.460 --> 00:40:05.320 Just choose the data center that's nearest to your users. 00:40:05.320 --> 00:40:12.020 You'll also receive 24-7, 365 human support with no tiers or handoffs regardless of your plan size. 00:40:12.460 --> 00:40:22.240 You can choose shared and dedicated compute instances or you can use your $100 in credit on S3 compatible object storage, managed Kubernetes clusters, and more. 00:40:22.240 --> 00:40:24.720 If it runs on Linux, it runs on Linode. 00:40:24.720 --> 00:40:29.400 Visit talkpython.fm/Linode or click the link in your show notes. 00:40:29.520 --> 00:40:32.100 Then click that create free account button to get started. 00:40:32.100 --> 00:40:37.780 The other thing that I wanted to bring up is, did you hear that Guido Van Rossum just joined Microsoft? 00:40:37.780 --> 00:40:39.200 Oh my God, yes. 00:40:39.200 --> 00:40:40.580 I think, yes. 00:40:40.580 --> 00:40:43.060 Yeah, I was like, it was really interesting. 00:40:43.060 --> 00:40:44.200 What was the thought on it? 00:40:44.360 --> 00:40:47.580 So he said, basically, he's been retired for six months. 00:40:47.580 --> 00:40:48.780 He's like, I'm really bored with this. 00:40:48.780 --> 00:40:49.840 I want to go back to do something. 00:40:49.840 --> 00:40:52.720 There's a ton of cool open source stuff going on there now. 00:40:52.720 --> 00:40:58.940 And, you know, he gets to work with some of the other language teams and make Python, basically just focus on Python and be around it. 00:40:58.940 --> 00:40:59.300 Yeah. 00:40:59.580 --> 00:41:00.480 And that's all interesting. 00:41:00.480 --> 00:41:02.820 And I think it's actually kind of a big deal that that happened. 00:41:02.820 --> 00:41:07.240 And it's like a really big contrast from Microsoft 10 years ago that this is even possible. 00:41:07.240 --> 00:41:16.760 But the thing that I want to bring up specifically now is somebody on Twitter asked him, so did you actually have to send in a resume, Guido, before they hired you? 00:41:16.760 --> 00:41:18.620 And he said, yes. 00:41:18.620 --> 00:41:18.940 Yeah. 00:41:18.940 --> 00:41:20.460 He had to send in a resume. 00:41:20.460 --> 00:41:22.100 He went through a bunch of interviews. 00:41:22.100 --> 00:41:28.240 The interviews make sense, but he had to send in his resume and he had to provide his degree he got in university. 00:41:28.660 --> 00:41:31.940 And like his transcript, like his grades and stuff he got in college. 00:41:31.940 --> 00:41:33.200 That's what I don't get. 00:41:33.200 --> 00:41:40.300 And I'm just thinking like, who cares if he got an F in literature or didn't? 00:41:40.300 --> 00:41:42.860 Like, look what he's accomplished since then. 00:41:42.860 --> 00:41:43.500 It doesn't matter. 00:41:43.500 --> 00:41:45.760 But that's just another one of these hiring things, right? 00:41:45.760 --> 00:41:46.820 Well, we got to check the box. 00:41:46.820 --> 00:41:49.040 We need his like university degree in transcript. 00:41:49.040 --> 00:41:50.400 This is so funny. 00:41:50.400 --> 00:41:51.760 Or this technical fellow. 00:41:51.760 --> 00:41:54.220 Does that remind me of like a few years ago? 00:41:54.220 --> 00:41:55.060 Do you know Malala? 00:41:55.400 --> 00:41:59.820 She was like the youngest Nobel recipient for Nobel in Peace? 00:41:59.820 --> 00:42:00.820 No, no, Malala? 00:42:00.820 --> 00:42:01.540 Yes. 00:42:01.540 --> 00:42:01.980 Yes, I do. 00:42:01.980 --> 00:42:02.220 Uh-huh. 00:42:02.220 --> 00:42:02.620 Yeah. 00:42:02.620 --> 00:42:06.060 It's really funny because like at the time she was like, oh, she wanted to study at Stanford. 00:42:06.060 --> 00:42:09.140 And like Stanford was like, yes, but what's her SAT score? 00:42:09.140 --> 00:42:13.540 And everyone was like, she's like youngest recipient in Nobel Prize for Peace. 00:42:13.540 --> 00:42:15.840 And you're asking her like, what's her SAT score? 00:42:16.240 --> 00:42:17.500 I thought it was just like, yeah. 00:42:17.500 --> 00:42:21.500 Just going to cram them through the bureaucratic pipeline. 00:42:21.500 --> 00:42:22.460 It's so funny. 00:42:22.460 --> 00:42:22.720 Yeah. 00:42:22.720 --> 00:42:23.520 All right. 00:42:23.520 --> 00:42:27.100 So what are some of the other takeaways that you're like hoping to give in this book? 00:42:27.100 --> 00:42:32.100 And you also have a chapter that's open on GitHub People Can Download, right? 00:42:32.100 --> 00:42:32.540 Yeah. 00:42:32.540 --> 00:42:33.760 So this is the chapter. 00:42:33.760 --> 00:42:38.760 So I think one part of the interview a lot of people ask is a machine learning system design. 00:42:39.120 --> 00:42:46.640 And so the question is usually like, yeah, like if you want to like build a system to do that, how would you do it? 00:42:46.640 --> 00:42:49.640 So it's very design high level kind of questions. 00:42:49.640 --> 00:43:00.340 And I think, so first of all, one question could be like, if you try to build a system to predict what keyword is trending on Twitter, then what would you go about it? 00:43:00.340 --> 00:43:02.460 Like what is considered trending and blah, blah. 00:43:02.460 --> 00:43:12.340 So I think this question is very interesting because it's usually like try to measure the understanding of like the different part of the system and not just like machine learning. 00:43:12.340 --> 00:43:22.720 But also find that questions can be like pretty, very hard for especially junior candidates because they don't have a good graph of like what is a production environment. 00:43:22.720 --> 00:43:23.920 So some companies. 00:43:23.920 --> 00:43:24.360 Yeah. 00:43:24.360 --> 00:43:30.020 A lot of times you have to see examples of that or have built examples of that to know like, well, these are the five pieces we got to put together. 00:43:30.020 --> 00:43:31.520 And then you do it, right? 00:43:31.520 --> 00:43:31.920 Yeah. 00:43:31.920 --> 00:43:43.660 So originally I wrote it as part of the interviews book, but then as I start writing more about it and I learning more about it, it was like, oh my God, there is like so much more in machine learning system design. 00:43:43.660 --> 00:43:47.180 So now it's actually become like a full blown book on its own. 00:43:47.180 --> 00:43:49.200 So that's why it's taking me longer. 00:43:49.200 --> 00:43:53.920 And I'm actually like teaching you a course on it, like machine learning system design, just on that part. 00:43:53.920 --> 00:43:54.700 Oh, that's cool. 00:43:54.700 --> 00:43:56.400 And you're teaching that in January. 00:43:56.400 --> 00:43:57.020 Is that right? 00:43:57.020 --> 00:43:57.360 At Stanford? 00:43:57.360 --> 00:43:57.900 Yes. 00:43:57.900 --> 00:44:00.020 And I'll be in January at Stanford. 00:44:00.020 --> 00:44:00.460 Yeah. 00:44:00.460 --> 00:44:05.180 It's a bit strange because I'm not sure how teaching online is going to go. 00:44:05.180 --> 00:44:07.540 I'm a bit, a little bit nervous about that. 00:44:07.540 --> 00:44:08.000 Yeah. 00:44:08.000 --> 00:44:11.180 It's not the same as standing in front of the class and having that experience. 00:44:11.180 --> 00:44:11.740 That's for sure. 00:44:11.740 --> 00:44:12.160 Yeah. 00:44:12.160 --> 00:44:18.240 But most people told me that it's a different experience because some students like it more because especially for the introvert. 00:44:18.420 --> 00:44:24.340 Now they can just like ask questions anonymously without having to raise their hands and having anyone stare at them. 00:44:24.340 --> 00:44:25.540 So it's going to be interesting. 00:44:25.540 --> 00:44:26.700 I'm looking forward to it. 00:44:26.700 --> 00:44:27.140 Yeah. 00:44:27.140 --> 00:44:28.300 It should definitely be interesting. 00:44:28.300 --> 00:44:28.960 All right. 00:44:28.960 --> 00:44:36.520 I think we're just about out of time, but maybe just real quickly, you could give us the elevator pitch on Storkel, Snorkel AI and what you guys got going on there. 00:44:37.020 --> 00:44:37.460 Ooh. 00:44:37.460 --> 00:44:38.560 Okay. 00:44:38.560 --> 00:44:43.240 So I think like for the pitch, I think you can just say why I decided to join Snorkel. 00:44:43.240 --> 00:44:49.240 So it's funny because it's a startup that comes out from Stanford AI lab and I have heard of them for a while. 00:44:49.240 --> 00:44:53.360 And when it first approached me, I was like, oh my God, another startup from Stanford. 00:44:53.360 --> 00:44:57.100 I know it sounds super smart, but I was like, oh, startup AI, whatever. 00:44:57.100 --> 00:44:59.880 But then I came across the paper. 00:44:59.880 --> 00:45:05.620 So most of the founding teams are like PhD students and they have been publishing a lot. 00:45:05.620 --> 00:45:09.420 I read one of their papers and I was like, this is really smart. 00:45:09.420 --> 00:45:15.640 So the key idea for their paper was that like, instead of manually label on the data, right? 00:45:15.640 --> 00:45:22.320 You can have some heuristics and causal heuristics into programming functions and apply to all the data at once. 00:45:22.320 --> 00:45:30.800 That's the really hard thing about training your models is you get like all the state that you have to say car, bicycle, ball, tree, right? 00:45:30.800 --> 00:45:32.980 And you just got to like go through it and teach it basically. 00:45:32.980 --> 00:45:33.260 Yeah. 00:45:33.260 --> 00:45:35.480 So you notice a helpful like labels. 00:45:35.480 --> 00:45:38.620 Like for example, you see like an email with a spam or not spam, right? 00:45:38.680 --> 00:45:40.520 You probably notice you probably have some heuristics. 00:45:40.520 --> 00:45:48.480 Like, hey, if you say like, hey, you're going to have like, hey, please send me money to like Nigerian Prince or something like you're going to spam. 00:45:48.480 --> 00:45:50.260 So, so you have some like heuristics in the brain. 00:45:50.260 --> 00:45:55.300 Like, so if you can find what you end cause of heuristics and you don't have to manually do it on at once. 00:45:55.300 --> 00:45:56.900 So I think that's the algorithm. 00:45:56.900 --> 00:46:02.540 So like how should I combine because some heuristics are going to be noisy and like overlapping and they can't see each other. 00:46:02.540 --> 00:46:08.420 So the current algorithm was like how should I combine one of them and I generate like what the techniques of. 00:46:08.420 --> 00:46:12.880 most likely to be correct gradsures because you don't have gradsures actually compare gradsures. 00:46:12.880 --> 00:46:19.020 So then you generate the set of gradsures and then you, and so they, they open source support. 00:46:19.020 --> 00:46:21.400 So like anyone can just go on GitHub and use it. 00:46:21.400 --> 00:46:25.240 So I went to the core and thought like, wow, these people are like good engineers. 00:46:25.240 --> 00:46:31.000 Because you think of like PhD students are like bad engineers, but then they, their core is like good, very clean. 00:46:31.000 --> 00:46:32.600 They have more testing and everything. 00:46:32.600 --> 00:46:33.740 Like unit tests. 00:46:33.740 --> 00:46:34.320 Oh my God. 00:46:34.320 --> 00:46:34.780 No. 00:46:34.780 --> 00:46:38.280 So, no, no. 00:46:38.280 --> 00:46:48.940 So, so I think the product now is a, it's not just a part because actually a lot of thing of snorkels is thing of that labeling part, but we actually be like a full non like a end to end platform. 00:46:49.120 --> 00:46:52.880 So we have your format data to like modeling training. 00:46:52.880 --> 00:47:02.540 Like we do a lot with like monitoring analysis because we believe that you can't, machine learning is because it's changing fast and each updates on models constantly. 00:47:02.800 --> 00:47:04.660 And so we believe in iterative development. 00:47:04.660 --> 00:47:08.280 So like you have a, you, you train model and you see it and it's not good. 00:47:08.280 --> 00:47:10.700 So you go back and see what's wrong and how do you improve it. 00:47:10.700 --> 00:47:11.860 And like you manage more data. 00:47:11.860 --> 00:47:14.460 So, so we version everything as a process, by the way. 00:47:14.460 --> 00:47:15.360 And so. 00:47:15.360 --> 00:47:15.920 That's cool. 00:47:15.920 --> 00:47:17.760 It's like agile data. 00:47:17.760 --> 00:47:23.300 We don't use agile yet, but if it's one of the buzzwords of sales and maybe we can adopt it. 00:47:23.300 --> 00:47:24.260 Yeah, I know. 00:47:24.260 --> 00:47:24.700 I'm just teasing. 00:47:24.700 --> 00:47:25.660 It's one of my buzzwords. 00:47:25.660 --> 00:47:25.960 I'm just kidding. 00:47:25.960 --> 00:47:27.060 Oh my God. 00:47:27.060 --> 00:47:29.240 Please, somebody from snorkel, please don't fire me. 00:47:29.240 --> 00:47:37.760 But, but yeah, so, so we do a lot of, so it's an end to end platform for people to build machine, AI applications. 00:47:37.760 --> 00:47:41.100 And it goes on data model, monitoring analysis. 00:47:41.100 --> 00:47:42.620 And I think it's pretty dope. 00:47:42.620 --> 00:47:44.340 You guys should totally check it out. 00:47:44.340 --> 00:47:44.800 Yeah. 00:47:44.800 --> 00:47:45.340 Right on. 00:47:45.340 --> 00:47:45.600 Awesome. 00:47:45.600 --> 00:47:51.880 Well, it sounds like a cool company to work for and definitely nice applied machine learning stuff. 00:47:51.880 --> 00:47:54.200 So building tools for machine learning folks, right? 00:47:54.200 --> 00:47:54.560 Yeah. 00:47:54.560 --> 00:47:55.140 Awesome. 00:47:55.140 --> 00:47:57.400 I think it's for machine learning folks, but I think like. 00:47:57.400 --> 00:47:57.640 All right. 00:47:57.720 --> 00:48:01.900 We recently wanted to lower the entry barriers for people to build AI applications. 00:48:01.900 --> 00:48:05.340 So I think like, so our platform is actually no code. 00:48:05.340 --> 00:48:09.160 So like you have the option to just build an application without any code at all. 00:48:09.160 --> 00:48:10.960 But we also have like our SDK. 00:48:10.960 --> 00:48:15.300 So like for people who want more like flexibility, then you can also like code. 00:48:15.300 --> 00:48:15.720 Yeah. 00:48:15.720 --> 00:48:16.360 Very nice. 00:48:16.360 --> 00:48:16.800 All right. 00:48:16.800 --> 00:48:19.000 Well, good luck with the whole company and the startup. 00:48:19.000 --> 00:48:20.440 Hopefully it takes off and does well. 00:48:20.440 --> 00:48:21.060 It sounds nice. 00:48:21.060 --> 00:48:21.400 Yeah. 00:48:21.400 --> 00:48:21.960 Yeah. 00:48:21.960 --> 00:48:22.420 Thank you. 00:48:22.420 --> 00:48:23.960 So we're pretty much out of time, but yeah. 00:48:24.020 --> 00:48:28.580 Thanks for all the advice on building machine learning teams or getting to be part of one. 00:48:28.580 --> 00:48:32.340 Now, before you go, there's always the two questions I ask at the end of the show. 00:48:32.340 --> 00:48:37.200 And one is if you're going to write some code, some Python code, what editor would you use these days? 00:48:37.200 --> 00:48:43.400 So sometimes I really want to be smart and say it's like I use Vim, but actually just use VS Code. 00:48:43.400 --> 00:48:47.180 VS Code is definitely the most popular answer these days. 00:48:47.180 --> 00:48:48.060 It's all good. 00:48:48.680 --> 00:48:54.920 And notable PyPI package, like something, some Python library or package that you've come across like, oh, this was so cool. 00:48:54.920 --> 00:48:56.580 People should know about X. 00:48:56.580 --> 00:48:57.400 I'm not sure. 00:48:57.400 --> 00:49:00.820 Is this, so, so, so, do you know about, I think it's like Paper Mill. 00:49:00.820 --> 00:49:02.800 So it just allows you to format. 00:49:02.800 --> 00:49:03.660 Yeah. 00:49:03.660 --> 00:49:04.720 I think it's pretty cool. 00:49:04.720 --> 00:49:07.480 It allows you to do a lot of experiments with like Jupyter Notebooks. 00:49:07.480 --> 00:49:08.140 I think. 00:49:08.140 --> 00:49:08.760 Yeah. 00:49:08.760 --> 00:49:10.280 And Paper Mill comes out in Netflix. 00:49:10.280 --> 00:49:11.800 So it's a neighbor of yours. 00:49:11.800 --> 00:49:12.320 And. 00:49:12.320 --> 00:49:12.860 Nice. 00:49:13.120 --> 00:49:13.360 Yeah. 00:49:13.360 --> 00:49:16.300 The idea is you can almost treat Notebooks like functions, right? 00:49:16.300 --> 00:49:20.920 Like you can pass arguments to them, run them and get like something out and then even chain them together. 00:49:20.920 --> 00:49:21.520 Yeah. 00:49:21.520 --> 00:49:27.880 And one of the things I heard was really nice about it is if you create sort of data pipelines of one notebook going to the next, the next. 00:49:27.880 --> 00:49:34.320 And if something goes wrong, like the notebook actually contains like all the data that came in and what it tried to do and how far it got. 00:49:34.320 --> 00:49:34.800 Yeah. 00:49:34.800 --> 00:49:38.100 And it's like almost a record instead of just like server failed with 500. 00:49:38.100 --> 00:49:40.080 Like, no, like here's all the details. 00:49:40.080 --> 00:49:41.060 You can go back and look at it. 00:49:41.060 --> 00:49:41.260 Yeah. 00:49:41.260 --> 00:49:41.980 It's pretty dope. 00:49:41.980 --> 00:49:43.100 I think, I think it's really cool. 00:49:43.100 --> 00:49:47.020 I think there's been like so many exciting work in the notebook space. 00:49:47.020 --> 00:49:48.920 I think it's a real like Streamlit. 00:49:48.920 --> 00:49:51.380 I think Showlit is like, it's like really cool. 00:49:51.380 --> 00:49:53.800 Like you create like very quick applications. 00:49:53.800 --> 00:49:55.340 I mean, there's just so many. 00:49:55.340 --> 00:49:55.840 Yeah. 00:49:55.840 --> 00:49:56.780 That's really nice as well. 00:49:56.780 --> 00:49:57.000 Yeah. 00:49:57.000 --> 00:49:59.160 What's your, which is your favorite? 00:49:59.160 --> 00:50:00.080 My favorite. 00:50:00.080 --> 00:50:00.820 Oh my gosh. 00:50:01.240 --> 00:50:04.880 You know, there's all these different ones that always blow me away. 00:50:04.880 --> 00:50:06.380 I go through so many of them. 00:50:06.380 --> 00:50:09.720 One that I came across recently that was pretty neat is called a back off. 00:50:09.720 --> 00:50:10.940 Someone told me about that. 00:50:10.940 --> 00:50:13.600 And back off what you do is just put a decorator on one of your functions. 00:50:13.600 --> 00:50:16.560 You say, if I get this kind of error, like this type of exception. 00:50:16.560 --> 00:50:17.080 Yeah. 00:50:17.080 --> 00:50:20.780 Like wait five seconds and then try again and then wait 10 seconds and then try again. 00:50:20.780 --> 00:50:26.400 So if you're doing like testing against like an API and you get like a too many requests error, 00:50:26.400 --> 00:50:29.520 you can say instead of fail the test, just wait one second and try again. 00:50:29.520 --> 00:50:31.520 This sounds pretty dope. 00:50:31.520 --> 00:50:33.000 I think I need to check it out. 00:50:33.000 --> 00:50:33.520 Yeah. 00:50:33.520 --> 00:50:34.060 Yeah. 00:50:34.060 --> 00:50:34.300 Yeah. 00:50:34.300 --> 00:50:38.560 It's super easy to use, but it kind of solves that problem of like mostly reliable, but not 00:50:38.560 --> 00:50:39.760 all the time reliable stuff. 00:50:39.760 --> 00:50:43.520 Do you like stars things on GitHub when you see repos that you like? 00:50:43.520 --> 00:50:44.140 I do. 00:50:44.140 --> 00:50:44.720 Yeah. 00:50:44.720 --> 00:50:46.120 I star stuff all the time. 00:50:46.120 --> 00:50:46.380 Yeah. 00:50:46.380 --> 00:50:50.480 Can I just go into those, can I see you go to the star list and let's see like what have 00:50:50.480 --> 00:50:51.800 you been like looking at? 00:50:51.800 --> 00:50:52.260 Yeah. 00:50:52.260 --> 00:50:54.380 So github.com/Mike C. Kennedy. 00:50:54.380 --> 00:50:59.680 And let's see, I'll pull up my stars and see where are the things that I've starred? 00:50:59.680 --> 00:51:00.600 There we go. 00:51:00.600 --> 00:51:04.380 So the things that I have up here right now, that's a really good question, by the way, 00:51:04.380 --> 00:51:05.560 like really cool way to look at it. 00:51:05.560 --> 00:51:08.600 So I have pip chill, which is like pip. 00:51:08.600 --> 00:51:11.380 You know, if you do pip freeze, it'll show you what you've installed. 00:51:11.380 --> 00:51:15.580 Pip chill will like pip freeze will include everything that was installed, including the 00:51:15.580 --> 00:51:15.980 dependencies. 00:51:15.980 --> 00:51:21.500 Pip chill will just show you just what you manually installed, not the dependencies, which is cool. 00:51:21.660 --> 00:51:22.100 Nice. 00:51:22.100 --> 00:51:28.460 Then link it, L I N Q I T adds like link functionality to the Python language is cool. 00:51:28.460 --> 00:51:31.140 I love the name, by the way, pip chill. 00:51:31.140 --> 00:51:33.960 Pip chill, yeah, it's so good. 00:51:33.960 --> 00:51:40.900 And then I have a FastAPI chameleon and FastAPI Jinja, which adds like those templating languages 00:51:40.900 --> 00:51:42.640 to FastAPI as a decorator. 00:51:42.640 --> 00:51:44.820 Oh, you saw MB black is cool. 00:51:44.820 --> 00:51:45.380 Yeah. 00:51:45.380 --> 00:51:45.880 Yeah. 00:51:45.880 --> 00:51:47.540 MB black adds black to notebooks. 00:51:47.540 --> 00:51:47.760 Yeah. 00:51:47.760 --> 00:51:49.880 So those are the ones I've starred recently, I guess. 00:51:49.880 --> 00:51:50.780 I'm so funny. 00:51:50.780 --> 00:51:51.960 So those are all good. 00:51:51.960 --> 00:51:55.460 So you saw like a lot of FastAPI, but still still a lot of flask. 00:51:55.460 --> 00:51:58.040 Do you have like, you prefer one of another? 00:51:58.040 --> 00:51:58.900 Yeah, I do. 00:51:58.900 --> 00:52:00.480 I really like FastAPI. 00:52:00.480 --> 00:52:01.760 I've been liking it a lot. 00:52:01.760 --> 00:52:02.820 Oh, it's brilliant. 00:52:02.820 --> 00:52:03.400 Yeah. 00:52:03.400 --> 00:52:05.440 I think it's, yeah, it's so brilliant. 00:52:05.440 --> 00:52:08.020 It takes all the cool modern features of Python and puts it together. 00:52:08.020 --> 00:52:10.400 So let me make one recommendation for you. 00:52:10.400 --> 00:52:11.420 Check this out. 00:52:11.420 --> 00:52:15.240 I just want to get your reaction to this for people as a data science machine learning 00:52:15.240 --> 00:52:15.600 person. 00:52:15.600 --> 00:52:16.460 Hand calcs. 00:52:16.460 --> 00:52:17.220 What is that? 00:52:17.220 --> 00:52:19.020 Have you, have you seen hand calcs? 00:52:19.020 --> 00:52:19.320 No. 00:52:19.320 --> 00:52:21.000 So hand calcs is crazy. 00:52:21.000 --> 00:52:23.540 So what this does is you write, you create a Jupyter notebook. 00:52:23.540 --> 00:52:24.080 Yeah. 00:52:24.080 --> 00:52:28.500 And you write some sort of math equation that is actually just the computation. 00:52:28.500 --> 00:52:32.880 And then you can ask it to show you, and it'll show you as if it wrote it in LaTeX. 00:52:32.880 --> 00:52:33.260 What? 00:52:33.460 --> 00:52:37.060 Like step by step, like how, yeah, like how it solved out the problem. 00:52:37.060 --> 00:52:38.640 So it'd have like, like the nice square root. 00:52:38.640 --> 00:52:43.240 So if you're doing some kind of like computation that's somewhat technical and hard. 00:52:43.240 --> 00:52:43.560 Yeah. 00:52:43.560 --> 00:52:48.860 In Jupyter, it'll actually show you like what you would put into like a math textbook or a 00:52:48.860 --> 00:52:52.880 physics textbook to derive the equations and even the steps you might take to go from what? 00:52:52.880 --> 00:52:54.040 Can it do proof for you? 00:52:54.040 --> 00:52:59.020 I don't know how far it can go with a proof, but if you just go to like Google hand calcs. 00:52:59.020 --> 00:52:59.980 Wait, how do you spell it? 00:52:59.980 --> 00:53:01.640 There's a bunch of animated GIFs. 00:53:01.640 --> 00:53:02.900 How can you say the name? 00:53:03.220 --> 00:53:05.220 H-A-N-D, C-A-L. 00:53:05.220 --> 00:53:06.240 H-A-N-D. 00:53:06.240 --> 00:53:07.120 C-S. 00:53:07.120 --> 00:53:07.380 Like. 00:53:07.380 --> 00:53:07.920 Oh. 00:53:07.920 --> 00:53:10.320 C-A-A-L-C-S. 00:53:10.320 --> 00:53:11.480 Like hand calculations. 00:53:11.480 --> 00:53:11.920 Okay. 00:53:11.920 --> 00:53:14.060 Is this, is this from corner first? 00:53:14.060 --> 00:53:14.940 Yes. 00:53:14.940 --> 00:53:16.100 Oh, that's dope. 00:53:16.100 --> 00:53:16.680 And. 00:53:16.680 --> 00:53:17.320 Oh yeah. 00:53:17.320 --> 00:53:20.440 If you just page down through it, you can see like all these amazing steps. 00:53:20.440 --> 00:53:24.780 You can like render like symbolic mathematics and like the steps between various things. 00:53:24.780 --> 00:53:24.960 Yeah. 00:53:24.960 --> 00:53:25.640 It's really, really. 00:53:25.640 --> 00:53:28.000 So if you were doing like complex calculations. 00:53:28.000 --> 00:53:28.860 Whoa. 00:53:28.860 --> 00:53:30.540 That you want to make sure you got right. 00:53:30.680 --> 00:53:36.520 Like reading the Python code to do it is harder than like reading the symbolic mathematics of it. 00:53:36.520 --> 00:53:37.700 Wait, how does it do this? 00:53:37.700 --> 00:53:38.220 Yes. 00:53:38.220 --> 00:53:38.960 I have no idea. 00:53:38.960 --> 00:53:43.580 But it uses like Sempy and a bunch of other LaTeX and all sorts of crazy stuff. 00:53:43.700 --> 00:53:47.500 So as a data scientist, like this thing is killer, I think. 00:53:47.500 --> 00:53:48.760 That is pretty dope. 00:53:48.760 --> 00:53:49.420 Nice. 00:53:49.420 --> 00:53:50.560 Thanks for showing me. 00:53:50.560 --> 00:53:51.720 I'm going to show it to my friends. 00:53:51.720 --> 00:53:52.300 Yeah. 00:53:52.300 --> 00:53:53.180 Yeah. 00:53:53.180 --> 00:53:53.560 There you go. 00:53:53.560 --> 00:53:56.260 So there's, there's a topical recommendation. 00:53:56.260 --> 00:53:56.760 How's that? 00:53:56.760 --> 00:53:57.280 Nice. 00:53:57.280 --> 00:53:58.060 That's helpful. 00:53:58.060 --> 00:53:58.740 Thank you. 00:53:58.740 --> 00:53:59.940 That's dope. 00:53:59.940 --> 00:54:00.980 Of course. 00:54:00.980 --> 00:54:01.500 Cool. 00:54:01.500 --> 00:54:02.340 All right, Chip. 00:54:02.540 --> 00:54:07.340 Well, I think we're about out of time, but I just want to say thank you for being on the show and sharing all of your advice. 00:54:07.340 --> 00:54:14.100 And I guess one final question, if people are interested, if they're out there looking to do some machine learning, are you guys hiring? 00:54:14.100 --> 00:54:14.820 Yes. 00:54:14.820 --> 00:54:18.160 We are hiring a lot, actually. 00:54:18.160 --> 00:54:21.400 That's actually one of our challenges. 00:54:21.620 --> 00:54:27.160 Like how to give on, yeah, hiring a large quality of like quantity of very good people. 00:54:27.160 --> 00:54:27.560 Yeah. 00:54:27.560 --> 00:54:31.800 That's, that is definitely a challenge, but we'll put a link maybe over to like the jobs page or something. 00:54:31.800 --> 00:54:33.380 If you want, people can check it out. 00:54:33.380 --> 00:54:37.960 This has been another episode of Talk Python To Me. 00:54:37.960 --> 00:54:42.860 Our guest in this episode has been Chip Hewn, and it's been brought to you by Datadog and Linode. 00:54:42.860 --> 00:54:46.920 Datadog gives you visibility into the whole system running your code. 00:54:46.920 --> 00:54:50.720 Visit talkpython.fm/datadog and see what you've been missing. 00:54:51.120 --> 00:54:52.900 We'll throw in a free t-shirt with your free trial. 00:54:52.900 --> 00:54:58.160 Simplify your infrastructure and cut your cloud bills in half with Linode's Linux virtual machines. 00:54:58.160 --> 00:55:01.540 Develop, deploy, and scale your modern applications faster and easier. 00:55:01.540 --> 00:55:06.500 Visit talkpython.fm/linode and click the create free account button to get started. 00:55:06.500 --> 00:55:08.460 Want to level up your Python? 00:55:08.460 --> 00:55:13.280 If you're just getting started, try my Python Jumpstart by building 10 apps course. 00:55:13.280 --> 00:55:18.840 Or if you're looking for something more advanced, check out our new async course that digs into all the 00:55:18.840 --> 00:55:21.420 different types of async programming you can do in Python. 00:55:21.420 --> 00:55:26.100 And of course, if you're interested in more than one of these, be sure to check out our Everything Bundle. 00:55:26.100 --> 00:55:27.980 It's like a subscription that never expires. 00:55:27.980 --> 00:55:30.140 Be sure to subscribe to the show. 00:55:30.140 --> 00:55:32.540 Open your favorite podcatcher and search for Python. 00:55:32.540 --> 00:55:33.760 We should be right at the top. 00:55:33.760 --> 00:55:38.600 You can also find the iTunes feed at /itunes, the Google Play feed at /play, 00:55:38.600 --> 00:55:42.760 and the direct RSS feed at /rss on talkpython.fm. 00:55:43.300 --> 00:55:44.840 This is your host, Michael Kennedy. 00:55:44.840 --> 00:55:46.340 Thanks so much for listening. 00:55:46.340 --> 00:55:47.380 I really appreciate it. 00:55:47.380 --> 00:55:49.160 Now get out there and write some Python code. 00:55:49.160 --> 00:55:49.540 Thank you. 00:55:49.540 --> 00:56:09.180 Thank you. 00:56:09.180 --> 00:56:39.160 Thank you.